Patent Application
20230223136
The invention relates to a system, method, a program element and a computer readable medium and for analysis of medical image data. Specifically, the analysis is performed so that it takes into account an interaction between a plurality of image quality metrics in their combined effect on a combined quality metrics.
In radiology departments, the functions of generating a radiograph and assessing its diagnostic quality are divided between the radiographer, also denoted as radiologic technologist, who operates the image acquisition unit on the one hand and the radiologist on the other hand. If an image shows severe quality degradation, for example caused by errors in positioning the patient relative to the imaging unit, the radiologist, when performing reject analysis, rejects the image classifying it as not being diagnostic. In such cases, it is necessary to acquire a repeat image from the patient. Repeat images, however, lead to additional costs, less productivity in the radiology department and higher radiation doses given to the patient. They also do not fit into a clinical workflow, which has to manage an ever-increasing amount of diagnostic data.
A similar problem exists in that some radiographers tend to directly reject images without consulting the radiologist in an effort to anticipate a possible rejection by the radiologist. This, however, leads to unnecessary repeat images, since the image was rejected by somebody who only has a basic medical training and is therefore not qualified to make a determination on whether or not the image is suitable for diagnosis.
Due to the deficiencies in current processes for generating medical images and assessing their diagnostic quality, it would be desirable to have a more efficient system for analyzing medical image data.
Document US 2010/0305441 A1 relates to a system for automatic ultrasound image optimization.
Document EP 3 644 273 A1 relates to a system for determining image quality parameters for medical images.
The present disclosure pertains to a system for analysis of medical image data, which represent a two-dimensional or three-dimensional medical image, which has been acquired from a human subject. The system comprises a data processing system, which is configured to read and/or determine, for the medical image, a plurality of image quality metrics. The data processing system is further configured to determine a combined quality metrics based on the image quality metrics. The data processing system is configured so that the determination of the combined quality metrics takes into account an interaction between the image quality metrics in their combined effect on the combined quality metrics.
The image data may be acquired using a medical imaging unit. The medical imaging unit may include an X-ray imaging unit and/or a magnetic resonance tomography unit and/or an ultrasonic imaging unit. The X-ray imaging unit may be configured for plane radiography and/or for X-ray computed tomography.
The acquired medical images of the human subjects may include images of a lung, a chest, a rib, bones, an ankle joint or a knee joint.
The three-dimensional medical images may represent tomographic images. The tomographic images may be generated from two-dimensional images using a reconstruction algorithm. The two-dimensional medical images may be generated using projection imaging, in particular using projectional radiography.
The data processing system may be configured to read at least a portion of the quality metrics from one or more files, which are stored in a storage device of the data processing system. Additionally or alternatively, the data processing system may be configured to determine at least a portion of the quality metrics using an analysis algorithm, which uses at least a portion of the medical image data as input. The analysis algorithm may include, but is not limited to, one or a combination of a segmentation algorithm, a registration algorithm for registering the image with an atlas, a classifier, a machine-learning algorithm. The machine-learning algorithm may be configured for supervised and/or unsupervised learning. Specifically, the machine-learning algorithm may be an artificial neural network (ANN).
Additionally or alternatively, the data processing system may be configured to receive at least a portion of the quality metrics via a user interface of the data storage system and/or from a further data processing system. The data processing system may determine the image quality metrics automatically or interactively (i.e. requiring user intervention).
One or more or each of the quality metrics may relate to a parameter of an imaging process for acquiring the medical image using the medical imaging unit. One or more or each of the quality metrics may be indicative of or may include a parameter of a position and/or an orientation of a body portion of the human subject relative to a component of the medical imaging unit. By way of example, the components may include a detector of the imaging unit, such as an X-ray detector and/or a radiation source of the imaging unit. One example of an orientation parameter is the rotation angle of an ankle about a longitudinal axis. One example of a position parameter is the position of the lung field relative to the field of view of an X-ray radiation sensitive detector. The field of view may be adjusted by adjusting a position of the collimator of the X-ray imaging unit and/or by adjusting a relative position between the subject and the radiation-sensitive detector. Additionally or alternatively, the extent of the lung field relative to the field of view can be adjusted by adjusting an opening formed by the collimator.
Additionally or alternatively, one or more or each of the quality metrics may be indicative of, or may include a parameter of an orientation of portions of the human subject relative to each other. For one or more or all of the body portions, the respective body portion may be imaged in the medical image. One example of an orientation of two body portions relative to each other is the flexion angle of an ankle (i.e. angle of dorsiflexion or plantarflexion).
The combined quality metric may be indicative of whether or not, or to which degree, the medical image is sufficient for a medical diagnosis. The data processing system may be configured to determine the combined quality metric using an algorithm stored in the data processing system. The determination of the combined quality metric may be performed automatically or interactively (i.e. requiring human intervention). By way of example, the data processing system may receive, via a user interface of the data processing system, user input, which is used for determining the combined quality metric. One example for such a user input is the intended diagnosis based on the medical image. Based on this user input, the data processing system may select one or more image quality metrics and/or an algorithm for determining the combined quality metric. Thereby, it is possible to determine a combined quality metric, which reliably allows determination whether the acquired image is suitable for making the intended diagnosis.
At least a portion of the combined quality metric may be a value of a plurality of predefined quality levels. One or more of the quality levels may indicate that the medical image meets a predefined criterion and one or more second levels may indicate that the medical image does not meet a predefined criterion. The predefined criterion may be indicative of whether or not, or to which degree the medical image is suitable for a predefined diagnosis.
The combined quality metric may be a tabulated function or an analytical function, which depends on the image quality metrics. The combined quality metric may include or may be a scalar value, a vector and/or a state variable. Combinations of values of the image quality metrics, which represent a same value of the combined quality metrics may represent a contour line in a height map, which is indicative of at least a portion of the combined quality metric.
According to a further embodiment, the interaction between the image quality metrics is so that a change of the combined quality metrics, which is caused by a change of a first one of the quality metrics is compensable by a change of one or more of the remaining quality metrics.
According to a further embodiment each of the image quality metrics is associated with a respective predefined optimum value or predefined optimum range. By way of example, one of the optimum values may be an optimum orientation of a body portion, in particular an imaged body portion relative to a component of the imaging unit, such as a detector and/or a source of the medical imaging unit. Additionally or alternatively, the optimum value may be an optimum orientation of body portions relative to each other. By way of example, the optimum angle may be an optimum angle of the ankle (i.e. angle of dorsiflexion or angle of plantarflexion).
The interaction between the image quality metrics may be so that an increased deviation from the optimum range or optimum value of a first one of the metrics may be compensable by a decreased deviation from the optimum range or optimum value of one or more of the remaining image quality metrics.
According to a further embodiment, the one or more or each of the quality metrics is indicative of one or more parameters of a position and/or one or more parameters of an orientation of an imaged body portion relative to a component of the imaging unit or relative to a further body portion. The component may be a detector and/or a source of the medical imaging unit. One or more or each of the body portions may be at least partially imaged in the medical image.
Further examples of quality metrics are but are not limited to: and inhalation status, a collimation parameter a field of view position parameter and a distance between the body portion and the component of the medical imaging unit.
By way of example, the inhalation state may be expressed using a number of ribs, which are shown in the image above the diaphragm. The analysis algorithm may be configured to detect the ribs, which are shown in the medical image above the diaphragm and determine the number of these ribs. Detection of the ribs may be performed using a segmentation algorithm.
Determination of the collimation parameter may include determining a distance between a border of a field of view of the image and an optimum collimation border that covers the target anatomy (such as the lung field) without under-collimation or over-collimation. The collimation parameter may be configured to allow detection of under-collimation and over-collimation in the medical images. Over-collimated images show only a portion of the target anatomy (such as the lung field) so that only a portion of the necessary information is available to perform a diagnosis. On the other hand, under-collimated images show body portions which do not need to be imaged to perform the diagnosis so that these body portions are exposed to radiation, which can be avoided through proper collimation.
The field of view position parameter may be a parameter of a position of the field of view of the image relative to an optimum field of view, which covers the target anatomy without under-collimation or over-collimation.
Other quality metrics may include but are not limited to: (a) a metric, which is indicative of whether or not predetermined anatomical structures, or a minimum amount of anatomical structures are visible in the image (e.g. whether one or both scapula are visible in the imaged lung field), and (b) a metric, which is indicative of a percentage of overlap between congruent anatomical structures (e.g. femoral condyles). Congruent anatomical structures may be defined as structures that are either symmetric or nearly identical. It may be advantageous, in particular for a two-dimensional projection image, to bring the congruent anatomical structures into overlap or substantial overlap with respect to their contours. This may increase the reliability of the diagnosis.
According to a further embodiment, the system is configured to determine, for one or more of the images, based on the determined combined quality metric, a parameter or state variable representing a change of imaging conditions for using the imaging unit. By way of example, the state variable represents an operation state of the medical imaging unit. Further, examples for the parameter representing the change of imaging conditions are but are not limited to: (a) an operation parameter of the medical imaging unit, (b) a parameter of a position and/or orientation of a body portion relative to a component of the medical imaging unit, and (c) a parameter of a position and/or orientation of two or more body portions relative to each other.
By way of example, the change in imaging conditions may be a change in a position and/or orientation of the body portion relative to the component of the imaging unit and/or a change in a position and/or orientation of body portions relative to each other. Additionally or alternatively, the change in imaging conditions is a change in operational parameters of the imaging unit, such as a position of a collimator, a tilt of the X-ray tube and/or X-ray sensitive detector, the position of an anti-scatter grid, or whether or not to use an anti-scatter grid, a geometrical configuration of a patient support device, such as the geometrical configuration of a flexion unit.
The changed imaging conditions may cause a change of one or more of the image quality metrics. In other words, the system may provide a recommendation for the user or for the medical imaging unit to adapt the imaging conditions so as to obtain a change in the combined quality metric.
According to a further embodiment, the determination of the image quality metrics includes segmenting at least a portion of the medical image. Additionally or alternatively, the determination comprises registering the medical image with an atlas. The registration of the image data with the atlas may be performed using the segmented image. The image segmentation may be configured to determine one or more image regions. The segmentation may be performed using the data processing system and/or using a further data analysis system from which the data processing system receives data.
The segmentation of the medical image may be performed using one or a combination of the following segmentation techniques: thresholding, region growing, Watershed transform, edge detection, shape model, appearance model and hand-segmentation using user interaction with the graphical user interface. Additionally or alternatively, the segmentation may be performed using an artificial neural network. The segmentation may be performed automatically or interactively (i.e. requiring user intervention). In interactive segmentations, the computer system may receive user input, which is indicative of one or more parameters of a location, an orientation and/or an outer contour of an image region. The artificial neural network may be trained using segmented images, in particular performed by interactive segmentation.
The atlas may include a statistically averaged anatomical map of one or more body portions. At least a portion of the atlas may be indicative of or may represent a two-dimensional or three-dimensional shape of a portion of the body, in particular an anatomical portion of the body. By way of example, at least a portion of the atlas may be indicative of or may represent a three-dimensional outer surface of one or more anatomical or functional portions of the body. By way of example, the anatomical portion of the body may be one or more bones and/or one or more portions of bones.
The atlas may be generated based on anatomical data acquired from a plurality of human bodies. The anatomical data may be acquired using medical imaging units, such as X-ray imaging units, magnetic resonance tomography units and/or ultrasonic imaging units. The human bodies, which are used for generating the atlas data may share a common feature or range of features, such as gender, age, ethnicity, body size, body mass and/or pathologic state.
The present disclosure pertains to a system for analysis of medical image data, which represent a plurality medical images, each of which being a two-dimensional or a three-dimensional image. The medical images are acquired from one or more human subjects using one or more medical imaging units, wherein each of the medical imaging units is configured to acquire medical images. The system comprises a data processing system, which is configured to read and/or generate, for each of the medical images, one or more quality metrics. The data processing system is further configured to receive, via the user interface, for at least a portion of the images, user input indicative of a user-specified quality rating for the respective image. The data processing system is further configured to: (a) determine or adapt an algorithm for determining a combined quality metric based on the image quality metrics and based on the user input; and/or to (b) determine or adapt an algorithm for classifying the images or selecting a portion of the images, wherein the classifying or the selecting is based on the user input and based on the combined quality metric. The combined quality metric depends on an interaction between the image quality metrics in their combined effect on the combined quality metrics.
The data processing system may be configured to read at least a portion of the quality metrics from one or more files, which are stored in a storage device of the data processing system. Additionally or alternatively, the data processing system may be configured to determine at least a portion of the quality metrics using an analysis algorithm, which uses at least a portion of the medical image data as input. The analysis algorithm may include, but is not limited to, one or a combination of a segmentation algorithm, a registration algorithm for registering the image with an atlas, a classifier, a machine-learning algorithm. The machine-learning algorithm may be configured for supervised and/or unsupervised learning. Specifically, the machine-learning algorithm may be an artificial neural network (ANN).
Additionally or alternatively, the data processing system may be configured to receive at least a portion of the quality metrics via a user interface of the data storage system and/or from a further data processing system. The data processing system may determine the image quality metrics automatically or interactively (i.e. requiring user intervention).
The user input, which is indicative of a user-specified quality rating may include one of a plurality of predefined levels. The user input may be received via a user interface. The user interface may include but is not limited to one or a combination of: a keyboard, a computer mouse, a touch control, a voice control and/or gesture control.
The algorithm for classifying the images may be an algorithm for classifying, each of the images, into one or more of a plurality of predefined classes. The classification may include binary classification data and/or probabilistic classification data. Probabilistic classification data may be defined as data, which include one or more probability values for one or more of the predefined classes. Binary classification data may be defined as data, which include for one or more of the predefined classes, either a value which indicates that the image is a member of the class or a value that the image is not a member of the class.
The predefined classes may include a class of diagnostic images and/or a class of non-diagnostic images. The predefined classes may be represented by the class of diagnostic images and the class of non-diagnostic images.
The algorithm for selecting may, for example, an algorithm for selecting a portion of the diagnostic images. In an alternative embodiment, the algorithm for selecting is an algorithm for selecting the non-diagnostic images.
According to a further embodiment, the determining of the algorithm for determining the combined quality metric and/or the determining of the algorithm for classifying the images includes training a machine learning algorithm, in particular an artificial neural network (ANN). The artificial neural network may include an input layer, one or more intermediate layers and an output layer. The artificial neural network may be configured as a convolutional neural network, in particular as a deep convolutional neural network.
According to a further embodiment, the system is configured to display, via the user interface, output, which is indicative of the combined quality metric for at least a portion of the images. Additionally or alternatively, the system may be configured to display output, which is indicative of a classification of the medical images, which classifies the medical images based on the combined quality metric. The output may be indicative of probabilistic classification data and/or binary classification data. The classification may be determined and/or adapted by the data processing system based on the user input, which is indicative of the user-specified quality rating.
According to an embodiment, the data processing system is configured to receive, via the user interface, a user-specified selection of one or more of the medical images for inputting the user input which is indicative of the user-specified quality rating for the images, which are selected by the user-specified selection. The user-specified selection may be received via a graphical user interface, in particular via a computer mouse of the graphical user interface. The data processing system may be configured to request the user, in response to the selection, to input the user input, which is indicative of the user-specified quality rating for the selected one or more images.
According to an embodiment, the system is configured to output, for the at least the portion of the images, output, which is indicative of one or more image quality metrics of the respective image.
According to a further embodiment, the system is configured to output a graphical representation which is indicative of a coordinate system of one or more of the image quality metrics. The graphical representation may be displayed using a graphical user interface. The graphical representation may be displayed on a display device of the data processing system. For one or more of the images, the graphical representation may further be indicative of a spatial relationship between the respective image and the coordinate system. The graphical representation may include, for each of the images, a graphical representation representing an image, such as one or more icons, wherein a position and/or orientation of the graphical representation representing the image, relative to the coordinate system is indicative of the image quality metrics of the respective image.
For each of the images and for the image quality metrics, which are used to form the coordinate system, the spatial relationship may be indicative of the values of the image quality metrics of the respective image. The spatial relationship may be a one-, two- or three-dimensional spatial relationship.
According to the embodiment, the data processing system is configured to receive, via the user interface, a user-specified selection of one or more of the images for which the graphical representation is indicative of the spatial relationship. The user interface may be configured as a graphical user interface in which one or more regions are user-selectable. The selectable regions may be displayed on a display device of the data processing system so that the location and extent of the regions are visually perceptible for the user. A spatial arrangement of the regions may depend on the graphical representation, in particular on the spatial relationship of which the graphical representation is representative. The graphical representation may be indicative of the user-selectable regions.
Each of the user-selectable regions may be user-selectable using a computer mouse of the user interface. Each of the user-selectable regions may represent a medical image. Each of the user-selectable regions may be arranged in a coordinate system of one or more of the image quality metrics. The graphical user interface may display, for each of the user-selectable region, a graphical representation, such as an icon, representing a medical image, which is selectable by the user input.
By way of example, the graphical representation includes a graph. The graph may illustrate the combined quality metric of the images as discrete values of a height function.
The coordinate system may be one, two or three-dimensional. Additionally or alternatively, the coordinate system may be a rectangular, spherical or cylindrical coordinate system. The data processing system may be configured to receive user input indicative of one or more selected image quality metrics. The data processing system may be configured to display a coordinate system, which includes the selected image quality metrics.
According to a further embodiment, the determining or adapting of the algorithm for classifying the images and/or for determining the combined quality metric is performed using one or a combination of: a maximum likelihood model and a machine learning algorithm. The machine-learning algorithm may be configured for supervised or unsupervised learning. The machine learning algorithm may be implemented using a support vector machine or an artificial neural network.
According to an embodiment, the system includes one or more medical imaging units, wherein each of the medical imaging units is configured to acquire medical images.
The present disclosure further pertains to a computer-implemented method for analysis of medical image data, which represent a two-dimensional or three-dimensional medical image, wherein the image is an image of at least a portion of a human subject. The method is performed using a data processing system, wherein the method comprises reading and/or determining, using the data processing system, for the medical image, a plurality of image quality metrics. The method further comprises automatically or interactively determining, using the data processing system, a combined quality metrics based on the image quality metrics. The determining of the combined quality metrics takes into account an interaction between the image quality metrics in their combined effect on the combined quality metrics.
The present disclosure pertains to a method for acquiring and analyzing medical images. The method includes acquiring, using a medical imaging unit a medical image, wherein the method further includes the computer-implemented method described in the previous paragraph for analyzing the acquired medical image.
The present disclosure further pertains to a computer implemented method for analysis of medical image data, which represent a plurality medical images, each of which being a two-dimensional or a three-dimensional image; wherein the medical images are images of portions of one or more human subjects. The method is performed using a data processing system, wherein the method comprises reading and/or generating, using the data processing system, for each of the medical images, one or more quality metrics. The method further includes receiving, via a user interface of the data processing system, for at least a portion of the images, user input indicative of a user-specified quality rating for the respective image. The method further comprises at least one of the following: (a) determining or adapting, using the data processing system, an algorithm for determining a combined quality metric based on the image quality metrics and based on the user input; and/or (b) determining or adapting, using the data processing system, an algorithm for classifying the images or selecting a portion of the images, wherein the classifying or the selecting is based on the user input and based on the combined quality metric. The combined quality metric depends on an interaction between the image quality metrics in their combined effect on the combined quality metrics.
The present disclosure further pertains to a method for acquiring and analyzing medical images. The method includes acquiring, using one or more medical imaging units a plurality of medical images; wherein each of the medical imaging units is configured to acquire medical images; wherein the method further includes the computer-implemented method described in the previous paragraph for analyzing the plurality of medical images.
The present disclosure pertains to a program element for analysis of medical image data, which represent a two-dimensional or three-dimensional medical image, wherein the image is an image of at least a portion of a human subject; wherein the method is performed using a data processing system. The program element, when being executed by a processor of the data processing system, is adapted to carry out reading and/or determining, using the data processing system, for the medical image, a plurality of image quality metrics. The program element, when being executed by a processor, is further configured to carry out automatically or interactively determining, using the data processing system, a combined quality metrics based on the image quality metrics. The determining of the combined quality metrics takes into account an interaction between the image quality metrics in their combined effect on the combined quality metrics.
The present disclosure pertains to a program element for analysis of medical image data, which represent a plurality medical images, each of which being a two-dimensional or a three-dimensional image. The medical images are images of portions of one or more human subjects. The program element, when being executed by a processor of the data processing system, is adapted to carry out reading and/or generating, using the data processing system, for each of the medical images, one or more quality metrics. The program element is further adapted to carry out receiving, via a user interface of the data processing system, for at least a portion of the images, user input indicative of a user-specified quality rating for the respective image. The program element, when being executed by a processor, is further adapted to carry out at least one of the following (a) determining or adapting, using the data processing system, an algorithm for determining a combined quality metric based on the image quality metrics and based on the user input; and/or (b) determining or adapting, using the data processing system, an algorithm for classifying the images or selecting a portion of the images, wherein the classifying or the selecting is based on the user input and based on the combined quality metric. The combined quality metric depends on an interaction between the image quality metrics in their combined effect on the combined quality metrics.
The present disclosure pertains to a computer program product having stored thereon the computer program element of any one of the above-described embodiments.
Each of the imaging units 2a, 2b, 2c is configured to acquire projection X-ray images from a subject 8a, 8b, 8c, which is positioned between the radiation source 3a, 3b, 3c and the radiation-sensitive detector 4a, 4b, 4c. The system 1 includes, for each of the medical imaging units 2a, 2b, 2c, a computer system 5a, 5b, 5c, which is operated by a radiographer who positions the subject 8a, 8b, 8c between the X-ray-radiation source 3a, 3b, 3c and the radiation-sensitive detector 4a, 4b, 4c so that a body portion of the subject 8a, 8b, 8c is imaged using the X-rays emitted from the radiation source 3a, 3b, 3c.
As is illustrated in
The system 1 further comprises a central computer system 7, which may be, for example, operated by a radiologist. The central computer system 7 may be a dedicated server that receives all images acquired using the medical imaging units 2a, 2b and 2c for analysis by the radiologist. The computer systems are in signal communication with each other via a computer network, which may include a LAN 6 (local area network) and/or the Internet.
Each of the medical imaging units 2a, 2b and 2c is configured to determine, for each of the acquired images, a plurality of image quality metrics based on the image data of the respective image. The determination of the imaging quality metrics may be performed automatically or semi-automatically (i.e. requiring user intervention). One or more of all of the image quality metrics may be indicative of, or may be a parameter of, a position and/or an orientation of the imaged body portion of the imaged subject 8a, 8b, 8c relative to a component of the medical imaging unit 2a, 2b, 2c. The component may include at least a portion of the X-ray radiation source 3a, 3b, 3c and/or the radiation-sensitive detector 4a, 4b, 4c.
It is to be noted that these image quality metrics are only examples and the present invention is not limited to these image quality metrics. Specifically, the image quality metrics, as well as the predefined ranges and/or the optimum values for the image quality metrics, may depend on the imaged body portion and/or on a certain diagnosis which is intended to be made based on the radiograph.
The determination of one or more or all of the image quality metrics may be performed using image processing applied to the image. The image processing may include a segmentation of the image. The segmentation of the medical image may be performed using one or a combination of the following segmentation techniques: thresholding, region growing, Watershed transformation, edge detection, using a shape model, using an appearance model and hand-segmentation using user interaction with the graphical user interface. Additionally or alternatively, the segmentation may be performed using an artificial neural network. The artificial neural network may be trained using hand segmentation. The segmentation may be performed automatically or interactively (i.e. requiring user intervention). In interactive segmentations, the computer system may receive user input, which is indicative of one or more parameters of a location, an orientation and/or an outer contour of an image region.
The segmentation may result in one or more segmented image regions and/or in one or more contours. The image regions and/or contours may be two-dimensional. In three-dimensional images, the image regions or contours may be three-dimensional. At least a portion of the image regions or contours may represent an anatomical or functional portion of the body or a surface thereof. An anatomical portion of the body may be a bone structure and/or a tissue structure of the body. A functional portion of the body may be a portion of the body which performs an anatomical function.
Additionally or alternatively, the determination of the image quality metrics may include registering at least a portion of the image with an atlas. The data processing system may be configured to extract features from the image, which are used for registering the image with the atlas. The features may be extracted using image processing. The atlas registration may be performed based on the segmentation of the image. Additionally or alternatively, the atlas registration may be performed based on landmarks, which are detected in the image using the computer system.
The segmentation of the image may be configured to extract one or more parameters of a position, one or more parameters of an orientation and/or one or more parameters of an extent and/or a shape of a portion of the body, such as an anatomical or functional portion of the body. The atlas registration may use one or more of the extracted parameters to register the image with the atlas.
It has been shown by the inventors that the atlas registration leads to a more reliable determination of the image quality metrics. Thereby, the combined quality metrics, which is determined based on the determined image quality metrics and which will be explained in more detail below, may more accurately indicate, whether or not an image is of diagnostic image quality. However, it is also been shown that a sufficient accuracy can be obtained without using segmentation and atlas registration techniques.
Each of the computer systems 5a, 5b and 5c (shown in
Each of the computer systems 5a, 5b and 5c is configured to determine, for each of the medical images, at least one combined quality metric based on the image quality metrics of the respective image. By way of example, the combined quality metric includes a parameter, a vector and/or a state variable which are calculated based on the calculated values of the image quality metrics. The combined quality metric may be a function of the image quality metrics. The function may include an analytical function and/or a tabulated function, which are stored in the computer systems 5a, 5b, 5c.
The inventors have found that the image quality metrics interact in their effect on the image quality, which is a measure of the suitability of the medical image for medical diagnosis.
By way of example, for ankle radiographs (such as the radiograph shown in
Additionally or alternatively, for images in which the deviation of the rotation angle α about the tibia's longitudinal axis from the optimum rotation angle is comparatively small, the flexion angle β of the ankle joint can have a comparatively large deviation from the optimum flexion angle.
The inventors have further shown that a combined quality metric, which takes into account this interaction between image quality metrics allow a more accurate determination of whether or not an image is of diagnostic quality.
Characteristics of such a combined quality metrics, which takes into account the interaction between the image quality metrics, are schematically illustrated in
Since the computer systems 5a, 5b and 5c (shown in
Therefore, the limit, which is represented by the contour line 10 represents a criterion for selecting images based on the combined quality metric.
Since the images represented by points 15, 16 and 17 are within the region limited by the contour line 10 or at the contour line 10, these images are of diagnostic image quality. On the other hand, image 18 is outside the contour line 10 so that this image is of a non-diagnostic image quality.
Since the contour line 10 for the combined quality parameter deviates from the shape of a rectangle 11, the range 12 within which the rotation angle about the tibia's longitudinal axis (parameter α in
Each of the computer systems 5a, 5b and 5c (shown in
The output may further be configured so that it is indicative for a change in imaging conditions so that the resulting image is suitable (or better suitable) for diagnosis.
By way of example, for the image 18, which is shown in
The determined image quality metrics and the combined quality metrics are transmitted from the computer systems 5a, 5b and 5c (shown in
The computer system 7 includes a user interface allowing the radiologist to review the determined combined quality metrics of a plurality of images acquired using the imaging units 2a, 2b and 2c. Based on the review, the central computer system 7 and/or the computer systems 5a, 5b and 5c are able to adjust the algorithm for determining the combined quality metrics or to adjust the algorithm for determining, based on the combined quality metrics, whether or not the image is a diagnostic image.
Therefore, the central computer system 7 is configured to determine, based on a user-specified quality rating for one or more of the images (provided by the radiologist), data, which is indicative of a criterion for selecting images (such as diagnostic images) based on the combined quality metric. Further, the central computer system 7 is configured to determine, based on the user specified quality rating, data, which are used for determining the combined quality metric based on the image quality metrics. Thereby, the determination of the combined quality metric may be adjusted based on user input provided by the radiologist.
It is conceivable that for each of the images, the combined quality metric of the respective image is indicated by other means, such icons, which are displayed in different color or which include a number.
It also is conceivable that the coordinate system is a one-dimensional coordinate system of one single image quality metric or that the coordinate system is a three-dimensional spatial coordinate system. It is also conceivable that one or more dimensions are illustrated using a color coding, icons and/or numbers so that it is possible to provide a graphical representation of more than three image quality metrics. The graphical user interface may be configured to receive user input for selecting one or more image quality metrics, which are used to generate the graphical representation. Additionally or alternatively, the central computer system may be configured to automatically or interactively (i.e. requiring user intervention) determine one or more image quality metrics based on a predefined criterion. The predefined criterion may depend on an intended diagnosis.
The graphical user interface of the central computer system is further configured so that the radiologist can select one or more of the images. By way of example, the graphical user interface may be configured so that each of the icons is indicative of a user-selectable region of the graphical user interface. The user-selectable regions may be selectable using a computer mouse of the data processing system. In response to the selection, the central computer system displays the selected image allowing the radiologist to review, whether the determined combined quality metric adequately indicates, whether or not the image is suitable for diagnosis.
Therefore, the data processing system may be configured to receive, via the graphical user interface, a user-specified selection of one or more of the medial images. In response to the user-specified selection, the central computer system displays the selected one or more images to the user and requests the user to input user input, which, for each of the selected one or more medical images, is indicative of a user-specified quality rating for the respective image.
In a same manner, as has been discussed in connection with
Additionally or alternatively, the central computer system may be configured to register the medical image with an atlas using one or more rigid transformations. The central computer system may further be configured to display at least a portion of the transformed image to the user using the graphical user interface.
The rigid transformations may include a rotation transformation, a scaling transformation and/or a translation of the image. Since the atlas has a fixed position, orientation and scale, applying the rigid transformations to a plurality of different images generates a representation of the anatomy, which is coherent between these images. Therefore, using these rigid transformations makes it easier for the radiologist to compare images and thereby to recognize anomalies within the image. This allows for a more time-efficient and more reliable review of the image quality conducted by the radiologist.
Therefore, similarly as has been discussed in connection with
As has further been explained in connection with
The graphical representation 21 allows the radiologist to select those images, which are close to the limit in order to check, whether the limit represents an adequate separation between images of diagnostic quality and images of non-diagnostic quality. The graphical user interface is configured so that the radiologist can select an image using the mouse cursor and the central computer system 7, in response to the selection, displays the image selected image to the user.
As is further schematically illustrated in
The central computer system 7 may further be configured to select images based on the limit and further based on the determined combined quality metric. Specifically, the selection may be so that it is not necessary for the radiologist to determine images, which need to be reviewed.
The central computer system 7 is further configured to receive, via the graphical user interface, for one or more of the images, input from the radiologist, which is indicative of a user-specified quality rating for the respective image. In particular, the user-specified quality rating may include an indication of whether or not the image is of diagnostic image quality.
Based on the input received via the user interface, the central computer system adjusts the limit for the combined quality metric (which is an example for algorithm for classifying the images based on the combined quality metric) and/or the central computer system adjusts the algorithm for determining the combined quality metric. It is also possible that the central computer system determines, based on the user input, a new algorithm for classifying the images and/or a new algorithm for determining the combined quality metric.
The adaptation or determination of the criterion for selecting images based on the combined user input and/or the adaptation or determination of the algorithm for determining the combined quality metric may include a machine learning process in which the user's input (i.e. the indication whether or not the image is suitable for diagnosis and/or the user-specified quality rating) is used as training data.
Additionally or alternatively, the determination of the algorithm for classifying the images based on the combined user input and/or the adaptation of the algorithm for determining the combined quality metric may be performed using dimensionality reduction techniques, such as kernel principal component analysis. By way of example, such techniques may be used to simplify the determination of the limit by reducing the required number of images to be reviewed by the radiologist. Specifically, for a given limit, these techniques can be used to find a lower dimensional phase-space representation of the limit for the combined quality metric. The lower dimensional phase-space representation may facilitate a graphical illustration of the limit via the graphical user interface and may also make the adaptation of the limit for the combined quality metric more efficient. After having adapted the limit, the dimensionality reduction technique may be applied again for further simplifying the phase-space.
The determination or adaptation of the algorithm for classifying the images and/or for determining the combined quality metric may be performed using one or a combination of: a maximum likelihood model and a machine learning algorithm. The machine learning algorithm may be configured for supervised and/or unsupervised learning. The machine learning algorithm may be implemented using a support vector machine or an artificial neural network.
Specifically, the maximum likelihood model may be implemented using the assumption that the image quality metrics span an Euclidean space and that the distribution of one of a plurality of predefined classes (such as the class of diagnostic images) in this space is comparatively compact and that a function f(p) is known to approximate it, such as a Gaussian normal distribution.
According to an embodiment, the central computer system is configured to generate a maximum likelihood model that indicates, for a plurality of points p in the Euclidean space, the probability that the corresponding image is a member of one of the plurality of predefined classes. By way of example, the predefined classes include a class of diagnostic images and a class of non-diagnostic images. The central computer system may further be configured to use a threshold Θ for deciding, for a combination of image quality metrics, of which class the corresponding image is a member (e.g. diagnostic if f(p)>Θ or non-diagnostic if f(p)<=Θ). The corresponding boundary may represent a Mahalanobis distance.
A classification algorithm, which uses machine learning may be implemented using a support vector machine and/or an artificial neural network. A description of support vector machines, which can be used for the embodiments described herein, can be found in the book “The nature of statistical learning”, written by Vladimir Vapnik in 1995 and published by “Springer Science+Business Media”, New York.
As can be seen from
A description of an ANN, which may be used for the embodiments described in the present disclosure, is described in the article “Foveal fully convolutional nets for multi-organ segmentation”, written by Tom Brosch and Axel Saalbach and published in SPIE 10574, Medical Imaging 2018: Image Processing, 105740U. The contents of this article is incorporated by reference herein for all purposes.
The ANN may be configured as a convolutional neural network. The term “convolutional neural network” may be defined herein as an artificial neural network having at least one convolutional layer. A convolutional layer may be defined as a layer which applies a convolution to the previous layer. The convolutional layer may include a plurality of neurons, wherein each neuron receives inputs from a predefined section of the previous layer. The predefined section may also be called a local receptive field. The weights for the predefined section may be the same for each neuron in the convolutional layer. Thereby, the convolutional layer may be defined by the two concepts of weight sharing and field accepting. The ANN may include one or more subsampling layers. Each of the subsampling layers may be arranged subsequent (in particular immediately subsequent) to a respective convolutional layer. The subsampling layer may be configured to downsample the output of the preceding convolution layer along the height dimension and along the width dimension. The number of convolution layers, which are succeeded by a pooling layer may be at least 1, at least 2 or at least 3. The number of layers may be less than 100, less than 50, or less than 20.
Before we go to set out the claims, the first set out the following clauses describing some prominent features of certain embodiments of the present disclosure:
1. A system for analysis of medical image data, which represent a two-dimensional or three-dimensional medical image, which has been acquired from a human subject; wherein the system comprises a data processing system, which is configured to: read and/or determine, for the medical image, a plurality of image quality metrics; and to automatically or interactively determine a combined quality metrics based on the image quality metrics; wherein the data processing system is configured so that the determination of the combined quality metrics takes into account an interaction between the image quality metrics in their combined effect on the combined quality metrics.
2. The system of clause 1, wherein the interaction between the image quality metrics is so that a change of the combined quality metrics, which is caused by a change of a first one of the quality metrics is compensable by a change of one or more of the remaining quality metrics.
3. The system of clauses 1 or 2, wherein each of the image quality metrics is associated with a respective predefined optimum value or predefined optimum range; wherein the interaction between the image quality metrics is so that an increased deviation from the optimum range or optimum value of a first one of the metrics is compensable by a decreased deviation from the optimum range or optimum value of one or more of the remaining image quality metrics.
4. The system of any one of the preceding clauses, wherein one or more or each of the quality metrics is indicative of one or more parameters of a position and/or one or more parameters of an orientation of an imaged body portion relative to a component of the imaging unit or relative to a further body portion.
5. The system of any one of the preceding clauses, wherein the system is configured to determine, for one or more of the images, based on the determined combined quality metric, a parameter or state variable representing a change of imaging conditions for using the imaging unit; wherein the changed imaging conditions are configured to change one or more of the image quality metrics.
6. The system of any one of the preceding clauses, wherein the determination of the image quality metrics include: segmenting at least a portion of the image; and/or registering the image with an atlas using the segmented image.
7. A system for analysis of medical image data, which represent a plurality of medical images, each of which being a two-dimensional or a three-dimensional image; wherein the medical images are acquired from one or more human subjects using one or more medical imaging units, wherein each of the medical imaging units is configured to acquire medical images; wherein the system comprises a data processing system, which is configured to: read and/or generate, for each of the medical images, one or more quality metrics; and to receive, via the user interface, for at least a portion of the images, user input indicative of a user-specified quality rating for the respective image; wherein the data processing system is further configured to: (a) determine or adapt an algorithm for determining a combined quality metric based on the image quality metrics and based on the user input; and/or to (b) determine or adapt an algorithm for classifying the images or selecting a portion of the images, wherein the classifying or the selecting is based on the user input and based on the combined quality metric; wherein the combined quality metric depends on an interaction between the image quality metrics in their combined effect on the combined quality metrics.
8. The system of clause 7, wherein the system is configured to display, via the user interface: output, which is indicative of the combined quality metric for at least a portion of the images; and/or output, which is indicative of a classification of the medical images, which classifies the images based on the combined quality metric.
9. The system of any one of clauses 7 or 8, wherein the system is configured to output, via the user interface, a graphical representation which is indicative of a coordinate system of one or more of the image quality metrics; wherein, for one or more of the images, the graphical representation is further indicative of a spatial relationship between the respective image and the coordinate system.
10. The system of any one of clauses 7 to 9, wherein the data processing system is configured to receive, via the user interface, a user-specified selection of one or more of the images for inputting the user input which is indicative of the user-specified rating for the images, which are selected by the user-specified selection.
11. A computer-implemented method for analysis of medical image data, which represent a two-dimensional or three-dimensional medical image, wherein the image is an image of at least a portion of a human subject; wherein the method is performed using a data processing system, wherein the method comprises: reading and/or determining, using the data processing system, for the medical image, a plurality of image quality metrics; and to automatically or interactively determining, using the data processing system, a combined quality metrics based on the image quality metrics; wherein the determining of the combined quality metrics takes into account an interaction between the image quality metrics in their combined effect on the combined quality metrics.
12. A computer-implemented method for analysis of medical image data, which represent a plurality medical images, each of which being a two-dimensional or a three-dimensional image; wherein the medical images are images of portions of one or more human subjects; wherein the method is performed using a data processing system, wherein the method comprises: reading and/or generating, using the data processing system, for each of the medical images, one or more quality metrics; receiving, via a user interface of the data processing system, for at least a portion of the images, user input indicative of a user-specified quality rating for the respective image; wherein the method comprises at least one of the following: (a) determining or adapting, using the data processing system, an algorithm for determining a combined quality metric based on the image quality metrics and based on the user input; and/or (b) determining or adapting, using the data processing system, an algorithm for classifying the images or selecting a portion of the images, wherein the classifying or the selecting is based on the user input and based on the combined quality metric; wherein the combined quality metric depends on an interaction between the image quality metrics in their combined effect on the combined quality metrics.
13. A program element for analysis of medical image data, which represent a two-dimensional or three-dimensional medical image, wherein the image is an image of at least a portion of a human subject; wherein the method is performed using a data processing system, wherein the program element, when being executed by a processor of the data processing system, is adapted to carry out: reading and/or determining, using the data processing system, for the medical image, a plurality of image quality metrics; and to automatically or interactively determining, using the data processing system, a combined quality metrics based on the image quality metrics; wherein the determining of the combined quality metrics takes into account an interaction between the image quality metrics in their combined effect on the combined quality metrics.
14. A program element for analysis of medical image data, which represent a plurality medical images, each of which being a two-dimensional or a three-dimensional image; wherein the medical images are images of portions of one or more human subjects; wherein the program element, when being executed by a processor of the data processing system, is adapted to carry out: —reading and/or generating, using the data processing system, for each of the medical images, one or more quality metrics; receiving, via a user interface of the data processing system, for at least a portion of the images, user input indicative of a user-specified quality rating for the respective image; wherein the program element, when being executed by the processor is further configured to carry out at least one of the following: (a) determining or adapting, using the data processing system, an algorithm for determining a combined quality metric based on the image quality metrics and based on the user input; and/or (b) determining or adapting, using the data processing system, an algorithm for classifying the images or selecting a portion of the images, wherein the classifying or the selecting is based on the user input and based on the combined quality metric; wherein the combined quality metric depends on an interaction between the image quality metrics in their combined effect on the combined quality metrics.
15. Computer program product having stored thereon the computer program element of clause 13 and/or clause 14.
The above embodiments as described are only illustrative, and not intended to limit the technique approaches of the present invention. Although the present invention is described in details referring to the preferable embodiments, those skilled in the art will understand that the technique approaches of the present invention can be modified or equally displaced without departing from the protective scope of the claims of the present invention. In particular, although the invention has been described based on a projection radiograph, it can be applied to any imaging technique which results in a projection image. In the claims, the word “comprising” does not exclude other elements or steps, and the indefinite article “a” or “an” does not exclude a plurality. Any reference signs in the claims should not be construed as limiting the scope.
| Number | Date | Country | Kind |
|---|---|---|---|
| 20178943.5 | Jun 2020 | EP | regional |
| Filing Document | Filing Date | Country | Kind |
|---|---|---|---|
| PCT/EP2021/065265 | 6/8/2021 | WO |
