Patent Application
20240212136
This invention relates to the field of Computer Aided Diagnosis (CADx) devices and methods for assisting the interpretation of medical images to support clinicians in healthcare. In particular, the field relates to risk Computer Aided Diagnosis devices to assist the reading and reporting of medical images by radiologists and the interpretation of the radiologist's report by the physician responsible for patient care.
In the field of medical imaging, a variety of technologies can be used to investigate biological processes and anatomy. The following examples are types of scan that may be used to provide medical images: X-Ray; Computed Tomography (CT); Ultrasound (US); Magnetic Resonance Imaging (MRI); Single Photon Emission Tomography (SPECT); and Positron Emission Tomography (PET). Each type of scan is referred to as an “imaging modality”.
Typically, a scan provides a “dataset”. The dataset comprises digital information about the value of a variable at each of a plurality of spatial locations in either a two-dimensional or (more typically) a three-dimensional space. As a specific example, a CT scan may provide images of the chest of a patient. Such a CT scan might, as a more specific example, show lung nodules in the chest.
Computer Aided Detection (CADe) devices serve to assist clinicians in assessing the medical images. CADe devices need to provide a clinician with standardised, objective and repeatable information. The information typically relates to particular anatomical regions, including both normal tissue and lesions, within a person. CADe devices may be used as a so-called ‘Second Reader’ device. Second Reader Devices are based on an approach whereby a radiologist first looks at an image resulting from a scan, for example a mammogram. The radiologist will then, based on training and experience, identify areas of the scan where the radiologist considers that there may need to be a further investigation, for example a biopsy. However, the radiologist can then consider the CADe findings. Those findings might involve a display to highlight any additional suspicious regions on the mammogram. The radiologist will then, based on training and experience, look at those further areas of the scan. The CADe device is thereby performing a second look at the scan. The results of the second look at the scan may be that the radiologist will be directed to areas of the scan that he/she had overlooked. In this way, CADe devices are designed to reduce ‘false negatives’, which are also termed ‘missed findings’. Thus, CADe devices perform a support role to clinicians. Examples of commercial CADe devices for lung health include ‘Veolity Lung™’ from MeVis, ‘Veye Chest™’ from Aidence, and ‘Lung AI™’ from Arterys.
Computer Aided Diagnosis (CADx) devices are a related technology to CADe. CADx devices attempt to solve a different problem and relate generally to risk assessment. Instead of focusing on potentially missed findings as in CADe, they try to assist the user to classify findings correctly, for instance, either as malignant or benign in the case of potentially cancerous lesions. They rely on the user to identify abnormalities, and typically provide a score that is indicative of the risk of malignancy. Examples of commercial CADx devices are the breast cancer screening product ‘Transpara™’ from ‘Screenpoint™’, and the lung cancer product Virtual Nodule Clinic (VNC) from Optellum. There are many non-clinical CADx devices in the academic literature.
In the space of lung cancer, CADx devices have traditionally focused on evaluating lung nodules to produce malignancy risk scores, with the aim of helping clinicians make more appropriate decisions for the patient. Examples of such lung cancer CADx devices include the VNC from Optellum, and qCT-Lung from qure.ai. The connection between risk assessment and clinical decision-making lies in that the most appropriate next step for a given patient with a lung nodule depends on the risk of it being malignant. For example, higher risk cases would warrant shorter term actions (e.g., PET/CT, biopsy, or short-term CT follow-up), while lower risk cases can be addressed with longer term CT follow-ups. The consensus of experts on how to link risk and clinical decision is collected in clinical guidelines published regularly by the relevant medical societies. In the case of lung cancer, these guidelines include those from the British Thoracic Society [1] and the Fleischner Society [2]. In these guidelines, risk assessment of the lung nodule is central to the decision-making process, hence it is thought that CADx devices that help clinicians better assess risk can lead to better management.
In addition to the risk assessment of nodules, the patient management process is critical in order to properly schedule follow-up action in line with nodule risk. The management of lung nodule patients is typically centralised to lung nodule clinics, where a team of thoracic medicine specialists (including radiologists, surgeons and pulmonologists) coordinate the optimal clinical decision for each patient. Lung nodule clinics in some cases use management software such as Optellum's VNC, which has been shown to reduce patient drop-out from nodule management and make the enrolment of new patients more efficient. Typically, a radiologist will make recommendations for patient follow-up based on findings in a new CT scan—for example if they identify a nodule with high risk of malignancy. These recommendations need to be passed on to the management team of the lung nodule clinic so that patients can be prioritised for follow-up actions.
Recommendations are generally made in a textual report which may or may not be integrated in the lung nodule clinic management software. With the recent rollout of lung cancer screening programs in the US and worldwide, radiologists typically need to make many decisions regarding lung nodule findings. For this reason, providing more efficient workflows for radiologists has the potential to improve patient outcomes by ensuring the quality and consistency of decision making by radiologists. However, the same considerations regarding workflow efficiency are potentially relevant to many medical imaging workflows and are not limited to a specific imaging modality or pathology. Moreover, other co-morbidities will typically play a role in the recommendation of the best clinical action for a given patient.
In the space of CADe and CADx systems, ‘end-to-end’ detection and risk assessment systems for lung nodules (i.e., combined CADe/CADx) can be found in the literature. Recently presented systems such as [3, 4, 5] perform end-to-end lung cancer risk assessment on a patient level. These could be used to decide which patients require close attention by the radiologist and have been tailored for lung cancer screening programs. The deployment of such systems could be expected to significantly reduce the radiology workload, which is already extremely high in many hospital systems. Nevertheless, efforts to increase the uptake of lung cancer screening (e.g., in the USA), or the raise of lung cancer screening programs and general lung health checks worldwide will mean that the radiologists will continue to face ever increasing workloads. Therefore, further innovations targeting the efficiency of the radiology workflow will still be very much needed.
In a first embodiment of the invention there is provided a computer implemented method for analysing medical scan images from a single patient comprising the steps of: receiving an input of a plurality of medical scan images for the single patient: providing an input of one or more characterised features from the medical scan images, in addition to the plurality of medical scan images to an optimisation model, where the optimisation model outputs the features and images sorted according to a predefined relevance criteria; providing an output to a user, related to the medical images according to the result of the predefined relevance criteria, that comprises a workflow showing the order in which the plurality of scans should be reviewed.
Preferably, wherein the output is at least one of: a display of medical images sorted according to the relevance criteria, a report containing information about the medical images sorted according to the relevance criteria.
In a preferred embodiment of the invention, the method further comprising the steps of: detecting one or more features of the plurality of medical scan images; and characterising the one or more detected features from the plurality of medical scan images; before providing the input to the optimisation model.
Preferably, the optimisation model only outputs the N most relevant features according to the relevance criterion, where N is a configurable parameter of the method.
In a preferred embodiment of the invention, the plurality of medical scan images is one of: CT scan, PET scan, MRI scan or SPECT scan. Further preferably, the medical scan image is a 2D image or a 3D image.
In a further preferred embodiment of the invention, the one or more medical scan images are images showing all or part of a lung, and the detected features are one or more of: lung nodule, features indicating the presence of at least one of emphysema, fibrotic tissue, consolidation and scarring. Preferably, the detected features are characterised according to one or more of: a malignancy score, an invasiveness score, a feature size.
In an embodiment of the invention, detecting the features is done with a first machine learning model, and characterising the features is done with a second machine learning model. Preferably, the first and second machine learning model uses a neural network, a support vector machine or a random forest algorithm.
In a preferred embodiment of the invention, the optimization model uses a machine learning model such as a neural network.
Preferably, the feature detection model is a single model that detects multiple feature classes simultaneously. Alternatively, the feature detection model comprises a plurality of models, each model detecting a certain class of feature.
Further preferably, the characterization model characterises all features in all classes for simultaneous consideration by the optimization model.
According to an embodiment of the invention, there is provided a device for analysing medical scan images from a single patient comprising: one or more processors configured to: receive an input of a plurality of medical scan images for the single patient: provide an input of one or more characterised features from the plurality of medical scan images, in addition to the medical scan images to an optimisation model, where the optimisation model outputs the features and images sorted according to a predefined relevance criteria; providing an output related to the medical images to a user according to the result of the predefined relevance criteria, that comprises a workflow showing the order in which the plurality of scans should be reviewed.
Preferably, the plurality of medical scan images is one of: CT scan, PET scan, MRI scan or SPECT scan, and the one or more medical scan images are images showing all or part of a lung.
In a preferred embodiment of the invention, the one or more processors are further configured to detect multiple classes of features in the plurality of input medical scan images.
Further preferably, the one or more processors are further configured to characterise the one or more detected features from the plurality of medical scan image, before the detected features are provided to the optimisation model.
In a preferred embodiment of the invention, detecting the features is done with a first machine learning model, and characterising the features is done with a second machine learning model, and the optimisation model also uses a machine learning model.
Further preferably, the output is at least one of: a display of medical images sorted according to the relevance criteria, a report containing information about the medical images sorted according to the relevance criteria.
This invention describes a radiology workflow optimisation method and system or device (100) to assist in the review of medical scan images for the reporting of features in medical image datasets and the issuing of recommendations, including clinical recommendations, the core of the day-to-day radiology workload. Preferably the method and device can be used in existing CADx or CADe devices. The system or device and method of this invention targets the optimization of a radiology workflow once the radiologist has started to read a medical image. In a first embodiment of the invention a computer implemented method for analysing medical scan images from a single patient is provided. The method comprising the steps of: receiving an input of a plurality of medical scan images for the single patient: providing an input of one or more characterised features from the medical scan images, in addition to the plurality of medical scan images to an optimisation model, where the optimisation model outputs the features and images sorted according to a predefined relevance criteria; providing an output to a user, related to the medical images according to the result of the predefined relevance criteria, that comprises a workflow showing the order in which the plurality of scans should be reviewed. In an embodiment of the invention, a device for carrying out the method is also provided. The device comprising: one or more processors configured to: receive an input of a plurality of medical scan images for the single patient: provide an input of one or more characterised features from the plurality of medical scan images, in addition to the medical scan images to an optimisation model, where the optimisation model outputs the features and images sorted according to a predefined relevance criteria; and providing an output related to the medical images to a user according to a result of the predefined relevance criteria, that comprises a workflow showing the order in which the plurality of scans should be reviewed.
The device includes one or more processors configured to control: an input circuit (105), a feature detection circuit (125), a feature characterisation circuit (150), a workflow optimisation circuit (175) and an output circuit (190). The input circuit (105) parses medical image datasets (110) as collected in hospital systems into a preferred format for analysis. The feature detection circuit (125) identifies occurrences of one or more type of feature in an input medical image dataset (110), preferably this identification is done automatically. The input medical image data set comprises a plurality of images from a single patient The feature characterisation circuit (150) classifies or scores each identified feature according to one or more pre-defined relevance criteria, again, this is preferably done automatically by the circuit. The workflow optimisation circuit (175) organizes the detected image features, considering their characterisation, according to at least one predefined relevance criteria, with the objective of producing a reading workflow for the radiologist to follow when reviewing the medical image for a single patient as to make more efficient use of their reading time. The output circuit (190) interfaces the workflow optimisation system with other hospital systems. In a preferred embodiment of the invention, the device for analysing medical scan images from a single patient comprises one or more processors configured to: receive an input of a plurality of medical scan images for the single patient: provide an input of one or more characterised features from the plurality of medical scan images, in addition to the medical scan images to an optimisation model, where the optimisation model outputs the features and images sorted according to a predefined relevance criteria; providing an output related to the medical images to a user according to the result of the predefined relevance criteria, that comprises a workflow showing the order in which the plurality of scans should be reviewed.
By detecting and characterizing clinically relevant features from the medical scan images, preferably automatically, then organizing the findings in a way that helps optimise a radiology workflow, and providing an output related to the medical images to a user according to the result of the predefined relevance criteria that comprises a workflow showing the order in which the plurality of scans should be reviewed, radiologists can spend less time in laborious but automatable tasks and shift focus to reviewing and curating results. As a consequence, radiologists can reduce the time invested in clinically trivial cases and prioritise those patients who require closer attention.
The input circuit then provides an input to a feature detection circuit (125). This circuit comprises a feature detection model (130), and feature detections (135). In some embodiments of the invention the feature detection model is a neural network. The output of the feature detection circuit is provided to a feature characterisation circuit (150), which comprises a feature characterisation model (155) that outputs a set of characterised detections (160). The output of the feature characterisation circuit (150) goes to the workflow optimisation circuit (175). This circuit comprises a workflow optimisation model (180), and an output generator (185). The results of the workflow optimisation circuit (175) pass to the output circuit (190), where the overall output is generated. These various components of the workflow optimisation will be described in more detail as this description progresses.
The feature detection circuit (125) constitutes the first part of a preferred embodiment of the workflow optimisation system. It takes as input a medical image dataset (110) consisting of one or more medical images of the same patient, and outputs a list of detected features alongside their locations for any feature found in the images (135). The feature detection circuit can also take as input additional (non-image) configuration parameters (115), although some embodiments of the device do not require this input. An example of the usage of configuration parameters for the feature detection circuit could be to set a size threshold below which smaller detected features are discarded
Radiological medical image datasets comprise either 2D images or 3D image volumes.
In its general form, the output of the feature detection circuit indicates the location of any features of interest in the 2D or 3D image. In some examples, the location of features of interest in a CT volume are indicated with {x,y,z} coordinates in the coordinate system of the CT scanner. In other examples, locations could be represented by bounding boxes, spheres or segmentations
As illustrated in
The feature detection model (130) within the feature detection circuit is typically based on a machine learning model trained to detect the feature of interest, while the specific architecture may take a variety of forms. In some example implementations, the feature detection model (130) is based on a neural network. In other examples, the underlying machine learning architecture of the feature detector is a Support Vector Machine or a Random Forest algorithm.
Typically, the feature detection model (130) operates by computing the likelihood of feature presence at every position in the input image dataset and returning any locations where the likelihood of feature presence is above a threshold. In other examples, the feature detection model (130) computes the likelihood of feature presence in locations of interest in the input image dataset such as salient image locations. In some examples of the feature detection circuit (125), an operating threshold is supplied as an additional parameter (115), with any detected features below this threshold being discarded. In embodiments of the circuit that detect multiple classes of features, multiple operating thresholds may be supplied. The values of the operating thresholds are typically chosen according to the clinical problem: if the cost of missing an instance of a feature is high, the threshold can be set to a lower value to increase the sensitivity of the device. Equally, the threshold may be set to a high value in order to reduce number of false positive detections.
The feature detection circuit (125) outputs a list of findings identified in the medical image, which is passed to the feature characterisation circuit (150). Preferably, the feature characterisation circuit may characterise the one or more detected features from the plurality of medical scan image; before providing the input to the optimisation model. In some examples, the feature detection circuit (125) returns only a maximum number of detections of each feature class in an image, specifiable as an additional parameter supplied to the circuit.
In this embodiment of the invention, the feature detection circuit (125) detects lung nodules in a medical image dataset, preferably, a thoracic CT scans. As shown in
An embodiment of the feature detection circuit (125) that detects multiple feature classes is shown in
Results of the feature detection circuit (135), for all the different embodiments of the invention are passed to the feature characterisation circuit (150) along with the medical image dataset (110), for further analysis and processing. Like the feature detection circuit (125), the feature characterisation circuit (150) is preferably based on a prediction model that has been built using machine learning algorithm, the feature characterisation model (155). The feature characterisation model (155) is in this case trained to predict characteristics of each of the features detected by the feature detection circuit (125), which in some examples consist of different categories of features.
Analogous to the feature detection circuit (125), the predictive model of the feature characterisation circuit may itself be an ensemble of models specialised to characterise different categories of feature, or different characteristics of single feature category. A characteristic is here defined as a property with a numerical value, or a categorisation, that is associated with a detected instance of a feature. For the purposes of the feature characterisation circuit (150), it should be possible to determine or predict the characteristic from the image. In many examples, predictions of characteristics are represented as probabilities.
In general, the feature characterisation circuit (150) extracts one or more image crops or sub-volumes from the 2D image or 3D image volume, respectively. The extracted regions from the image or image volume are typically centred on each of the detected features. For each detected feature supplied as input to the feature characterisation circuit (150), the feature characterisation model (155) outputs a score or categorisation given the appearance of the detected feature on the image. In some cases, multiple detections of a given category of features may be present in the same image crop or sub-volume. Here, the feature characterisation model (155) typically only outputs a single score or categorisation which considers the presence of all detected features.
The feature characterisation model (155) may take a variety of forms including neural networks and other machine learning models such as Support Vector Machines or Random Forests.
As illustrated in
Various embodiments of the feature characterisation circuit (150) are described below. All embodiments describe the characterisation of features in a medical scan dataset. Preferably, the features are lung nodules detected in thoracic CT scans by e.g., Embodiment 1 of the feature detection circuit (125). It would be clear to a person skilled in the area how the feature characterisation circuit would be applicable to conditions other than lung nodules if they can be diagnosed from medical images.
In an embodiment of the invention features detected by the feature detection circuit (125) are characterised by the feature characterisation circuit (150) according to a particular criterion. In a preferred embodiment of the invention, the features are lung nodules that are characterised according to their malignancy risk. This embodiment is appropriate e.g., for a use case of the workflow optimisation system (100) specialised for thoracic CT scans; for instance, optimising the workflow of radiologists carrying out imaging as part of a lung cancer screening program. Assessment of the risk of identified lung nodules being malignant can be time consuming and can vary widely according to the experience and expertise of the radiologist, among other subjective factors. Automated risk assessment can therefore make the radiology workflow more accurate and efficient by presenting the highest risk findings to the radiologist first.
This embodiment of the feature characterisation circuit (150) for lung cancer risk as shown in
For this task the feature characterisation model (155) is trained on a dataset of thoracic CT scans, some of which containing lung nodules annotated as either benign or malignant.
In another embodiment of the feature characterisation circuit (150), lung nodules on a medical scan image are characterised by their potential invasiveness, assuming they were malignant. Predicted cancer invasiveness is an important factor to consider when deciding how to prioritise the reporting of lung nodules in patients with multiple detected nodules as it affects the urgency with which the patient should be followed up.
As in the previous embodiment, training the feature characterisation circuit (150) to estimate the level of invasiveness of a lung nodule assuming it is a cancer can be done from a dataset of images of lung nodules annotated with information relevant to invasiveness. In some examples, the invasiveness information comes in the form of the histological type of the cancer. For instance, in the case of lung cancer, Adenocarcinoma In Situ (AIS), Minimally Invasive Adenocarcinoma (MIA) and Invasive Adenocarcinoma (IA) represent different levels of invasiveness of a similar type of lung cancer as determined by their histological characterisation.
As in the previous embodiment, the feature characterisation model (155) also operates on sub-volumes of a thoracic CT scan provided as input (110), centred on the lung nodule detections also given as input (135). Here, the circuit outputs a list of lung nodule detections now annotated with invasiveness scores (160).
In a further embodiment of the feature characterisation circuit (150), multiple characteristics are computed for a single category of feature. Specifically, both lung cancer risk and lesion invasiveness are computed for each detected lung nodule in a thoracic CT scan. As illustrated in
The feature characterisation model (155) in this case consists of a single neural network trained to carry out multiple tasks i.e., prediction of both lung cancer risk and lung cancer invasiveness.
Referring back to
Depending on the imaging modality, imaging protocol, and image features of interest, the set of findings in an image dataset is presented with the most clinically relevant findings displayed first. In this way, a radiologist using the system is not always required to review all system findings e.g., in the case that the most relevant finding determines the clinical recommendation issued by the radiologist, or can prioritise the review of certain findings over others that are less clinically relevant.
In an embodiment of the invention, input is provided to the workflow optimisation circuit and the top N most clinically relevant features are output by the output generator, where N is configurable by the user.
The central element of the workflow optimisation circuit (175) is a workflow optimisation model (180), which carries out the logic of the circuit (175). It takes as input the list of characterised detections (160) and any required configuration parameters (115) which are set according to the clinical setting. In a preferred embodiment of the invention the input to the optimisation model further comprises information on the location of the detected features.
The main output of the optimisation model is a list of the detected features, of all relevant categories, sorted according to their clinical relevance. Preferably, the output is at least one of: a display of medical images sorted according to the relevance criteria, a report containing information about the medical images sorted according to the relevance criteria. In some examples of the workflow optimisation circuit, the optimisation model also outputs a clinical recommendation guidance e.g., recommendations for the patient's clinical pathway. Recommendation guidance is typically based on the corresponding clinical guidelines applicable to each of the categories of features detected. In a preferred embodiment of the invention a clinical recommendation is generated for at least the most clinically relevant feature in at least one of the feature categories.
The workflow optimisation model (180) may take a variety of forms depending on the embodiment of the invention. For example, the logic of the workflow optimisation model may be rule based, or use a more complex computation such as a machine learning model. These are illustrated in the following example embodiments:
In this embodiment of the workflow optimisation model (180), only a single type of feature category and with a single characterisation is given as input, for instance lung nodules with associated risk of malignancy. The list of detected lung nodules is then sorted by their associated malignancy risk. Other examples of appropriate feature characterisations in this case include: regions of pulmonary fibrosis, characterised by the specific type of fibrosis; regions of emphysema, characterised by the severity of the emphysema. In some versions of the embodiments, configuration parameters (115) supplied to the circuit are used to refine the output of the workflow optimisation model e.g., based on thresholds for minimum diameter of the nodule, so as to exclude the smallest detected nodules which are not associated with clinical risk.
In addition to the sorted list of nodule detections according to malignancy risk, in this embodiment the workflow optimisation model provides a patient-wise clinical recommendation considering all detected and characterised nodules. For example, the appropriate follow-up for the nodule with the highest risk of malignancy, according to the relevant clinical guidelines.
In this embodiment of the workflow optimisation model (180), multiple feature characterisations are provided as input, all belonging to a single category of feature. The workflow optimisation model (180) then computes an ordering of the findings by considering all feature characterisations simultaneously. For example, lung nodules are detected by the feature detection circuit (125), and risk of malignancy and predicted invasiveness (e.g., according to their histological type) of a potential malignant nodule are given as characterisation of each lung nodule by the feature characterisation circuit (150).
In an example of this embodiment, where the nodule invasiveness potential is codified as a probability of invasiveness within a pre-specified time period t, the clinical relevance of each of the lung nodule findings can be computed as:
That is, the relevance of each lung nodule finding is scored as the probability of malignancy, as estimated form the medical image, multiplied by the probability of the lung nodule invading surrounding tissue within as time period t.
In other example embodiments of the lung nodule characterised by malignancy risk and invasiveness, where the invasiveness is extracted from the histological type predicted from the image, the relevance of the detected nodules is computed as:
Here, the relevance of each lung nodule finding is scored as the probability of malignancy, as computed from the medical image, multiplied by the probability of the lung nodule being of a histological type considered invasive (e.g., Invasive Adenocarcinoma) as opposed to not a non-invasive type (e.g., Adenocarcinoma in Situ).
In this embodiment of the workflow optimisation model (180) multiple feature characterisations are provided as input for each of multiple categories of feature. Therefore, the logic behind the organisation of findings may be more complex. As an example of this embodiment, the feature detection circuit (125) detects different features, such as lung nodules as well as pulmonary fibrosis and lung emphysema. The feature characterisation circuit (150) then characterises the feature for a specific criterion, for example: the risk of malignancy for the lung nodules, type of fibrosis, and severity of emphysema. In this example embodiment, the relevance of lung nodules is determined, as before, from their malignancy risk. However, the workflow optimisation circuit (175) needs to prioritise over the categories of findings, that is, between the reporting of the fibrosis, lung nodules, and emphysematous lung tissue. In this example embodiment of the workflow optimisation model, the emphysema finding may take the lowest priority over the remaining categories, while the top category is decided between the fibrosis and lung nodules such that the fibrosis category is prioritised over the lung nodule category if the probability of a concerning type of fibrosis, e.g., IPF, is higher than the highest probability of malignancy:
Here, N is the list of detected lung nodules.
In another example of this embodiment where there is a more extensive list of categories and characterisations for each category, the organisation model is based on a machine learning model such as a neural network trained to rank the clinical relevance of the various characterised findings.
The training protocol for some examples of this model is shown in
Preferably, the training dataset is constructed by collecting historic clinical recommendations from a plurality of radiology reports. This required that a ranking of the clinical relevance of each finding can be discerned from each report. In some examples of the training method, additional training samples are gathered by sampling synthetic sets of characterised feature detections from an appropriate prior distribution and asking radiologists to annotate the ranking of the findings according to their clinical relevance.
In this embodiment of the workflow optimisation model, a patient-wise clinical recommendation is included in the output (195). Here, the workflow optimisation model (180) is also trained to issue a patient-wise clinical recommendation form the same set of characterised feature detections of the form (160). This task is non-trivial given the non-linear dependence of pathologies on certain features, making a machine learning model trained for this task the most appropriate way to provide recommendation guidance to the radiologist using the system.
In this embodiment of the invention the workflow optimisation model determines, preferably automatically the clinical relevance of an image finding relative to other classes of image findings based, where each image finding is associated to at least one characterisation
An alternative example training protocol is shown in
The final element of the workflow optimisation circuit (175) of
The various embodiments of the workflow optimisation model (180) and the output generator (185) are not mutually exclusive; thus, some embodiments of the workflow optimisation circuit (175) are constructed from different embodiments of the building blocks. As an example, an embodiment of the workflow optimisation circuit (175) is illustrated in
Embodiments of the output generator are further described next.
In this embodiment of the output generator (185), the results from the workflow optimisation model (180) are preferably presented to a user in the form of a DICOM secondary capture stack, as illustrated in
In an example of this embodiment, the detected features preferably correspond to lung nodules, and they are characterised by a parameter such as risk of malignancy. Each characterised feature reported by the feature characterisation circuit (150) is saved as a single DICOM file of secondary capture type, containing a snapshot of the finding along with risk of malignancy characterisation. This (non-image) information may be saved as metadata associated with the DICOM file, or displayed as an overlay on the snapshot (810) alongside any relevant metadata associated with the patient and the medical image dataset (805).
If multiple findings are reported in a single image dataset, the individual DICOM files associated to each finding are saved as part of the same DICOM series, along with their associated metadata modified to ensure they are loaded in the DICOM viewer according to ordering defined by the workflow optimisation model (180). In this way, a radiologist can quickly see the most relevant findings first, and in some cases, may be able to make a clinical decision without needing to review all automatic findings in the results. Multiple findings are in this case referenced on each individual secondary capture, allowing for a quick overview of all findings (820).
Additional information can be integrated into the DICOM secondary capture to assist the radiologist's workflow. In a further example of this embodiment, each detected feature is labelled and/or outlined in the secondary capture images. Finally, in an example of the workflow optimisation model (180) where diagnostic guidance is also produced, it can also be displayed on the secondary capture, or all secondary captures in the stack if multiple feature detections are reported. For example, information to guide interpretation of a malignancy risk score can be overlayed (815).
Having reviewed a medical image dataset a radiologist will typically submit their findings to a reporting system in the form of a report. Preferably, the output generator is configured to fill structured reports following the feature ordering produced by the workflow optimisation model. In this embodiment of the output generator (185), results from the workflow optimisation model (180) are pre-filled into a structured report which can be reviewed and edited by the radiologist before submitting to the reporting system. Pre-filling of results allows users to review any findings that will be reported, remove findings according to their own assessment of the image dataset, e.g., by removing findings that are not clinically relevant such as false positives of the feature detection circuit (125), add findings such as the false negatives of the feature detection circuit (125), and add any other information needed to complete the report. In this way, a major component of the cross-checking and summarising of results is automated, making the radiologist's workflow faster and less labour intensive, and reducing the chances of clinical errors.
In some examples of this embodiment, the report includes all findings in the medical image dataset. In other examples, only the most relevant findings are pre-filled into the report according to the configuration of the workflow optimisation circuit (175).
In a preferred embodiment of the invention, the workflow optimisation circuit, consists of a workflow optimisation model and an output generator, which receives a list of at least one category of features found in a medical scan image. Preferably, the output generator also receives at least one clinical characterisation of each of the medical scan images, and orders the features in the medical scan image in according to their clinical relevance.
A preferred embodiment of the invention comprises a combination of a feature detector circuit, feature characterisation circuit and workflow optimisation circuit.
Preferably, the workflow optimisation model determines the clinical relevance of a type of image finding based on more than one feature characterisation (i.e. embodiment 2 of the workflow optimisation model). Preferably, this determination is performed automatically.
This invention targets the optimization of a radiology workflow once the radiologist has started to read a medical image. This contrasts with the rather common systems that help decide which medical images should be reviewed first or even reviewed at all. The innovation mainly builds on top of systems that simultaneously detect lesions and assess their risk of disease from medical images. In particular, this invention adds the workflow optimization circuit which transforms the output of the detection and characterization circuits into an organized workflow for the radiologist to follow when reviewing the medical image in a way that they make a more efficient use of the reading time.
The invention is suitable for a system that can be installed in the radiology department of a hospital, clinic, healthcare facility or dedicated radiology or tele-radiology service, in help radiologist increase their efficiency when processing long backlogs of medical images. This can be, for instance, a software product that is installed directly in the radiology workstations, or delivered through standard radiology software such as PACS viewers.
The present invention has been described with reference to the accompanying drawings. However, it will be appreciated that the present invention is not limited to the specific examples herein described and as illustrated in the accompanying drawings. Furthermore, because the illustrated embodiments of the present invention may for the most part, be implemented using electronic components and circuits known to those skilled in the art, details will not be explained in any greater extent than that considered necessary as illustrated above, for the understanding and appreciation of the underlying concepts of the present invention and in order not to obfuscate or distract from the teachings of the present invention.
The invention may be implemented in a computer program for running on a computer system, at least including code portions for performing steps of a method according to the invention when run on a programmable apparatus, such as a computer system or enabling a programmable apparatus to perform functions of a device or system according to the invention.
A computer program is a list of instructions such as a particular application program and/or an operating system. The computer program may for instance include one or more of: a subroutine, a function, a procedure, an object method, an object implementation, an executable application, an applet, a servlet, a source code, an object code, a shared library/dynamic load library and/or other sequence of instructions designed for execution on a computer system. Therefore, some examples describe a non-transitory computer program product having executable program code stored therein for automated contouring of cone-beam CT images.
The computer program may be stored internally on a tangible and non-transitory computer readable storage medium or transmitted to the computer system via a computer readable transmission medium. All or some of the computer program may be provided on computer readable media permanently, removably or remotely coupled to an information processing system. The tangible and non-transitory computer readable media may include, for example and without limitation, any number of the following: magnetic storage media including disk and tape storage media; optical storage media such as compact disk media (e.g., CD ROM, CD R, etc.) and digital video disk storage media; non-volatile memory storage media including semiconductor-based memory units such as FLASH memory, EEPROM, EPROM, ROM; ferromagnetic digital memories; MRAM; volatile storage media including registers, buffers or caches, main memory, RAM, etc.
A computer process typically includes an executing (running) program or portion of a program, current program values and state information, and the resources used by the operating system to manage the execution of the process. An operating system (OS) is the software that manages the sharing of the resources of a computer and provides programmers with an interface used to access those resources. An operating system processes system data and user input, and responds by allocating and managing tasks and internal system resources as a service to users and programs of the system.
The computer system may for instance include at least one processing unit, associated memory and a number of input/output (I/O) devices. When executing the computer program, the computer system processes information according to the computer program and produces resultant output information via I/O devices.
In the foregoing specification, the invention has been described with reference to specific examples of embodiments of the invention. It will, however, be evident that various modifications and changes may be made therein without departing from the scope of the invention as set forth in the appended claims and that the claims are not limited to the specific examples described above.
Those skilled in the art will recognize that the boundaries between logic blocks are merely illustrative and that alternative embodiments may merge logic blocks or circuit elements or impose an alternate decomposition of functionality upon various logic blocks or circuit elements. Thus, it is to be understood that the architectures depicted herein are merely exemplary, and that in fact many other architectures can be implemented which achieve the same functionality.
Any arrangement of components to achieve the same functionality is effectively ‘associated’ such that the desired functionality is achieved. Hence, any two components herein combined to achieve a particular functionality can be seen as ‘associated with’ each other such that the desired functionality is achieved, irrespective of architectures or intermediary components. Likewise, any two components so associated can also be viewed as being ‘operably connected,’ or ‘operably coupled,’ to each other to achieve the desired functionality.
Furthermore, those skilled in the art will recognize that boundaries between the above described operations merely illustrative. The multiple operations may be combined into a single operation, a single operation may be distributed in additional operations and operations may be executed at least partially overlapping in time. Moreover, alternative embodiments may include multiple instances of a particular operation, and the order of operations may be altered in various other embodiments.
However, other modifications, variations and alternatives are also possible. The specifications and drawings are, accordingly, to be regarded in an illustrative rather than in a restrictive sense.
In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word ‘comprising’ does not exclude the presence of other elements or steps then those listed in a claim. Furthermore, the terms ‘a’ or ‘an,’ as used herein, are defined as one or more than one. Also, the use of introductory phrases such as ‘at least one’ and ‘one or more’ in the claims should not be construed to imply that the introduction of another claim element by the indefinite articles ‘a’ or ‘an’ limits any particular claim containing such introduced claim element to inventions containing only one such element, even when the same claim includes the introductory phrases ‘one or more’ or ‘at least one’ and indefinite articles such as ‘a’ or ‘an.’ The same holds true for the use of definite articles. Unless stated otherwise, terms such as ‘first’ and ‘second’ are used to arbitrarily distinguish between the elements such terms describe. Thus, these terms are not necessarily intended to indicate temporal or other prioritization of such elements. The mere fact that certain measures are recited in mutually different claims does not indicate that a combination of these measures cannot be used to advantage.
