Patent Application
20160232295
The invention relates to a system and a method for enabling review of a medical image and for generating a report on the review of the medical image. The invention further relates to a workstation and an imaging apparatus comprising said system, and to a computer program product for causing a processor system to perform said method.
It is common in the medical field for users such as radiologists to review medical images and to document the review by generating a report. For example, a clinician may have a clinical query. In order to obtain an answer to such a clinical query, the clinician may issue a radiology request to the radiologist. In response, the radiologist may generate an imaging request for imaging a patient, review the resulting medical image(s), and generate a report comprising medical findings, conclusions, and other medical information. The radiologist or another medical user may be supported by a system in generating a report. Such a system may enable the user to generate the report by typing or dictating the medical information as free text, i.e., substantially unconstrained in content or structure.
It is known to obtain such reporting of medical information in a more standardized manner. For example, a publication titled “Medical Imaging on the Semantic Web: Annotation and Image Markup” by Daniel L. Rubin et al., AAAI Spring Symposium: Semantic Scientific Knowledge Integration, page 93-98, 2008, describes an image annotation tool which enables a user to provide a context for an annotation using a drop-down box. The image annotation tool uses the user-specified context in inferring the data fields that the user should collect for that annotation context. Accordingly, the image annotation tool requires users to record different types of information depending on the context.
It is also known to use macros to obtain reporting of medical information in a more standardized manner. In particular, such macros enable structured and/or codified descriptions to be generated of the medical information provided by the user. Here, the term ‘structured’ refers to the medical information being organized in accordance with standards or rules, such as those provided by radiology reporting guidelines. Moreover, the term ‘codified’ refers to the medical information being coded in accordance with similar or the same standards or rules to obtain the description. For example, the reporting guidelines may provide a standardized nomenclature which causes the medical information of “organ =liver” to be coded so as to provide a codified description being “organ=RID56”.
The user may manually select such a macro. It is also known to suggest macros to the user based on the user starting to type free text and matching the free text to one or more macros. The macros may be suggested via auto-completion of the free text.
However, even if a system provides such macros to obtain a more standardized reporting of medical information, said standardized reporting still poses a significant burden to the user as the user has to manually provide the medical information, e.g., via the aforementioned typing in of the medical information as free text.
It would be advantageous to provide a system or method which enables standardized reporting of medical information at a reduced burden to the user.
To better address this concern, a first aspect of the invention provides a system for enabling review of a medical image, comprising:
In a further aspect of the invention, a workstation and imaging apparatus is provided comprising the system set forth.
In a further aspect of the invention, a method is provided for enabling review of a medical image, comprising:
In a further aspect of the invention, a computer program product is provided comprising instructions for causing a processor system to perform the method set forth.
The aforementioned measures provide an image interface which obtains the medical image, e.g., from an internal or external storage medium. The medical image may be from a medical study constituted by a plurality of medical images. Moreover, a user interface subsystem is provided for interacting with a user. The user interface subsystem enables the user to interactively apply one or more graphical elements to the medical image during the review of the medical image. Here, the term ‘apply to’ refers to the graphical elements being placed on, overlaid over or otherwise attached to the medical image when the medical image is displayed to the user during the review. For example, the user interface subsystem may enable the user to select a graphical element by operating a mouse, and subsequently drag-and-drop the selected graphical element onto the medical image. Examples of such graphical elements include markers, annotations and parts of measurement tools. As a result, an application of one or more graphical elements to the medical image is obtained, as well as application data which represents the application of the one or more graphical element to the medical image. In other words, the application data expresses the characteristics of said application. For example, the application data may indicate which type of graphical element is applied to the medical image, its position relative to the medical image, etc.
Furthermore, a reporting subsystem is provided. The reporting subsystem is enabled to access a medical database which comprises medical information items. The medical information items represent computer-readable medical reference information such as text passages or schematic drawings from a medical textbook or medical guideline. Here, the term ‘representing’ refers to the medical reference information being directly expressed by the medical information item, e.g., by the medical information item comprising the medical reference information as a text passage or schematic drawing, or by such medical reference information being derivable from the medical information item, e.g., by decoding. The reporting subsystem retrieves a medical information item from the medical database. In selecting the medical information item amongst other medical information items, the reporting subsystem makes use of the application data which indicates the application of the one or more graphical elements to the medical image, as well as association data which enables the reporting subsystem to associate the application of the one or more graphical elements to at least one of the medical information items in the medical database. Finally, the reporting subsystem generates a report based on the retrieved medical information item.
The aforementioned measures have the effect that the system is enabled to generate a report without requiring the user to manually input medical information for the report. The present invention is at least in part based on the insight that the application of one or more graphical elements during the review of the medical image is indicative of what type of medical information the user is seeking to report during the review of the medical image. For example, if the user applies graphical elements from a lesion measurement tool to the medical image, it is likely that the user wishes to obtain and report medical information relating to the staging of such lesions. The present invention enables such medical information to be automatically obtained, namely based on association data linking the lesion measurement tool to relevant medical information items in the medical database.
The user therefore does not need manually input the medical information. Advantageously, the system reduces or entirely avoids the burden posed to the user when reporting on a review of a medical image. At the same time, by avoiding or reducing manual input of medical information by the user, the medical information is reported inherently in a standardized way, i.e., as determined by the contents of the association data and the medical information items in the medical database rather than at the hand of the user's discretion. Advantageously, a more structured reporting is obtained compared to free text input.
Optionally, the medical information item is associated with a codified version of the medical reference information, and the reporting subsystem is arranged for using the codified version of the medical reference information in generating the report.
The reporting subsystem is thus enabled to obtain a codified version of the medical reference information based on the medical information item. The codified version of the medical reference information adheres (more) to standards or rules, such as those provided by radiology reporting guidelines, than the medical reference information itself. For example, the medical reference information may be provided for the purpose of informing the user about a medical condition, whereas the codified version of the medical reference information may be specifically provided for the purpose of reporting on the medical condition. By using the codified version of the medical reference information to generate the report, a more standardized reporting on the medical image is obtained. Advantageously, a same standardized nomenclature and/or reporting structure may be obtained as if the system were to use a macro to generate a codified description of medical information provided by the user, yet without the user having to manually provide said medical information.
Optionally, the user interface subsystem is arranged for displaying the medical reference information as a selectable graphical element to the user, and the reporting subsystem is arranged for using the codified version of the medical reference information in generating the report based on the user selecting the selectable graphical element.
The medical reference information is thus presented onscreen as a graphical element such as a textbox, an inserted graphic or other type of overlay. The graphical element is a selectable graphical element in that the user may select the graphical element, e.g., by operating a mouse. The reporting subsystem is arranged for, upon selection of the graphical element by the user, using the codified version of the medical reference information in generating the report. Accordingly, the user is enabled to decide whether or not the report should be generated based on the retrieved medical information item, namely by selecting the graphical element or by refraining from said selecting. Such selecting of the graphical element signals to the system that the report should be generated based on the retrieved medical information item. Moreover, upon selecting the graphical element, the reporting subsystem uses the codified version of the selected medical reference information in generating the report. This aspect of the present invention is based on the insight that medical reference information is typically provided for the purpose of informing the user about a medical condition, whereas the codified version of the medical reference information is typically provided for the purpose of reporting on the medical condition. Accordingly, during it may be desirable to display the medical reference information to the user during the review, i.e., to enable easily determining whether the retrieved medical information item is applicable to the present case, while subsequently including its codified version in the report.
Optionally, the reporting subsystem is arranged for using a macro to generate the codified version of the medical reference information from the medical reference information. By using a macro, the codified version of the reference information can be generated dynamically, e.g., at a time when the codified version is needed. It is therefore not needed to pre-generate and store the codified version of the medical reference information. Advantageously, the use of a macro enables codifying other information together with the medical reference information. For example, the medical reference information may be supplemented with patient-specific information when generating the report.
Optionally, the medical information item comprises the macro or a reference to the macro. In order to codify different types of medical information items, different macros may be required. By including the macro or a reference thereto in the medical information item itself, the system can easily obtain the required macro for a particular medical information item. Advantageously, it is not needed to otherwise search for a macro.
Optionally, the medical information items are cross-referenced via hyperlinks, and the user interface subsystem is arranged for enabling the user to interactively browse to another medical information item by selecting a hyperlink in the selectable graphical element. The user is thus enabled to browse from an initially retrieved medical information item to cross-referenced, and thus typically related, medical information items. Accordingly, if the system inadvertently retrieves a medical information item which is of lesser relevance, the user can interactively browse to a more relevant medical information item. Accordingly, if the initially retrieved medical information item is considered as a suggestion from the system, the user is enabled to fine-tune the suggestion before signaling the system to use the fine-tuned medical information item in generating the report. Advantageously, the user is enabled to influence and improve the reporting by the system in an intuitive manner.
Optionally, the selectable graphical element is a selectable overlay, and the user interface subsystem is arranged for displaying the medical reference information in the form of text or graphics in the selectable overlay. The medical reference information is thus displayed in the form of an overlay over the medical image. Such display is intuitive since it adheres well to current user interface paradigms. Advantageously, the overlay may be used to enable the user to select specific parts of the medical reference information, such as a text passage or a part of a schematic drawing, which is to be used in generating the report.
Optionally, the one or more graphical elements provide a measurement tool for enabling the user to perform a measurement on the medical image using the user interface subsystem, and the association data is constituted at least in part by measurement data representing the measurement. The reporting subsystem thus uses the measurement data to identify the medical information item. This aspect of the invention is based on the insight that the outcome of a measurement on the medical image may more specifically indicate a current medical or pathological condition. Accordingly, the reporting subsystem is enabled to retrieve a more relevant medical information item, namely one which takes into account the current medical or pathological condition. For example, if the user applies graphical elements from a lesion measurement tool to the medical image, it is likely that the user wishes to obtain and report medical information relating to the staging of such lesions. The measurement of the lesion may then be used to retrieve a medical information item for the stage of the lesion in the medical image. For example, in the case the lesion exceeds 5 cm in diameter, the retrieved medical information item may be for a so-termed T2 or T3a stage.
Optionally, the reporting subsystem is arranged for using the measurement data in generating the report. The reporting subsystem thus automatically uses the measurement data to generate the report, e.g., so as to include the measurement in the report, thereby avoiding the need for the user to manually report the measurement. Advantageously, in case the reporting subsystem uses a macro to generate the codified version of the medical reference information from the medical reference information, the macro may use the measurement data as input, thereby providing a codified description which takes into account the measurement. Advantageously, a more complete report is obtained.
Optionally, the association data is constituted at least in part by a plurality of rules each linking a respective one of the one or more graphical elements to a respective one of the medical information items. Such a set of rules is well suited to link the application of the one or more graphical elements to the medical information item.
Optionally, the application data represents one or more of the group of: a type of the one or more graphical elements, a position of the one or more graphical elements in the medical image, and an image structure marked by the one or more graphical elements. The type of graphical element may indicate which medical information items are of relevance and which are not. For example, a lesion measurement tool may indicate that medical information items relating to lesions are of relevance. Similarly, a position of the one or more graphical elements in the medical image and an image structure marked by the one or more graphical elements may indicate medical information items of relevance. Advantageously, the reporting subsystem is enabled to retrieve a more relevant medical information item.
Optionally, the one or more graphical elements comprise one or more of the group of: a marker, an annotation and a measurement tool.
It will be appreciated by those skilled in the art that two or more of the above-mentioned embodiments, implementations, and/or aspects of the invention may be combined in any way deemed useful.
Modifications and variations of the workstation, the imaging apparatus, the method, and/or the computer program product, which correspond to the described modifications and variations of the system, can be carried out by a person skilled in the art on the basis of the present description.
A person skilled in the art will appreciate that the method may be applied to multi-dimensional image data, e.g. to two-dimensional (2-D), three-dimensional (3-D) or four-dimensional (4-D) images. A dimension of the multi-dimensional image data may relate to time. For example, a 3-D image may comprise a time domain series of 2-D images.
The medical image may be acquired by various acquisition modalities such as, but not limited to, standard X-ray Imaging, Computed Tomography (CT), Magnetic Resonance Imaging (MRI), Ultrasound (US), Positron Emission Tomography (PET), Single Photon Emission Computed Tomography (SPECT), and Nuclear Medicine (NM).
The invention is defined in the independent claims. Advantageous yet optional embodiments are defined in the dependent claims.
These and other aspects of the invention are apparent from and will be elucidated with reference to the embodiments described hereinafter. In the drawings,
It should be noted that items which have the same reference numbers in different Figures, have the same structural features and the same functions, or are the same signals. Where the function and/or structure of such an item has been explained, there is no necessity for repeated explanation thereof in the detailed description.
The system 100 further comprises a user interface subsystem 140. In general, the user interface subsystem 140 may enable the user to interact with the system 100 via a user interface established by the user interface subsystem 140 on a display 010. According to the present invention, the user interface subsystem 140 is arranged for enabling a user to apply one or more graphical elements to the medical image during the review of the medical image, thereby establishing application data representing application of the one or more graphical elements to the medical image.
The user input 150 may be arranged for receiving user input data 022 from a user input device 020 such as a keyboard, mouse or touch sensitive surface. Moreover, the display processor 160 may be arranged for receiving the user input data 022 from the user input 150 via an internal exchange of messages 154. The display processor 160 may be arranged for presenting a user interface on the display 010. For that purpose, the display processor 160 may output display data 162 to the display 010. The user interface subsystem 140 may display the medical image 042 on the display 010, e.g., to enable the user to apply the one or more graphical elements to the medical image 042. For that purpose, the display processor 160 is shown to receive the medical image 042 from the image interface 120.
The system 100 further comprises a reporting subsystem 180. The reporting subsystem 180 is arranged for accessing a medical database 060 comprising medical information items which each represent medical reference information.
An operation of the system 100 may be briefly explained as follows. The image interface 120 obtains the medical image 042. The user interface subsystem 140 enables the user to apply the one or more graphical elements to the medical image 042 during the review of the medical image. The user interface subsystem 140 thereby establishes application data representing the application of the one or more graphical elements to the medical image. The reporting subsystem 180 accesses the medical database 060 which comprises the medical information items. The reporting subsystem 180 then retrieves the medical information item 062 from the medical database based on the application data and the association data. Finally, the reporting subsystem 180 generates the report 182 based on the retrieved medical information item 062. For example, the reporting subsystem 182 may directly include the medical information item 062 in the report 182.
The method 200 comprises, in a first step titled “OBTAINING MEDICAL IMAGE”, obtaining 210 the medical image. The method 200 further comprises, in a second step titled “OBTAINING APPLICATION DATA”, enabling 220 a user to apply one or more graphical elements to the medical image during the review of the medical image, thereby establishing application data representing application of the one or more graphical elements to the medical image. The method 200 further comprises, in a third step titled “ACCESSING MEDICAL DATABASE”, accessing 230 a medical database comprising medical information items which each represent medical reference information. The method 200 further comprises, in a fourth step titled “RETRIEVING MEDICAL INFORMATION ITEM FROM MEDICAL DATABASE”, retrieving 240 a medical information item from the medical database based on the application data and association data, the association data linking the application of the one or more graphical elements to the medical information item. The method 200 further comprises, in a fifth step titled “GENERATING REPORT BASED ON MEDICAL INFORMATION ITEM”, generating 250 a report based on the medical information item. It is noted that the above steps may be performed in any suitable order. In particular, the third step of accessing 230 the medical database may be performed in parallel with the earlier steps, i.e., the first step 210 and the second step 220.
An operation of the system may be further explained as follows.
To select the medical information item, the reporting subsystem makes use of association data. The association data may be constituted at least in part by a plurality of rules each linking the application of a respective one of the one or more graphical elements to a respective one of the medical information items. For example, the one or more graphical elements may comprise one or more of the group of: a marker, an annotation and a measurement tool. The association data may then represent a type of the one or more graphical elements in that a first rule links application of a marker to a different medical information item than application of a measurement tool. The rules may also distinguish between different types of markers, annotations and measurement tools. Additionally or alternatively, the rules may take into account a position of the one or more graphical elements in the medical image. The position may be a relative position between the multiple graphical elements, e.g., representing a measurement in the medical image. The position may also be an absolute position in the medical image, e.g., representing an anatomical location in the medical image. Accordingly, the rules may link different measurements and/or different anatomical locations to different medical information items. Additionally or alternatively, the rules may take into account an image structure marked by the one or more graphical elements. The system may detect which type of image structure is marked, e.g., using a region of interest detection technique. Alternatively, the one or more graphical elements may be associated with the image structure. As such, the application of the one or more graphical elements may imply or suggest that a particular type of image structure is marked.
It will be appreciated that the association data may also take various other forms. For example, the association data may be constitutes by the application data itself In particular, instead of directly linking a respective one of the one or more graphical elements to a respective one of the medical information items, the reporting subsystem may take various data into account which is available to the system, which together constitutes the association data and which indirectly rather than directly identifies the medical information item to be retrieved. For example, the reporting subsystem may take contextual information into account such as metadata of the medical image, a patient record of a patient, a computer readable version of the clinical query and similar types of information. Together with the application data representing the application of the one or more graphical elements, the reporting subsystem is enabled to determine which medical information item is to be retrieved from the medical database, e.g., using known per se reasoning techniques.
“A: 6.1 cm” being included in the report 182. It will be appreciated that the macro 080 may also codify various further information relevant to the review of the medical image.
It is noted that the second overlay 412 may also be directly presented to the user after application of the virtual ruler 310 to the medical image 042, thereby constituting another example of the user interface subsystem displaying medical reference information as a selectable graphical element to the user. Moreover, the medical information item which represents the medical reference information may be hyperlinked, thereby enabling the user to browse through the medical textbook by selecting parts of the medical reference information 412. Accordingly, the overlay may progressively disclose relevant textbook information. Moreover, the hyperlinks may be dynamically established based on a current clinical context, such as which body part or organ is displayed. Such contextual information may be obtained from, e.g., metadata of the medical image or a patient record.
It will be appreciated that the present invention may enable a user to obtain display of medical reference information and to conveniently include a codified version of the medical reference information to a report, namely by selecting the displayed medical reference information. Moreover, the displayed medical reference information may be hyperlinked, thereby enabling the user to refine which medical reference information is used as a basis for generating the codified version of the reference information.
It will be appreciated that the invention also applies to computer programs, particularly computer programs on or in a carrier, adapted to put the invention into practice. The program may be in the form of a source code, an object code, a code intermediate source and an object code such as in a partially compiled form, or in any other form suitable for use in the implementation of the method according to the invention. It will also be appreciated that such a program may have many different architectural designs. For example, a program code implementing the functionality of the method or system according to the invention may be sub-divided into one or more sub-routines. Many different ways of distributing the functionality among these sub-routines will be apparent to the skilled person. The sub-routines may be stored together in one executable file to form a self-contained program. Such an executable file may comprise computer-executable instructions, for example, processor instructions and/or interpreter instructions (e.g. Java interpreter instructions). Alternatively, one or more or all of the sub-routines may be stored in at least one external library file and linked with a main program either statically or dynamically, e.g. at run-time. The main program contains at least one call to at least one of the sub-routines. The sub-routines may also comprise function calls to each other. An embodiment relating to a computer program product comprises computer-executable instructions corresponding to each processing step of at least one of the methods set forth herein. These instructions may be sub-divided into sub-routines and/or stored in one or more files that may be linked statically or dynamically. Another embodiment relating to a computer program product comprises computer-executable instructions corresponding to each means of at least one of the systems and/or products set forth herein. These instructions may be sub-divided into sub-routines and/or stored in one or more files that may be linked statically or dynamically.
The carrier of a computer program may be any entity or device capable of carrying the program. For example, the carrier may include a storage medium, such as a ROM, for example, a CD ROM or a semiconductor ROM, or a magnetic recording medium, for example, a hard disk. Furthermore, the carrier may be a transmissible carrier such as an electric or optical signal, which may be conveyed via electric or optical cable or by radio or other means. When the program is embodied in such a signal, the carrier may be constituted by such a cable or other device or means. Alternatively, the carrier may be an integrated circuit in which the program is embedded, the integrated circuit being adapted to perform, or used in the performance of, the relevant method.
It should be noted that the above-mentioned embodiments illustrate rather than limit the invention, and that those skilled in the art will be able to design many alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. Use of the verb “comprise” and its conjugations does not exclude the presence of elements or steps other than those stated in a claim. The article “a” or “an” preceding an element does not exclude the presence of a plurality of such elements. The invention may be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In the device claim enumerating several means, several of these means may be embodied by one and the same item of hardware. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.
| Number | Date | Country | Kind |
|---|---|---|---|
| 13186303.7 | Sep 2013 | EP | regional |
| Filing Document | Filing Date | Country | Kind |
|---|---|---|---|
| PCT/EP2014/070471 | 9/25/2014 | WO | 00 |
