The invention relates to a system and method for visualizing image data. The invention further relates to a server comprising the system and to a client device enabling a user to interact with the server. The invention further relates to a computer program product comprising instructions for causing a processor system to perform the method.
Image data may be presented in various ways to a user. For example, in case the image data is 3D image data, the 3D image data may be presented by using a volume rendering technique to generate a 2D projection of the 3D image data. In another example, if the image data is high dynamic range (HDR) image data such as a 2D Computed Tomography (CT) slice, the HDR image data may be presented to the user by using a window level/width rendering to generate a standard dynamic range (SDR) view of the HDR image data. In general, such presentations of the image data are referred to as views of the image data, and said generating of the views of the image data is referred to as view generation. Several of such techniques are known from the field of image processing.
It may be desirable to present a plurality of different views of the image data simultaneously to a user. For example, in case the image data is 3D medical image data of a patient, it may be desirable to simultaneously present different views of the 3D medical image data so as to allow a clinician to quickly obtain an overview of the 3D medical image data. Examples of such views are, e.g., a coronal view, a sagittal view and an axial view. Such a simultaneous presentation of a plurality of different views is henceforth referred to as a gallery of the plurality of different views.
It may be desirable to enable the user to adjust a view generated from the image data. US 20100049740 A1 describes a workflow management system for processing medical image data generated by a medical imaging device. It is said that a so-termed scene contains metadata generated from one workflow stage. The metadata in the scene is generated by the image processing operations of the workflow stage, and allows, when applying the scene to the medical image data, to produce a set of medical image views.
When reviewing the medical image views reproduced from the scene, a user may adjust these image views by making updates to the image processing parameters (metadata) contained in the scene. Afterwards, the updated image processing parameters can be saved to the scene to replace the previously stored image processing parameters. The newly updated scene can also be stored in the workflow scene to replace the old scene.
The above system thus allows a user to adjust a particular medical image view by updating the metadata element associated with the medical image view.
A problem of the above system is that it is too inconvenient for a user to obtain a desired view of the image data.
It would be advantageous to provide a system or method which enables a user to more conveniently obtain a desired view of image data.
To better address this concern, a first aspect of the invention provides a system for visualizing image data, comprising:
an image interface for accessing the image data;
a display processor for i) generating a plurality of different views of the image data based on one or more view parameters defining each one of the plurality of different views, and ii) generating a gallery comprising the plurality of different views; and
a user interface subsystem for j) enabling the user to select one the plurality of different views from the gallery, thereby obtaining a selected view, and jj) enabling the user to adjust the selected view by adjusting a view parameter of the selected view, thereby obtaining an adjusted view parameter;
wherein the display processor is arranged for updating the gallery by adjusting at least another one of the plurality of different views based on the adjusted view parameter, thereby establishing an adjusted other view in the gallery.
In a further aspect of the invention, a method is provided for visualizing image data, comprising:
accessing the image data;
generating a plurality of different views of the image data based on one or more view parameters defining each one of the plurality of different views;
generating a gallery comprising the plurality of different views;
enabling the user to select one the plurality of different views from the gallery, thereby obtaining a selected view;
enabling the user to adjust the selected view by adjusting a view parameter of the selected view, thereby obtaining an adjusted view parameter;
updating the gallery by adjusting at least another one of the plurality of different views based on the adjusted view parameter, thereby establishing an adjusted other view in the gallery.
In a further aspect of the invention, a computer program product is provided comprising instructions for causing a processor system to perform the method set forth.
The aforementioned measures enable the user to obtain a visualization of the image data by providing the user with a plurality of different views of the image data. For that purpose, the image data is accessed, e.g., from an internal or external location. Several presentations of the image data are generated, thereby obtaining a plurality of different views. Each view is defined at least in part by one or more view parameters. The views differ from each other, e.g., by having different view parameter values and/or by being generated from different parts of the image data, e.g., from different objects comprised in the image data. The plurality of different views is simultaneously presented by being included in a gallery.
The user can select one of the different views from the gallery, e.g., by clicking on the respective view with an input device. The user can adjust the selected view by adjusting one of the view parameters. As a result, the selected view is adjusted, thereby obtaining an at least somewhat different view of the image data. Based on the adjusting of the selected view by the user, at least another one of the views is adjusted automatically. The gallery is then updated to reflect the selected view being adjusted by the user and the other view(s) being adjusted automatically. Hence, the user is presented with an updated gallery.
The aforementioned measures have the effect that the user is presented with gallery comprising a plurality of different views of the image data. Advantageously, compared to showing a single view of the image data, it is more likely that the user is already shown with a view that corresponds to a desired view of the image data, or one that forms a satisfactory basis for obtaining the desired view of image data. The user is further enabled to select one of the views and adjust the view to his or her preference by adjusting a view parameter of said view. The adjusted view parameter thus embodies a preference of the user, e.g., leading to the desired view. By then adjusting at least another one of the plurality of different views based on the adjusted view parameter, the user is presented with a gallery that comprises at least a further view that is automatically adjusted based on his or her preference.
The present invention is based on the insight that a large degree of freedom exists in visualizing image data. This typically corresponds to views being defined by large numbers of view parameter, say n view parameters. It is difficult for a user to obtain a desired view of the image data since this corresponds to obtaining a particular set of view parameters within an n-dimensional parameter space constituted by the n view parameters, i.e., constituting an n-dimensional optimization problem for the user. The present invention enables the user to more conveniently obtain the desired view by simultaneously presenting a plurality of different views, by enabling the user to adjust said views, and by automatically adjusting one or more other views in the gallery based on the adjustment. Advantageously, the user can more quickly and/or more conveniently obtain the desired view. Advantageously, the user does not need to individually adjust each of the plurality of different views. Rather, a manual adjustment of one the views results in an automatic adjustment of one or more of the other views in the gallery. Advantageously, a preference as embodied by the adjusted view parameter is automatically applied to other views.
Optionally, the user interface subsystem is arranged for enabling the user to iteratively select and adjust a number of views from of the plurality of different views, and the display processor is arranged for updating the gallery in response to each adjusting of one of the number of views. The user can thus obtain a desired view by iteratively manually adjusting a view parameter, with the system, in response, automatically adjusting one or more of the other views in the gallery. Advantageously, the user can more quickly and/or more conveniently obtain the desired view.
Optionally, the display processor is arranged for adjusting said other view by i) adjusting a view parameter of the other view based on the adjusted view parameter, thereby obtaining a further adjusted view parameter, and ii) generating the adjusted other view based on the further adjusted view parameter. The other view is thus adjusted by adjusting a view parameter of the other view. Advantageously, the manner of adjusting the other view by the system matches the manner of adjusting the selected view by the user.
Optionally, the view parameter of the other view and the adjusted view parameter differ in type. In response to the user adjusting a certain type of view parameter, the system thus adjusts the other view by adjusting a different type of view parameter. This aspect of the present invention is based on the insight that an adjustment of a type of view parameter of the selected view may necessitate or desire a change in a different type of view parameter of the other view. A reason for this is that the other view may not have a same type of view parameter. In this case, the system is nevertheless able to adjust the other view by adjusting a different type of view parameter. Moreover, even if the other view has a same or similar type of view parameter, it may nevertheless be desirable to adjust a different type of view parameter, e.g., due to the views showing different images and thus resulting in the user having different visualization needs for said images.
Optionally, the display processor is arranged for selecting the view parameter of the other view amongst a plurality of view parameters of the other view based on a similarity in type to the adjusted view parameter. The similarity in type is thus taken into account when selecting the view parameter of the other view, e.g., by selecting a view parameter of the other view that is most similar to the adjusted view parameter.
Optionally, the display processor is arranged for converting a value of the adjusted view parameter to a value of the view parameter of the other view based on a difference in type between the adjusted view parameter and the view parameter of the other view. An adjustment of a value of a type of view parameter may necessitate or desire a different adjustment of a value of a different type of view parameter. By converting the value of the adjusted parameter to a value of the view parameter of the other view based on said difference in type, said necessary or desired different adjustments can be effected.
Optionally, the adjusted view parameter and/or the further adjusted view parameter is one of the group of:
a view position;
a view orientation;
a rendering parameter;
a projection parameter;
a slab thickness;
a zoom factor;
a brightness or window level; and
a contrast or window width.
Optionally, the display processor is arranged for omitting adjusting the view parameter of the other view if said view parameter is indicated as a non-adjustable view parameter. It is therefore possible to fix certain view parameters in that they are not automatically adjusted in response to the adjustment of the view parameter by the user.
Optionally, the gallery is constituted by views having a first display quality, and the display processor is arranged for displaying the selected view at a second display quality during the adjusting of the view parameter, with the second display quality being higher than the first display quality. The gallery effectively constitutes a gallery of previews, and the user is enabled to adjust a selected one of the plurality of previews while it is presented at a higher display quality, e.g., constituting a non-preview version of said view.
Optionally, the user interface subsystem is arranged for enabling the user to adjust the view parameter by i) causing the display processor to generate a further gallery comprising variants of the selected view which are generated based on different values of the view parameter, and ii) enabling the user to establish the adjusted view parameter by selecting one of the variants of the selected view from the further gallery. Advantageously, an intuitive and consistent user interface is provided to the user in that a manner of selecting a view for adjustment and the actual adjustment of the view parameter of the selected view are similar or the same since they are both based on selecting a view from a gallery.
Optionally, the display processor is arranged for i) bookmarking one or more views of the updated gallery by generating bookmark data comprising view parameters defining said one or more views, and/or ii) generating one or more of the plurality of different views of the gallery based on bookmark data comprising view parameters defining said one or more views.
Optionally, the display processor is arranged for generating the plurality of different views based on at least one of: a user identifier identifying the user, metadata of the user, usage history information, metadata of the image data, and bookmarked views associated with the image data.
Optionally, the image interface is arranged for accessing the image data from a Picture Archiving and Communication System.
Optionally, a server is provided comprising the system set forth, wherein the user interface subsystem is connectable to a client device, the client device comprising a display and a user input for enabling the user to interact with the server, wherein the user interface subsystem is arranged for i) providing display data of the display processor to the display of the client device, and ii) receiving user data from the user input of client device.
Optionally, a client device is provided comprising a display and a user input for enabling the user to interact with the server set forth.
Optionally, the client device is constituted by a tablet device or Smartphone.
It will be appreciated by those skilled in the art that two or more of the above-mentioned embodiments, implementations, and/or aspects of the invention may be combined in any way deemed useful.
Modifications and variations of the method, the computer program product, the server and/or the client device, which correspond to the described modifications and variations of the system, can be carried out by a person skilled in the art on the basis of the present description.
A person skilled in the art will appreciate that the method may be applied to multi-dimensional image data, e.g. to two-dimensional (2-D), three-dimensional (3-D) or four-dimensional (4-D) images. A dimension of the multi-dimensional image data may relate to time. For example, a three-dimensional image may comprise a time domain series of two-dimensional images. The image may be acquired by various acquisition modalities such as, but not limited to, standard X-ray Imaging, Computed Tomography (CT), Magnetic Resonance Imaging (MRI), Ultrasound (US), Positron Emission Tomography (PET), Single Photon Emission Computed Tomography (SPECT), and Nuclear Medicine (NM).
The invention is defined in the independent claims. Advantageous yet optional embodiments are defined in the dependent claims.
These and other aspects of the invention are apparent from and will be elucidated with reference to the embodiments described hereinafter. In the drawings,
a shows an example of a gallery which comprises a plurality of different views from medical image data of a patient;
b shows a top-left one of said plurality of views having been selected;
c shows the selected view having been adjusted based on an adjustment of a zoom parameter, a pan parameter and a geometry position parameter;
d shows the adjusted selected view being further adjusted in contrast based on a selection from a further gallery comprising variants of the adjusted selected view;
e shows a top-right one of said views having been selected; and
f shows an updated gallery in which a plurality of other views has been automatically adjusted based on the adjusted selected view.
The external storage 180 may be a Picture Archiving and Communication System (PACS). The system 100 and the PACS 180 may both be part of a Hospital Information System (HIS). The client device may be connectable to the HIS.
An operation of the system 100 may be briefly explained as follows. The display processor 140 receives the image data 122 and generates a plurality of different views of the image data based on one or more view parameters defining each one of the plurality of different views. The display processor further generates a gallery comprising the plurality of different views. The gallery is provided, possibly as part of other displayable output, to the user interface subsystem 160 in the form of display data 162. The user interface subsystem 160 provides the display data 162 to the display 192 for display to the user. The user interface subsystem 160 further receives user data 164 from the user input 194 to enable the user to interact with the system 100 based on the display of the display data 162.
Accordingly, the user can select one the plurality of different views from the gallery. As a result, a selected view is obtained. The user can then adjust the selected view by adjusting a view parameter of the selected view. As a result, an adjusted view parameter is obtained. In response, the display processor 140 updates the gallery by adjusting at least another one of the plurality of different views based on the adjusted view parameter. As a result, an updated gallery is obtained comprising an adjusted other view.
The method 200 comprises, in a first step titled “ACCESSING IMAGE DATA”, accessing 210 the image data. The method 200 further comprises, in a second step titled “GENERATING PLURALITY OF VIEWS”, generating 220 a plurality of different views of the image data based on one or more view parameters defining each one of the plurality of different views. The method 200 further comprises, in a third step titled “GENERATING GALLERY”, generating 230 a gallery comprising the plurality of different views. The method 200 further comprises, in a fourth step titled “USER SELECTING VIEW”, enabling 240 the user to select one the plurality of different views from the gallery, thereby obtaining a selected view. The method 200 further comprises, in a fifth step titled “USER ADJUSTING VIEW”, enabling 250 the user to adjust the selected view by adjusting a view parameter of the selected view, thereby obtaining an adjusted view parameter. The method 200 further comprises, in a step titled “ADJUSTING OTHER VIEW AND UPDATING GALLERY”, updating 260 the gallery by adjusting at least another one of the plurality of different views based on the adjusted view parameter, thereby establishing an adjusted other view in the gallery.
The operation of the system 100 and the method 200 may be explained in more detail as follows.
In this particular example, the different views are obtained from a CT image acquisition which provided image data comprising a thousand 2D slices. The plurality of views 301-312 differ in various aspects, including visualization type (Multi-Planer Reformatting (MPR), Maximum Intensity Projection (MIP), and Surface Volume Rendering (SVR)), patient orientation (Coronal, Sagittal, Axial) and slab thickness (2 mm, 15 mm, full volume). More specifically, the first view 301 corresponds to a coronal MPR view, the second view 302 to sagittal MPR view, the third view 303 to an axial MPR view, the fourth view 304 to a coronal MIP view, the fifth view 305 to a sagittal MIP view, and the sixth view 306 to an axial MIP view. The plurality of different views 301-312 also includes a seventh view 307, an eight view 308 and a ninth view 309 constituting bookmarked views 307-309 which correspond to views of the image data which were generated earlier in time and then bookmarked by the system for later viewing. Said bookmarked views may correspond to adjusted views, i.e., as manually adjusted by the user or automatically adjusted by the system. Moreover, a tenth view 310, an eleventh view and a twelfth view 312 correspond to coronal volume rendered views with different Look-Up Tables (LUTs) for converting Hounsfield Unit (HU) values to color and opacity values, with each LUT being optimized for presenting different anatomies.
The gallery 320 shown in
Alternatively or additionally, the display processor 140 may use metadata of the image data 122 which may provide contextual information of the image data, e.g., a procedure code, a body part, an acquisition modality or parameter, etc. The metadata may be included in the image data 122, e.g., by means of DICOM attributes, or may be associated with the image data 122 through another mechanism, e.g., associated HL7 messages relating to a same patient or study. The metadata may also be associated with the user or the system 100 itself. Alternatively or additionally, contextual information of a user interface which is provided on the client device may be used. For example, if a presentation of the gallery 320 is requested by the user using a Graphical User Interface (GUI) element which is associated with contrast/brightness, the display processor 140 may establish a gallery 320 comprising a plurality of views which each are generated based on different contrast/brightness parameters. Alternatively or additionally, existing workflow results may be used which are associated with the image data 122. For example, bookmarked views which were generated from the image data 122 at an earlier stage in a workflow are likely of high interest for a user at a later stage in the workflow. Alternatively or additionally, usage history information may be used which may be indicative of, e.g., historic actions of the user or other users in a similar context. Alternatively or additionally, view parameters of an initial 3D presentation of the image data may be used. For example, a zoom and view orientation in the initial 3D presentation of the image data may be used to establish similar view parameters of the plurality of views 301-312 when the user requests presentation of a gallery 320.
The above contextual information, i.e., the user identifier identifying the user, the metadata of the user, the usage history information, the metadata of the image data 122, and the bookmarked views associated with the image data, may also be used to rank the individual views within the gallery, e.g., to present a most relevant view as the first view 301 and a least relevant view as the twelfth view 312. It is noted that, although not shown in
The user may select one of the plurality of different views 301-312 from the gallery 320. In the example of
d shows an optional aspect of the present invention in that the user may adjust a view parameter by means of a further gallery 330. For that purpose, the user interface subsystem 160 may be arranged for enabling the user to adjust the view parameter by causing the display processor 140 to generate a further gallery 330 comprising variants of the selected view which are generated based on different values of the view parameter.
In the example of
In general, the further gallery 330 constitutes an addition or alternative to other means of adjustments, e.g., using GUI elements or keyboard buttons to vary view parameters. The further gallery 330 may therefore also be used to adjust the aforementioned zoom parameter, pan parameter, geometry position parameter, etc, of the first image 301 so as to obtain the adjusted first image 301A. It is noted that the different view parameters, i.e., the different variants, may be established automatically by the system based on, e.g., medical image viewing protocols. In general, such variants may also be obtained by, e.g., automatically varying a value of the view parameter (non-)uniformly across a range.
In response to the user selecting the fourth view 301D from the further gallery 330, the display processor 140 may display the fourth view 301D at a higher resolution, or in general, at a higher display quality. A reason for this may be to enable the user to closely view the selected view 301D or to further fine-tune the value of the view parameter which corresponds to the selected view 301D.
In general, the view parameter of the other view and the adjusted view parameter may differ in type. For example, if the user adjusts an opacity parameter of a MIP view, the display processor 140 may adjust a LUT constituting a view parameter of a volume rendering of another view. The display processor 140 may be arranged for selecting the view parameter of the other view amongst a plurality of view parameters of the other view based on a similarity in type to the adjusted view parameter. For example, the display processor 140 may select a conceptually most similar view parameter of the other view. The display processor 140 may also be arranged for converting a value of the adjusted view parameter to a value of the view parameter of the other view based on a difference in type between the adjusted view parameter and the view parameter of the other view. For that purpose, the display processor 140 may use conversion data which indicates how to convert values between different types of view parameters. Examples of view parameters are a view position and a view orientation, e.g., in case the image data is 3D image data. Another term for the combination of view position and view orientation is geometry position. Other examples are a rendering parameter, e.g., in case the view is generated using a volume rendering technique, a projection parameter, e.g., in case the view is generated using a projection technique, a zoom factor, a brightness or window level, and a contrast or window width. Other examples of view parameters are slab thickness, view rotation, view roll, or so-termed render region. The latter may comprise values such as ‘full’, ‘slab’ or ‘beyond’, indicating an extent of the rendering orthogonal to the viewing plane, i.e., along the view direction. The display processor 140 may be arranged for omitting adjusting the view parameter of the other view if said view parameter is indicated as a non-adjustable view parameter. For example, a geometry position of a sagittal view may be fixed. Hence, although other view parameters of the sagittal view may in principle be adjusted, its geometry position is not adjusted.
The user interface subsystem 160 may arranged for enabling the user to iteratively select and adjust a number of views from the plurality of different views, and the display processor 140 may be arranged for updating the gallery in response to each adjusting of one of the number of views. The user may thus adjust further views of the updated gallery 322 by repeating a similar process as described in reference to
It will be appreciated that the present invention enables a user to obtain a desired view of image data based on a What You See Is What You Get (WYSIWYG) gallery of alternative presentations, i.e., different views, of the image data. Since the gallery shows the different views simultaneously, with typically one of the different views being already close to a desired view, there is no need for the user to ‘guess’ which adjustments are most effective to reach a desired view. The gallery may be advantageously used on client devices such as tablet devices which comprise a touch interface and lack convenient mouse and/or keyboard input. It is noted that the WYSIWYG gallery selection may be an iterative process, in which successive galleries show refinements of previously selected and possibly adjusted views. In particular, the view parameters already determined in earlier steps, either by means of the further gallery-based selection or other means of interacting with the visualization, are taken into account when presenting the next gallery.
It is noted that the present invention may be to medical image data as well as non-medical image data. An example of the latter is geographical image data.
It will be appreciated that the invention also applies to computer programs, particularly computer programs on or in a carrier, adapted to put the invention into practice. The program may be in the form of a source code, an object code, a code intermediate source and an object code such as in a partially compiled form, or in any other form suitable for use in the implementation of the method according to the invention. It will also be appreciated that such a program may have many different architectural designs. For example, a program code implementing the functionality of the method or system according to the invention may be sub-divided into one or more sub-routines. Many different ways of distributing the functionality among these sub-routines will be apparent to the skilled person. The sub-routines may be stored together in one executable file to form a self-contained program. Such an executable file may comprise computer-executable instructions, for example, processor instructions and/or interpreter instructions (e.g. Java interpreter instructions). Alternatively, one or more or all of the sub-routines may be stored in at least one external library file and linked with a main program either statically or dynamically, e.g. at run-time. The main program contains at least one call to at least one of the sub-routines. The sub-routines may also comprise function calls to each other. An embodiment relating to a computer program product comprises computer-executable instructions corresponding to each processing step of at least one of the methods set forth herein. These instructions may be sub-divided into sub-routines and/or stored in one or more files that may be linked statically or dynamically. Another embodiment relating to a computer program product comprises computer-executable instructions corresponding to each means of at least one of the systems and/or products set forth herein. These instructions may be sub-divided into sub-routines and/or stored in one or more files that may be linked statically or dynamically.
The carrier of a computer program may be any entity or device capable of carrying the program. For example, the carrier may include a storage medium, such as a ROM, for example, a CD ROM or a semiconductor ROM, or a magnetic recording medium, for example, a hard disk. Furthermore, the carrier may be a transmissible carrier such as an electric or optical signal, which may be conveyed via electric or optical cable or by radio or other means. When the program is embodied in such a signal, the carrier may be constituted by such a cable or other device or means. Alternatively, the carrier may be an integrated circuit in which the program is embedded, the integrated circuit being adapted to perform, or used in the performance of, the relevant method.
It should be noted that the above-mentioned embodiments illustrate rather than limit the invention, and that those skilled in the art will be able to design many alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. Use of the verb “comprise” and its conjugations does not exclude the presence of elements or steps other than those stated in a claim. The article “a” or “an” preceding an element does not exclude the presence of a plurality of such elements. The invention may be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In the device claim enumerating several means, several of these means may be embodied by one and the same item of hardware. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.
Filing Document | Filing Date | Country | Kind |
---|---|---|---|
PCT/IB2013/058884 | 9/26/2013 | WO | 00 |
Number | Date | Country | |
---|---|---|---|
61708331 | Oct 2012 | US |