The present invention relates to navigating in a series of related images, in particular in a series of medical images.
A common way of inspecting a large dataset comprising images is to load the dataset from a storage means and display the images one after the other. The images of interest within the dataset are found by scrolling through the large number of images. This is a time-consuming and cumbersome task, which includes navigating to the image(s) of interest in order to view an area of interest. When large datasets are being inspected, there is little help or assistance available to make it easier or faster to find the image(s) of interest.
The patent U.S. Pat. No. 7,212,661 B2 describes a technique for generating navigational scout images, based upon a series of related images. One or more of the related images may be accessed by review of the navigational image. The navigational images are sample images selected from the related images from a large image dataset. The technique aims at facilitating identification of specific images in the image series,
It is an object of the present invention to improve navigating in a series of related images.
In particular, it may be seen as an object of the present invention to provide a method, system and computer program product for fast and easy navigation in a series of related images.
In a first aspect, the invention provides a method of navigating in a series of related images, comprising the steps of creating and storing one or more navigational entities, each navigational entity relating to an object depicted in one or more related images of the series of related images, and facilitating navigation within said series of related images by means of said navigational entities, wherein each navigational entity is created based on segmentation of the object depicted in the one or more related images and on identification of the segmented object.
The inventive method provides navigational entities created by segmentation of objects and by identification of the segmented objects; the navigational entities are used for user-friendly and fast navigation within the series of images. The segmented and identified objects provide an intuitive or direct means of navigating to images of interest to a user.
The navigational entities may be stored together with, i.e. in the same storage means, and optionally interleaved with the series of related images. The related images may be images of a subject forming a series of images so that an object depicted in one or more of the related images may be an object depicted in one or more successive images in the series of images. The term “related images” is meant to denote images related to each other, such as different images of one subject. When the images are sequential in the meaning that they are ordered, e.g. as images taken along an axis of a subject, the term “successive images” is meant to describe images ordered in space by acquisition to form the series of images.
It should be noted that the method may furthermore comprise a preceding step of acquiring the series of related images and/or segmenting the image data. The creation of the navigational entities may be performed simultaneously with the acquisition of the images, or it may be performed after the acquisition of the series of related images has been concluded.
The term segmentation refers to the process of partitioning a digital image into multiple regions. The goal of segmentation is to simplify and/or change the representation of an image into something that is more meaningful and easier to analyze. Image segmentation is typically used to locate objects and boundaries (lines, curves, etc.) in images in order to separate objects of interest from each other and from the background.
The result of image segmentation may be a set of regions that collectively cover the entire image, or a set of contours extracted from the image. Each of the pixels in a region is similar with respect to some characteristic or computed property, such as color, intensity, or texture. Adjacent regions are significantly different with respect to the same characteristic(s).
Within medical imaging, typical examples of the use of image segmentation are: locating tumors and other pathologies, measuring tissue volumes, computer-guided surgery, diagnosis, treatment planning, study of anatomical structures.
A range of methods of segmentation is known. Any appropriate method of segmentation allowing subsequent identification of objects may be used. Such an appropriate method of segmentation could e.g. be chosen from amongst the following common methods of segmentation:
The above mentioned methods of segmentation are only a number of examples of segmentation techniques, and it should be understood that any other appropriate method of segmentation may be used.
According to an aspect of the invention, each navigational entity comprises a first navigational part to be displayed and a second navigational part, said second navigational part comprising a reference to an image within the series of related images. Hereby, means are provided for navigating quickly within the series of images of the area of interest comprising the object, in that a reference or association is created between a first navigational part which is to be displayed and a particular image within the series of images.
According to another aspect, navigation is facilitated by displaying the first navigational part of the navigational entity on a display, providing means for letting a user choose said first navigational part and displaying the image referenced by the second navigational part of the navigational entity, if the first navigational part of said navigational entity is chosen by the user. Thus, by choosing a first navigational part of a navigational entity, a user may be able to view an image in the series of related images corresponding to the first navigational part. Therefore, easy switching between or navigation in different images of interest is facilitated, in particular when a number of navigational entities, appropriate for the objects depicted or shown in the images, is chosen.
According to an aspect of the invention, the first navigational part of said navigational entity comprises an image representative of the identified object, an icon and/or a keyword representative of the identified object. This provides a user friendly presentation of the object identification and thereby a user-friendly user interface.
According to another aspect, the first navigational part of said navigational entity is an image from the series of related images, shown in reduced size and/or in reduced resolution. In this case, the second navigational part should comprise a reference to the corresponding image in full resolution and full size in the series of images, so that the image in full resolution and full size may be displayed, if the reduced image is chosen by the user. It should be noted that the term “a reduced image” is meant to denote an image from the series of related images shown in reduced size and/or resolution. The navigational entities in the form of reduced images may be shown on a display together with an image in full size and full resolution.
According to an aspect, the series of related images may be displayed sequentially, beginning with the image referenced by the second navigational part. Thus, by choosing a navigational entity or a first navigational part of a navigational entity, a user may be able to view an image in the series of related images corresponding to the navigational entity and he/she may continue viewing subsequent or preceding images in the series of related images. Therefore, easy switching between or navigation in different images of interest is facilitated. The term “images referenced by the second navigational part” is meant to denote an image within the series of images to which the second navigational part refers.
According to an aspect of the invention, the series of images comprises medical images obtained by scanning a subject, and wherein each image of the series of related images represents data acquired in a slice through the subject. The medical images may e.g. be obtained by computed tomography (CT), magnetic resonance imaging (MRI), ultrasound (US) or positron emission tomography (PET). Typically, the subject is a human or animal body and the objects identified comprise organs of said human or animal body. In this case, the related images form a sequence of images depicting slices of the body. Typically, hundreds or even thousands of images of a human body will be created for reviewing by clinicians to identify possible features of interest.
According to yet another aspect, the invention relates to an apparatus for navigating in a series of images, comprising processor means for creating navigational entities, each navigational entity relating to an object depicted in one or more images of the series of related images; storage means for storing navigational entities created by the processor means; user interface means facilitating navigation within said series of images by means of said navigational entities; wherein said processor means are arranged for creating each navigational entity, based on segmentation of the object depicted in the one or more images and on identification of the segmented object. The apparatus has advantages and additional aspects similar to those described above in connection with the method.
According to yet another aspect, the invention relates to a computer program product for performing the steps of the method of the invention, when said computer program product is executed by a computer.
The individual aspects of the present invention may each be combined with any of the other aspects. These and other aspects of the invention will be apparent from the following description with reference to the described embodiments.
The method and apparatus according to the invention will now be described in more detail with reference to the accompanying figures. The figures show one way of implementing the present invention and are not to be construed as being limited to other possible embodiments falling within the scope of the attached claim set. Throughout the figures, like reference numerals denote like elements.
a-5c and 6a-6b show examples of display means displaying medical images.
The method 10 continues at step 12, wherein segmentation of objects depicted in images within one or more of the related images is performed. The segmentation may be any appropriate segmentation allowing subsequent identification of objects, such as organs.
Subsequently, the method 10 continues at step 14, wherein segmented objects are identified. Thus, step 14 may comprise analyzing the segmented objects and comparing them with predefined objects. Identification of an object may result in creation of a navigational entity comprising a first navigational part and a second navigational part. The first navigational part is to be displayed to a user, e.g. in a display of an apparatus, in a way that a user may choose the first navigational part by interacting with a user interface of the apparatus. The corresponding second navigational part of the navigational entity comprises a reference to an image within the series of images, e.g. a bookmark in the series of images, a pointer, etc. Hereby, quick access to the bookmarked or referenced images within the series of images is facilitated.
Typically, an object may be depicted in a range of successive images. In this case one of the images within the range of successive images may be chosen as the image referenced by the second navigational part of the navigational entity; the navigational entity may render fast access to this bookmarked image, as explained below. The navigational entity may be an icon, a keyword or a piece of text or one of the images in the series of related images shown in reduced size and/or reduced resolution.
In the next step, step 16, navigation within the series of images is facilitated by means of the navigational entity/entities. As explained above, the creation of a navigational entity comprises the provision of a second navigational part, being a pointer, a bookmark or a reference to an appropriate image within the series of related images. It is well known to provide navigation in sets of data or images by means of such pointers, bookmarks or references.
The method 10 ends with step 17.
Images may be loaded into the storage 26 as received via the input 27. The processor 24 may segment and identify objects present in one or more images in order to create navigational entities. Each navigational entity comprises a first navigational part to be displayed in the display 22 of the apparatus so as to allow a user to choose it. Each navigational entity also comprises a second navigational part comprising a pointer, a reference, an association or a bookmark to an image within the series of related images or a bookmark within the series of related images. In the case of medical images, such as images from a CT scan of a human body, the images correspond to slices through the body. The organs of the body, such as the liver, the spleen, the heart, the lungs, etc., may be depicted in a range of successive or otherwise related images. The processor 24 may segment objects depicted in the images and identify the segmented objects as e.g. the organs of the human body. Typically, an organ of the human body will appear in a number of successive or otherwise related images corresponding to a number of slices through the organ. The processor 24 may be arranged for choosing or selecting one image amongst the number of images as the reference or associated image or the bookmarked images. If an organ appears in e.g. N successive images, the processor 24 may choose the image in the middle, i.e. number N/2 or (N+1)/2 as the reference image or the bookmarked image. In this example, the reference image is an image in the middle of a set of N images. However, the reference image does not have to be in the middle of an object or close to the middle. Instead, the reference image could be at a position cutting through the object that is particularly representative of that object. Alternatively or additionally, the reference image could be the image through the center of gravity of the object, which image may be far from the middle depending on the shape of the object. Alternatively, the position of the reference image could be related to some specific shape feature of particular types of objects, e.g. a centerline or axis of symmetry, if a visually interesting location in a segmented object may be detected automatically or interactively, for instance as part of the segmentation process.
The first navigational parts may be shown in the display 22, alone or in conjunction with one or more images from the series of related images. Typically, the display 22 is divided into two or more display areas, wherein images from the series of related images may be shown in one display area whilst the first navigational parts may be shown in another display area. However, these display areas may be overlaid or one may be embedded in the other.
The apparatus 20 moreover comprises a user interface. This may comprise a keyboard or other navigational devices, such as a mouse associated with a display unit shown on the display. The user interface may enable a user to navigate in the series of images by highlighting or choosing, e.g. by clicks of the mouse, a navigational entity.
The apparatus 20 may be integrated in an apparatus for acquiring medical images, such as a CT-scanner, a PET-scanner, etc., in an apparatus for displaying images obtained by an acquisition apparatus or it may be implemented as a stand-alone apparatus. In the case where the apparatus is integrated in an apparatus for displaying images obtained by an acquisition apparatus, the display of the apparatus 20 may be the display of the apparatus for displaying images.
The terms “reference image”, “associated image” and “bookmarked images” are meant to be synonymous and relate to an image within the series of related images to which a reference, pointer or bookmark refers.
When an object is depicted or shown in images from a range of successive images, one of the images in the range may be chosen as the image referred to by the second navigational part of the navigational entity. This referenced image in reduced size and/or reduced resolution may be the first navigational part of the navigational entity.
In
As described above, each navigational entity comprises a first and a second navigational part. The first navigational part is an image, text, icon or a combination thereof, corresponding to an image in the series of related images, whilst the second navigational part is a reference, a pointer or a bookmark pointing to said corresponding image in the series of related images. Thus, when a user chooses one of the first navigational parts 31-34 in the first display part 30, the apparatus 20 (see
a-5c show examples of display 22 displaying medical images.
b shows a layout of the display 22 corresponding to
The first display means 30 shows first navigational parts, viz. four keywords 35-38, whilst the second display means 40 shows an image 42. Again, if a user chooses one of the first navigational parts 35-38, an image corresponding to the chosen keyword 35-38 will be displayed in the second display means 40. A white arrow is shown in the first display means 30, indicating a part of the user interface arranged for letting a user choose one of the keywords 35-38 in order to activate the display in the second display means 40 of the image corresponding to the chosen keyword.
a and 6b show two pictures of a display 22 of the layout shown in
The invention can be implemented by means of hardware, software, firmware or any combination of these. The invention or some of the features thereof can also be implemented as software running on one or more data processors and/or digital signal processors or as software running in an apparatus for obtaining medical images or data, such as an apparatus for computed tomography (CT), magnetic resonance imaging (MRI), ultrasound (US) or positron emission tomography (PET).
The individual elements of an embodiment of the invention may be physically, functionally and logically implemented in any suitable way such as in a single unit, in a plurality of units or as part of separate functional units. The invention may be implemented in a single unit, or be both physically and functionally distributed between different units and processors.
Although the present invention has been described in connection with the specified embodiments, it should not be construed as being in any way limited to the presented examples. The scope of the present invention is to be interpreted in the light of the accompanying claim set. In the context of the claims, the terms “comprising” or “comprises” do not exclude other possible elements or steps. Also, the mentioning of references such as “a” or “an” etc. should not be construed as excluding a plurality. The use of reference signs in the claims with respect to elements indicated in the figures shall also not be construed as limiting the scope of the invention. Furthermore, individual features mentioned in different claims, may possibly be advantageously combined, and the mentioning of these features in different claims does not exclude that a combination of features is not possible and advantageous.
Number | Date | Country | Kind |
---|---|---|---|
07123161.7 | Dec 2007 | EP | regional |
Filing Document | Filing Date | Country | Kind | 371c Date |
---|---|---|---|---|
PCT/IB08/55111 | 12/5/2008 | WO | 00 | 6/9/2010 |