Patent Application
20080317382
The invention relates to the field of digital imaging. In particular, the present invention relates to a method of registering a first image to a second image, to an image processing device and to a software program for registering a first image to a second image.
The goal of image registration, for example, in medical imaging applications, is to compensate for differences in images, for example, due to patient movements, different scanner modalities, changes in the anatomy, etc. Global registration methods such as rigid or affine transformations often cannot cope with local differences. A solution for this is known as elastic registration. Robust elastic registration of medical images is a difficult problem, which is currently the subject of intensive research.
Point-based elastic registration comprises the steps of defining a set of control points relative to a first image and then performing elastic deformation of the first image at these control points, so as to bring the first image into an optimal spatial correspondence with a second image, where the alignment is quantified by a similarity measure. In the case of parametric geometrical transformations, the optimal alignment is reached by computing an optimal parameter setting, which for elastic registration in general means the optimal number and positions of control points as well as the displacement parameters (defining the degree of elastic deformation of the first image) at these control points.
The most widely-used transformation class for elastic image registration are B-splines, which are defined on a regular grid of control points. In general, when a highly elastic deformation of the first image is required, a high density of control points is required to be defined. In the case of a regular grid of control points, this high density would be required to be provided in respect of the whole first image, even if such highly elastic deformation were only required in respect of a small area thereof. At least the displacement parameters in respect of each control point needs to be determined, such that in this case a huge number of parameters would be required to be optimised, which requires a long computation time.
The above-mentioned drawback may be overcome by using transformations based on irregular grids of control points. The positions on the first image of a fixed number of control points are considered as free parameters (to be optimised), which can be changed, together with the control point displacement parameters, during the optimisation process. This allows control points to be moved as required, and enables a high density of control points to be provided in respect of a region of the first image where highly elastic deformation is required, whereas in other image regions, the control point density can be much lower. For example, International Patent Application No. WO 2005/057495 describes a method of elastic deformation in which a force field is applied at several control points to a first image, and the optimal positions of the control points at which the forces are applied are found automatically, so as to minimise the difference between the first and the second images.
However, the number of control points is fixed at the start of the image registration process and remains fixed throughout the process. Since the optimal number and initial relative position of the control points cannot be known in advance of the registration process, a larger number of control points than would otherwise be necessary is required to achieve an acceptable image registration result, which in turn means that the computation capacity and time required to perform the optimisation process is also unnecessarily high.
It is an object of the present invention to provide a method of image registration, wherein the number of control points used can be optimised so as to minimise the computational capacity and time required to perform image registration without loss of quality of the registration result. It is also an object of the invention to provide a corresponding image processing device and software program.
In accordance with the present invention, there is provided a method of registering a first image and a second image, the method comprising:
placing at least one control point within said first image, and determining a first parameter setting defining a position and displacement parameters in respect of said at least one control point so as to elastically deform said first image and thereby to improve the similarity between said first image and said second image, and then repeating the steps of:
placing at least one additional control point within said first image, determining a second parameter setting in respect of said at least one additional control point defining a position and displacement parameters so as to elastically deform said first image and thereby to further improve said similarity between said first image and said second image; until a predetermined criteria is met.
Thus, the object of the invention is achieved by starting with one or more control points (preferably a single control point) and iteratively adding new control points after each elastic deformation operation, until a predetermined criteria is met. In this way, the number of control points does not need to be specified in advance and can be automatically adapted to the complexity of the deformation field. In a preferred embodiment, new control points are iteratively added after each elastic deformation operation until the similarity between the first image and the second image reaches at least a predetermined level.
Preferably, each time one or more additional control points are added, optimal parameter settings in respect of all control points in said first image are determined. Thus, in general, a set of N control points is optimised and the resulting configuration serves as the starting point for the next optimisation of a set of N+M control points, wherein N and M are integers. In one exemplary embodiment of the present invention, M=1, with the single additional control point preferably being placed randomly within the first image prior to the next optimisation operation. In an exemplary, preferred embodiment therefore, starting from an initial configuration of just one (N=1) randomly-placed control point, control points are added one-by-one until no further (significant) improvement of the similarity between the first image and the second image can be achieved.
Beneficially, the parameter settings of each control point are optimised so as to optimise a similarity measure (which may, as an example, be the squared difference between the first and second images, but many other types of similarity measure may be used, including mutual information or cross-correlation, and the present invention is not necessarily intended to be limited in this regard). In a preferred embodiment, a similarity measure is obtained after each elastic deformation operation and the amount by which the similarity between the first image and the second image has improved (i.e. the improvement in the similarity measure caused by the last iteration) may be determined and compared with a predetermined criterion, wherein an additional one or more control points are added only if said predetermined criterion is not met.
Also in accordance with the present invention, there is provided an image processing device for performing registration of a first image and a second image, the device comprising a memory for storing said second image, means for receiving image data in respect of said first image, and processing means configured to:
initially place at least one control point within said first image, and determine a first parameter setting defining a position and displacement parameters in respect of said at least one control point so as to elastically deform said first image and thereby to improve the similarity between said first image and said second image, and then repeat the steps of:
placing at least one additional control point within said first image, determining a second parameter setting in respect of said at least one additional control point defining a position and displacement parameters so as to elastically deform said first image and thereby to further improve said similarity between said first image and said second image; until a predetermined criterion is met.
Still further in accordance with the present invention, there is provided a software program for registering a first image and a second image, wherein the software program causes a processor to:
initially, place at least one control point within said first image, and determine a first parameter setting defining a position and displacement parameter in respect of said at least one control point so as to elastically deform said first image and thereby to improve the similarity between said first image and said second image, and then repeat the steps of:
placing at least one additional control point within said first image, and determining a second parameter setting in respect of said at least one additional control point defining a position and displacement parameters so as to elastically deform said first image and thereby to further improve said similarity between said first image and said second image; until a predetermined criterion is met.
These and other aspects of the present invention will be apparent from, and elucidated with reference to, the embodiments described herein.
Embodiments of the present invention will now be described by way of examples only and with reference to the accompanying drawings, in which:
In spite of the fact that the method is described in the following with reference to medical applications, it should be noted that the present invention can be applied to any multi-dimensional data sets or images required to be registered. For example, the present invention may be applied to quality testing of products, where images of actual products are compared to images of reference products. Also, the method may be applied for material testing, for example, for monitoring changes to an object of interest over a certain period of time.
Next, an additional control point is randomly placed inside the first image region at step S6, and the optimal parameter settings for both of the control points within the first image region are computed at step S7 in order to achieve the best alignment of the first and second images.
Once elastic deformation of the first image with the appropriately positioned control points has been effected at step S8, a new similarity measure is calculated at step S9. The new similarity measure is compared at step S10 with the previously-computed similarity measure according to some predetermined stopping criterion (e.g. the difference is compared with a threshold value). If, at step S11, the predetermined stopping criterion is not met (e.g. the difference between the current and previous similarity measures is at least equal to the threshold value indicating that the similarity between the first and second images has been improved by at least a predetermined amount), the method returns to step S6, where a further control point is added, and the above process is repeated. Once the stopping criterion is fulfilled (e.g. the difference between the current and previous similarity measures falls below the above-mentioned threshold value), the method ends, at step S12, and the image registration process is complete.
In general, the registration of two images I1, I2 consists of finding a transformation t, such that the difference between t(I1) and I2 is minimal according to a predefined similarity measure sim. In image registration, it is commonly formed as an optimisation problem such that the parameter vector c, which represents the ideal transformation tc, will maximise an objective function f(c)=corr tc(I1), I2). According to an exemplary embodiment of the present invention, therefore, the optimisation problem can be formulated as searching, in respect of each iteration, for optimal positions of a given set of control points in the first image, and their optimal displacement parameters. As will be apparent to a person skilled in the art, many different types of transformation may be used, and examples can be found in, for example, D. Rueckert et al. Comparison and evaluation of rigid, affine and non-rigid registration of breast MR images. Journal of Computer Assisted Tomography 23(5), pp. 800-805, 1999 and V. Pekar, E. Gladilin, K. Rohr. An adaptive irregular grid approach for 3-D deformable image registration. Physics in Medicine and Biology 2005, in press.
The formulated optimisation problem may be solved using standard numerical optimisation techniques, such as, for example, the downhill simplex method as described in J. A. Nelder and R. Mead, A simplex method for function minimisation, Computer Journal, (7): 308-313, 1965.
Thus, starting from a single, randomly-placed control point within the first image region, a locally convergent optimisation strategy is used to find the optimal configuration for the control point set, where the position and displacement parameters of all control points (including the ones optimised in the previous step) are considered as free parameters. In the first few iterations, the optimisation step in respect of just one or a few control points can be performed very quickly due to the small number of parameters to be optimised. Compared with prior art methods, whereby image registration which uses local optimisation strategies based on a fixed number of control points, the proposed method yields comparable or even better results with a much smaller number of control points. Hence, the proposed method can significantly speed up the image registration process, and meet application-specific quality requirements. This is most important in time-critical applications, such as intra-surgery registration, where optimal registration accuracy has to be achieved over an application-specific region of interest (clinical focus) only. Furthermore, the iterative increase in the number of control points enhances the robustness of the registration algorithm. For applications requiring high accuracy, which can only be achieved using a large number of control points, the termination criterion can be defined in an appropriate way.
It should be noted that the present invention may be applied to CT images, magnetic resonance images (MRI), positron emitted tomography images (PET), single photon emission computed tomography images (SPECT) or ultrasound (US) modalities. Also, other data sets may be used.
It should be noted that the above-mentioned embodiments illustrate rather than limit the invention, and that those skilled in the art will be capable of designing many alternative embodiments without departing from the scope of the invention as defined by the appended claims. In the claims, any reference signs placed in parentheses shall not be construed as limiting the claims. The word “comprising” and “comprises”, and the like, does not exclude the presence of elements or steps other than those listed in any claim or the specification as a whole. The singular reference of an element does not exclude the plural reference of such elements and vice-versa. The invention may be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In a device claim enumerating several means, several of these means may be embodied by one and the same item of hardware. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.
| Number | Date | Country | Kind |
|---|---|---|---|
| 05300910.6 | Nov 2005 | EP | regional |
| Filing Document | Filing Date | Country | Kind | 371c Date |
|---|---|---|---|---|
| PCT/IB06/54183 | 11/9/2006 | WO | 00 | 5/8/2008 |
