The disclosures relate to a method and a system for non-rigid image registration.
Image registration, as a procedure of establishing spatial relationship between images, has played an important role in medical image processing, group analysis and many other clinical applications. Many medical image analysis methods, such as structure segmentation, population-based studies, volumetric and morphologic measurements, employ the registration-based scheme to achieve high performance. Through decades of effort, this research field has witnessed great advances improving image registration accuracy by using various approaches. However, in recent years, with an ever increasing amount of acquired medical images and a continuously increased image resolution, the computational cost of a registration algorithm has become a critical problem and the bottleneck which limits the application of image registration to many clinical practices which require quick responses.
In particular, computation time is a big constraint for 3D non-rigid neuroimage registration in clinical practices. Non-rigid image registration, due to the complexity of optimization on the high degree-of-freedom transformation, is an extremely time-consuming procedure, especially when the current neuroimage sizes have increased tremendously with the advent of advanced scanners. The non-rigid registration has many parameters (frequently up to a 106 dimensional space) to be optimized, leading to a typical runtime of a registration algorithm in an order of 10 minutes, up to hours.
In the recent decades, there were increasing efforts dedicated to improve the image registration efficiency. In general, the acceleration of registration algorithms can be addressed on two fronts. One is to design a new registration algorithm with improved computational efficiency in mind; and the other is to make the best use of the computational power of the hardware to speed up the registration process.
As for algorithm-oriented registration efficiency improvement, one idea to speed up the non-rigid registration is rather than to use all the information in the whole image domain, instead the particular attention has been paid to a subset of important regions. A hybrid method was proposed to utilize sparse salient region feature correspondences to estimate both the global rigid transformation and the local deformation field with a local free-form deformation model in a closed form. A strategy of using a priori knowledge to increase the confidence of certain regions is also investigated, which can reduce the dimensionality of the registration problem.
In addition, some prior knowledge of the deformation field can be used to speed up the registration process. However, the registration accuracy of those newly developed efficient registration algorithms cannot yet compete with the conventional top-performing registration methods, e.g., ANTs, ART, IRTK. As for hardware-based registration efficiency improvement, the acceleration of image registration algorithms was usually achieved by exploiting parallelism, and the high-performance computing systems, such as computer clusters, shared-memory multi-processors and GPUs, have all been exploited for this purpose. However, parallelism is not applicable for all the registration methods. One essential requirement for the parallelizable algorithm is that it should be easy to be divided into a number of parallel portions, so that each portion can be assigned to one individual device. Some inherently serial algorithms cannot be split up into parallel portions, as they require the results from a preceding step to effectively carry on with the next step.
Therefore, it is desired to propose a general registration acceleration framework for speeding up the existing non-rigid registration methods without any modification of the original registration methods.
The following presents a simplified summary of the disclosure in order to provide a basic understanding of some aspects of the disclosure. This summary is not an extensive overview of the disclosure. It is intended to neither identify key or critical elements of the disclosure nor delineate any scope of particular embodiments of the disclosure, or any scope of the claims. Its sole purpose is to present some concepts of the disclosure in a simplified form as a prelude to the more detailed description that is presented later.
In an aspect, disclosed is a method for non-rigid image registration, comprising: dividing a subject image and a target image into a plurality of subject sub-images and a plurality of target sub-images, respectively; registering, in parallel, each of the subject sub-images and corresponding one of the target sub-images to obtain a transformation matrix of the each of the subject sub-images relative to the corresponding one of the target sub-images; transforming, based on the obtained transformation matrix, the subject sub-images into images corresponding to the target sub-images; and merging the transformed images to form a final registered image having a resolution of the target image.
In one embodiment of the present application, the dividing is implemented by interleaving down-sampling.
In one embodiment of the present application, the subject image and the target image are divided such that each of the subject sub-images corresponds to one of the target sub-images.
In one embodiment of the present application, the dividing further comprises: dividing the subject image and the target image into a plurality of first lower-resolution images and a plurality of second low-resolution images, respectively.
In one embodiment of the present application, the merging comprises: interpolating, the transformed images to obtain converted images having the resolution of the target image; and averaging the obtained images to form the final registered image having the resolution of the target image.
In one embodiment of the present application, the method further comprises: acquiring, by an image acquisition system/device, a subject image and a target image from patients, the subject image and the target image being obtained independently. In one embodiment of the present application, the image acquisition system/device comprises a first system and a second system to obtain the subject image and the target image independently. In one embodiment of the present application, the image acquisition system is configured to obtain the subject image and the target image from a same patient at different time. In one embodiment of the present application, the image acquisition system is configured to acquire the subject image and the target image from different patients.
In an aspect, disclosed is a system for non-rigid image registration, comprising: a down-sampler for dividing a subject image and a target image into a plurality of subject sub-images and a plurality of target sub-images, respectively; a registration unit for registering, in parallel, each of the subject sub-images and corresponding one of the target sub-images to obtain a transformation matrix of the each of the subject sub-images relative to the corresponding one of the target sub-images; a transformer for transforming, based on by the obtained transformation matrix, the subject sub-images into images corresponding to the target sub-images; and a merger for merging the transformed images to form a final registered image having a resolution of the target image.
In an aspect, disclosed is a system for non-rigid image registration, comprising: an image acquisition system/device configured to obtain a subject image and a target image from patients, wherein the subject image and the target image are obtained independently; a controller retrieving the obtained subject image and the target image and comprising: a down-sampler for dividing a subject image and a target image into a plurality of subject sub-images and a plurality of target sub-images, respectively; a registration unit for registering, in parallel, each of the subject sub-images and corresponding one of the target sub-images to obtain a transformation matrix of the each of the subject sub-images relative to the corresponding one of the target sub-images; a transformer for transforming, based on by the obtained transformation matrices, the subject sub-images into images corresponding to the target sub-images; and a merger for merging the transformed images to form a final registered image having a resolution of the target image. The system may further comprises a displayer electronically communicated with the controller and displaying the final registered image received from the merger, thereby causing at least one of monitoring of tumor growth and determination of treatment position based on the displayed final registered image.
Exemplary non-limiting embodiments of the present application are described below with reference to the attached drawings. The drawings are illustrative and generally not to an exact scale. The same or similar elements on different figures are referenced with the same reference numbers.
Reference will now be made in detail to some specific embodiments of the invention including the best modes for carrying out the invention. Examples of these specific embodiments are illustrated in the accompanying drawings. While the invention is described in conjunction with these specific embodiments, it will be appreciated by one skilled in the art that it is not intended to limit the invention to the described embodiments. On the contrary, it is intended to cover alternatives, modifications, and equivalents as may be included within the spirit and scope of the invention as defined by the appended claims. In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present application. The present application may be practiced without some or all of these specific details. In other instances, well-known process operations have not been described in detail in order not to unnecessarily obscure the present application.
The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used herein, the singular forms “a”, “an” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms “comprises” and/or “comprising,” when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
In practical application, in order to match a subject image and a target image, a non-rigid image registration usually needs to be applied between the subject image and the target image to convert the subject in to a coordinate system of the target image, after an affine registration. The runtime of one pair of non-rigid image registration is closely related to the resolution of the images. In general, the registration between the higher-resolution images involves more transformation parameters to be estimated, which consequently increases the computational cost. Based on this principle, the present disclosure discloses a method for non-rigid image registration. The difference between the present method and the traditional registration method is illustrated in
The image acquisition device 201 operates to obtain a subject image and a target image, for example, from patients, wherein the subject image and the target image are obtained independently. According to one embodiment, the image acquisition device 201 comprises a first acquirer and a second acquirer (not shown) to obtain the subject image and the target image independently. For example, the subject image may be obtained by the CT scanner, and the target image may be obtained by the MRI device. In some medical applications, the subject image and the target image may be obtained from a same patient at different times, to observe disease progress, such as tumor growth, or to determine treatment position by matching the subject image obtained before operation and the target image obtained during operation. According to another embodiment, the image acquisition device 201 is configured to acquire the subject image and the target image from different patients. For example, in some medical applications, the subject image may be obtained from a patient under diagnosing, and the target image may be obtained from a patient whose disease has been determined, then the registration between the subject image and the target image may assist to determine the disease of the patient under diagnosing.
The controller 20 operates to retrieve the obtained subject image and the target image, for example, directly from the image acquisition device 201, and the retrieved images are then processed by the down-sampler 2021, the registration unit 2022, the transformer 2023 and the merger 2024, which will be further discussed in reference to
Yk=DkMkX k=1, . . . , N (1)
where Dk is the decimation operator and Mk is the geometric motion operator, wherein operator D indicates the motion of a sampling region and the operator M indicates the density of the sampling. As shown in
Specifically, as shown in
The subject image and the target image may be divided into a plurality of first lower-resolution images and a plurality of second lower-resolution images by the above method, respectively, wherein the term “first lower-resolution image” only indicates the lower-resolution image divided from the subject image and the term “second lower-resolution image” only indicates the lower-resolution image divided from the target image. The first lower-resolution images may be different from each other, and the second lower-resolution images may be different from each other.
In some embodiment, selections of the sampling region and the sampling density of the subject image and the target image may be different, and the numbers of the first and second lower-resolution images are same.
Now return to
After registration, a transformation matrix of the each of the first lower-resolution images relative to the corresponding one of the second lower-resolution images can be obtained. Then, at step S303, the transformation matrix is inputted into the transformer 2023, and the transformer 2023 transforms each of first lower-resolution images by the transformation matrix. For example, as shown in
After obtaining transformed first lower-resolution images, at step S304, the transformed first lower-resolution images are inputted into the merger 2024, and the merger 2024 then merges the transformed first lower-resolution images to form a final registered image having the resolution of the target image. In some embodiment, as shown in
While the other pixels are all indicated by x in
The step S304 may be described by Eq. 2:
where DkT and MkT are the inverse operation of decimation and motion compensation. hk is the transformation field, such as transformation matrix estimated by kth pair of lower-resolution images. {tilde over (X)}(h) is the final registered image. The solutions of each pair of lower-resolution registration (DkTMkTYk(hk)) are then fused in an average manner to form the final solution. In this way, the registration errors of each lower-resolution image pair can be averaged out.
The displayer 203 is electronically communicated with the controller and may display the final registered image received from the merger 2024, in some embodiment, the doctor can perform the monitoring of tumor growth and the determination of treatment position based on the displayed final registered image.
As will be appreciated by one skilled in the art, the present application may be embodied as a system, a method or a computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment and hardware aspects that may all generally be referred to herein as a “unit”, “circuit,” “module” or “system.” Much of the inventive functionality and many of the inventive principles when implemented, are best supported with or integrated circuits (ICs), such as a digital signal processor and software therefore or application specific ICs. It is expected that one of ordinary skill, notwithstanding possibly significant effort and many design choices motivated by, for example, available time, current technology, and economic considerations, when guided by the concepts and principles disclosed herein will be readily capable of generating ICs with minimal experimentation. Therefore, in the interest of brevity and minimization of any risk of obscuring the principles and concepts according to the present application, further discussion of such software and ICs, if any, will be limited to the essentials with respect to the principles and concepts used by the preferred embodiments.
Although the preferred examples of the present application have been described, those skilled in the art can make variations or modifications to these examples upon knowing the basic inventive concept. The appended claims are intended to be considered as comprising the preferred examples and all the variations or modifications fell into the scope of the present application.
Obviously, those skilled in the art can make variations or modifications to the present application without departing the spirit and scope of the present application. As such, if these variations or modifications belong to the scope of the claims and equivalent technique, they may also fall into the scope of the present application.