Patent Application
20240289951
The present invention relates to a method for motion correction of MR data, an apparatus for motion correction of MR data, a system for medical imaging, and computer program element.
Magnetic resonance imaging, MRI, is known from the state of the art. MRI is used to obtain medical images of an anatomy, for example an organ of a human. MRI uses strong magnetic fields, magnetic field gradients, and radio waves for obtaining medical images. The quality of the medical image is crucial for a comparison with a reference image or for analysis of the medical image (e.g. determining of certain areas in an organ, etc.). The quality of the medical image depends among others on an alignment of desired object to be imaged and an imaging system. The information of the alignment sometimes may be incorrect or may change over the time and may therefore have negative effects on the quality of the resulting medical image.
The article “Marker-free optical stereo motion tracking for in-bore MRI and PET-MRI application” by A. Kyme et al., Medical Physics, vol. 47, no. 8 (1 Jun. 2020) discloses a method for prospective motion correction.
The article “Markerless motion tracking of awake animals in positron emission tomography” by A. Kyme et al., IEEE transactions on medical imaging, vol. 33, no. 11 (1 Nov. 2014) discloses a method for motion-compensation in positron emission tomography.
United States patent application US 2018/0325415 A1 discloses a method for measuring motion information from a human or animal subject during a medical imaging examination.
There may, therefore, be a need for an improved determining of a position of a subject in an MRI system. The object of the present invention is solved by the subject-matter of the independent claims, wherein further embodiments are incorporated in the dependent claims.
According to a first aspect, a method for motion correction of MR data is provided, comprising: generating, by a calculation unit, a three-dimensional model, 3D model, of a region of interest of a subject comprising at least one landmark inherent to the subject. The method further comprises obtaining, by a first measuring device, a two-dimensional image, 2D image, of at least a part of the subject inside a MRI system, wherein the measuring device is arranged at least partially inside a bore of the MRI system. In addition, the method comprises determining, by the calculation unit, at least one landmark in the 2D image, wherein the at least one landmark in the 2D image corresponds to the at least one landmark of the 3D model. The method further comprises determining, by the calculation unit, a position of the region of interest of the subject in the MRI system based on the determined at least one landmark in the 2D image and providing, by the calculation unit, the 3D position of the region of interest of the subject for motion correction of MR data.
The term MR data, as used herein, is to be understood broadly and relates to data associated with the MRI procedure. The MR data may relate to data used to control the MRI system in a preparation phase of the MRI procedure and/or in an operating phase of the MRI procedure. The MR data may comprise data and/or information used to generate an adaption signal and/or control signal for magnetic resonance gradients, radiofrequency pulses, and receiver frequency in a scanner of the MRI system. The motion correction of MR data in a preparation phase and/or operating phase of the MRI procedure is also known as prospective motion correction. Thereby, the preparation phase may precede the operating phase. The MR data may relate to MR images obtained, e.g. acquired, with the MRI procedure. The motion correction of MR images in a post processing phase of the MRI procedure is also known as retrospective motion correction.
The term calculation unit, as used herein, is to be understood broadly and means a unit configured to process data, in particular to determine one or more landmarks in a 2D image by data processing. The calculation unit may be a hardware unit (e.g. a controller, a workstation, a server) or a software unit (e.g. a virtual machine executed on a hardware unit), or a combination of hardware and software. The control unit may be in a single entity or distributed on several entities, wherein an entity may be a hardware unit and/or a software unit.
The term 3D model, as used herein, is to be understood broadly and relates to a model configured to describe a region of interest of a subject. The 3D model may be a point model, a line model, a surface model, or a volume model. The 3D model may, for example, comprise information of facial features, bones, tissues, organs, and/veins, wherein other body parts, organs, etc. may also be taken as information. The 3D model may be based on statistical information from e.g. a database or the like, considering e.g. age, sex, weight, or the like. For example, the 3D model may be based on an anatomy atlas. The 3D model may also be based on historical data from a subject to be imaged (e.g. previous images from a previous medical imaging exam). The 3D model may be described with e.g. a vector, wherein the vector may comprise 3D information of the at least one landmark, preferably of several landmarks.
The term subject, as used herein, means a human or animal. The region of interest, as used herein, relates to any part of a human or animal body (e.g. bone, tissue, organ, or combination thereof, wherein other body parts, organs, etc. may also be taken as information).
The term landmark, as used herein, is to be understood broadly and relates to at least one marker configured to be determined in a 2D image and an image with 3D data. The landmark may be a structural, e.g. inherent, component of the region of interest (e.g. a bone, eye, nose, hand etc.), a color combination or color transition (e.g. mole on left check), or a virtual marker adjacent to the region of interest (e.g. derived from two or more physical markers of the region of interest, e.g. two bones). The landmark is in particular not a separate entity that is attached to the subject, but is rather inherent to the subject. The landmark may be part of the subject or of the adjacent area visible in the 2D image. The landmark may be determined by an image analysis algorithm, for instance an edge detection algorithm. The landmark may be determined automatically by the image analysis algorithm in the 2D image, wherein the 2D image is continuously obtained by a video stream from an in-bore camera. In particular, the at least one landmark in the 2D image may be obtained by searching for the at least one landmark of the 3D model. As an example, the image analysis algorithm may specifically scan the 2D image for certain features of the at least one landmark of the 3D model. As another example, the image analysis algorithm may find landmarks in the 2D image and then match them to the at least one landmark of the 3D model. The landmark may preferably be predefined. In case more than one landmark is predefined, for example five, and inside the bore of the MRI system only e.g. four landmarks are determined, the method may continue the execution with the e.g. four landmarks.
The term first measuring device, as used herein, is to be understood broadly and relates to any measuring devices configured to obtain a 2D image of a part of a subject inside a bore of a MRI system. The first measuring device may be a sensor unit. The measuring device may be an optical camera sensor, an infrared sensor, a laser interferometer, or the like. The first measuring device may be one single entity or distributed on two or more entities, wherein an entity relates to a senor unit. The first measuring device may, for example, be an RGB sensor or a CCD sensor. The first measuring device may be an infrared camera or the like, continuously streaming from inside the bore. The first measuring device may obtain 2D images continuously or after discrete times. The first measuring device may be arranged in direct visual contact to the subject, in particular the part of the subject, more particularly the region of interest of the subject. The first measuring device may be arranged in indirect visual contact by means of a mirror to the subject, in particular the part of the subject, more particularly the region of interest of the subject. The first measuring device may be in wired connection (e.g. Ethernet) with the calculation unit, data storage, a server, a workstation. The first measuring may be in wireless in connection with the abovementioned entities (e.g. WIFI).
The MRI system, as used herein, relates to a state of the art MRI system configured to execute an MRI procedure. The MRI system, as used herein, comprises at least an MRI control configured to control the MRI system, a bore in which a movable support structure bearing a subject is positioned, one or more MR source coils, one or more MR detection coils. The MRI system may advantageously enhance by means of a first measuring device.
The term 3D position, as used herein, means at least the three translatory coordinates (e.g. x, y, z coordinates) of a single point of the region of the interest of the subject. The 3D position of one or more single points may reveal an orientation of the region of interest of the subject. The 3D position may comprise further three rotational coordinates.
The invention is based on the finding, that the knowledge of the 3D position of a subject, in particular, the knowledge of the position of the region of interest of the subject in relation to the MRI system (e.g. expressed in MRI system coordinates) in an MRI procedure is crucial for a quality of a resulting MR image. The resulting measuring data from the MRI procedure have to be further processed in order to obtain the MR image. The further processing for obtaining the MR image requires the 3D position of the region of interest of the subject. In case the used 3D position of the region of interest of the subject in further processing for obtaining the MR image is inaccurate, the resulting MR image is also inaccurate. The 3D position of the region of interest of the subject may change over the time, as the subject may inhale, exhale or simply moves. However, the changed 3D position of the region of interest will influence the meta data of MRI procedure, and therefore the corresponding obtained MR image in case no motion correction is carried out. The invention determines the 3D position of the region of interest of the subject inside the bore of the MRI system and uses the 3D position of the region of interest of the subject to correct the MR images after they were imaged (i.e. retrospective motion correction). Furthermore, the determined 3D position of the region of interest of the subject may also be used to adapt the MRI procedure, in particular the magnetic resonance gradients, radiofrequency pulses, and receiver frequency of MRI system before further MRI procedure (i.e. prospective motion correction). The determining of the 3D position of the region of interest of the subject is carried by a 2D image obtained by a state of the art measurement device, in particular a 2D camera sensor implemented inside the bore of MRI system. This may be advantageous as only a single 2D camera, in particular no 3D depth camera, is required. The 2D camera may be advantageously operated inside the magnetic field inside the bore of the MRI system in comparison to a 3D camera, e.g. depth camera. The invention enables the use of the simple but robust 2D camera by mapping information from the obtained 2D image to the 3D model and thereby determining the 3D position of the region of interest of the subject. The invention enables Magnetic resonance motion artefact correction using a single in-bore camera and an additional depth camera arranged outside the bore (e.g. in the scanner room).
In an embodiment, the method further may comprise obtaining at least one modelling image of the subject, by a second measuring device arranged outside the bore of the medical imaging system, wherein the at least one modelling image is used for generating the 3D model of the region of interest of the subject. The term modelling image, as used herein, means that modelling is merely used for generating the 3D model of the region of interest of the subject in a preparation phase of the MRI procedure. For instance, a subject may lie on a support structure outside the bore of an MRI system and the second measuring device is arranged over the subject (e.g. at the ceiling). The measuring device may take one or more modelling images of the region of interest from one or more perspectives. The 3D model may comprise a default 3D model with default dimensions which is adapted by the at least one modelling image of the region of interest of the subject. Actual dimensions of the region of interest of the subject may directly be obtained from the modelling image by analysis of the modelling image. The analysis may comprise utilizing e.g. a neural network in case the modelling image comprises merely 2D data. The analysis may comprise simply a reading from the modelling image in case the modelling image comprises 3D data. The 3D model may be newly generated, e.g. during performing the method described herein, etc., which means that no default 3D model is present. In sum, this may be advantageous as the accuracy of the 3D model may increase and therefore the resulting quality of the MR data. The modelling image may be continuously received from the second measuring device. The modelling image may be continuously analyzed to detect the at least one landmark, wherein the analyzing may comprise one or more mathematical algorithms. The at least one landmark may be predefined for the region of interest (e.g. a cheek bone for the head). The mathematical algorithm may comprise an edge detection algorithm, a neural network trained for this purpose, or other suitable calculation methods. The mathematical algorithm may determine whether the subject is on a bed of the MRI system, and/or whether the subject is outside or inside the bore of MRI system. The mathematical algorithm may determine different parts of the subject (e.g. head, leg, arm, etc.). The mathematical algorithm may use any data stream from the second measuring device (e.g. 2D information (RGB output from depth camera), 3D information from a depth camera or combination thereof). The mathematical algorithm may first detect a predefined region of interest (e.g. head) and then a predefined at least one landmark (e.g. a cheek bone). In ease more than one modelling image is obtained, generating the 3D model of the region of interest of the subject may comprise the calculation of an average 3D model. In case more than one modelling image is obtained, generating the 3D model of the region of interest of the subject may comprise a merger of different 3D models of the region of interest of the subject.
In an embodiment, the at least one modelling image may comprise 3D data of the region of interest of the subject. The 3D data may comprise for example translatory coordinates (x, y, z direction) of each pixel in the modelling image. The 3D data may be obtained from a depth camera, a laser interferometer scanner, and/or two or more 2D cameras (e.g. two RGB sensor cameras in different perspectives that enable computer stereo vision). Computer stereo vision is the extraction of 3D information from digital images. By comparing information from a region of interest from two perspectives, 3D data can be extracted by examining relative positions of objects (e.g. landmarks) in the digital images. The 3D data of the region of interest of the subject may advantageously increase the accuracy of the 3D model and therefore of the resulting MR data. The 3D data from the modelling images may be used to obtain corresponding 3D spatial positions in a suitable coordinate system (e.g. MRI coordinate system, region of interest (e.g. head) coordinate system). For instance, a patient-centered coordinate system may be used, where the at least one landmark is the origin of the coordinate system. The coordinate system may comprise homogenous coordinates in order to simplify the calculation.
In an embodiment, the second measurement device may be a depth camera. The depth camera may advantageously provide very accurate 3D information data of the region of interest of the subject. The depth camera may use a time of flight principle to determine 3D information from a 2D image. The depth camera may use structured light to determine 3D information from a 2D image. The depth camera may use coherent light and measure a phase shift between of reflected light relative to a source light (i.e. laser interferometry).
In an embodiment, the second measuring device may be at least one optical camera. The optical camera may be a RGB camera. The RGB camera provides merely a 2D image. Hence, either two images from two different perspectives are necessary to derive 3D data from an RGB camera (i.e., computer stereo vision, which preferably requires at least two optical cameras) or the 2D data from the 2D image obtained by the RGB camera has to be aligned to a 3D model. The second option may be carried out by means of a mathematical algorithm. The mathematical algorithm may be trained to process one or more inputs into one or more outputs by means of an internal processing chain that typically has a set of free parameters. The internal processing chain may be organized in interconnected layers that are traversed consecutively when proceeding from the input to the output. The mathematical algorithm may be trained by using records of training data. A record of training data comprises training input data and corresponding training output data. Training input data as used herein, may be 2D images from the region of interest of the subject and training output data may be 3D data of the region of interest of the subject (e.g. measured with a 3D depth camera). The training input data and training output data may also be simulated data in order to reduce the effort of providing training data. In sum, this may advantageous in terms cost reduction and accuracy of the 3D model. The optical camera may be an infrared camera. The optical camera may use a charged-coupled device (CCD) sensor or an active-pixel sensor (i.e. complementary metal-oxide-semiconductor (CMOS) sensor).
In an embodiment, the generating of the 3D model may be based on a machine learning system representing a mathematical algorithm processing at least one landmark of a region of interest of a subject, wherein the machine learning system is trained to describe a relation between geometrical data of a region of interest of a subject and at least one landmark of the region of interest of the subject. The training data may be derived from records of 2D images and corresponding 3D images. The 2D images and corresponding 3D images may be simulated in order to reduce the effort providing training data. The machine learning system may be implemented by a neural network, machine learning algorithm, convolutional neural network, or a generative adversarial network. The use of the machine learning system may be advantageous in terms of efficiency and accuracy of generating the 3D model.
In an embodiment, the determining of the position of the region of interest of the subject in the MRI system may comprise determining of one or more rotations and one or more translations of the 3D model in relation to a position of the first measuring device in order to obtain a projection of the 3D model that fits to the 2D image obtained by the first measuring device inside the bore. In other words, the method checks what 3D position of the region of interest of the subject may lead to the obtained projection of the region of interest of the subject (i.e. 2D image). The determining therefore may further consider the position of the first measuring device inside the bore. The determining may calculate in reference from the position of the first measuring device inside the bore a perspective. The determining may use physical equations of intercept theorems. The determining may use numerical approximation method to determine the position of the subject in the MRI system. The determining may further use a neural network, which may be trained for this purpose, to determine the position of the subject in the MRI system. In sum, this may be advantageous to accurately determine the position the subject in the MRI system.
In an embodiment, the first measuring device may be arranged in a coil, or in housing or the like thereof. The arrangement of the first measuring device in a coil may be advantageous as nothing may hide or obscure the view of between the measuring device and the region of interest of the subject. The coil may be a head coil, or another body coil. The coil may be a stationary coil inside of MRI system. The first measuring may also be mounted on the ceiling of the bore. The first measuring device may also change its perspective in dependency of the subject and/or the region of interest (e.g. different sizes of the subject require different perspectives). The measuring device may also be arranged in combination with one or more mirrors in order to image concealed areas (e.g. underside chin). In case a mirror is used also the position of the mirror is used beside the position of the measuring device for determining the position of the subject in the MRI system.
In an embodiment, the modelling image may be used to obtain the corresponding 3D positions in a suitable coordinate system. For example, a patient-centered coordinate system may be used, where one landmark of a plurality of predefined landmarks is taken to be the origin. Since multiple modelling images of the region of interest may usually be available during an exam preparation, the resulting 3D model can be averaged and stitched to improve accuracy. Using homogeneous coordinates, the resulting individual reference landmark positions may be described by a vector
To describe the 3D model, these positions may be aggregated into a single reference vector
Once the subject has been moved into the bore, the one or more landmarks are determined on the 2D image inside the bore of the MRI system. In case only a subset of the plurality of landmarks are determined with high accuracy, all calculations may be restricted to this subset of landmarks by simply dropping the non-detected landmark variables. The detected in-bore landmark positions may be described by (again in homogeneous coordinates) a vector
A further in bore 2D model may be obtained by aggregating these vectors to
The parameters describing the transformation of the reference 3D vector to the in-bon; 2D vector are then found by solving:
C is a block-diagonal matrix consisting of 3×4 matrices that describe the projection realized by the in-bore camera, and Mθ is a block-diagonal matrix describing the 3D transformation from the reference vector to the 3D vector inside the bore (parametrized by θ). For a rigid motion model, Mθ=Tθ
Δθ=θ(i)-θ(0)
These 3D positions may then be further used for motion correction. In case of retrospective motion correction, the parameters may be stored time synchronized with the acquired MR data profiles for each acquired camera image of a particular MR scan, and sent to the reconstruction software once the scan is completed. In case of prospective motion correction, the calculated motion parameters may be directly sent to the scanner software to adjust the Magnetic resonance gradients, radiofrequency pulses, and receiver frequency and phase to account for the motion.
In an embodiment, the first measuring device may be an optical camera. The optical may be an RBG camera. The RGB camera is a measuring device with a robust operating behavior in magnetic environments such a MRT system. This may be advantageous in terms of robustness and reliability of the method. The optical camera may be an infrared camera. This may also be advantageous in terms of robustness and reliability of the method. It shall be noticed here that a single optical camera as first measuring device may be sufficient to execute the method. However, it may also be advantageous to use more than RGB camera in the bore in order to get a better coverage of the region of interest of the subject in the MRI system.
In an embodiment, the region of interest of the subject may be a head of the subject.
In an embodiment, generating the 3D model of the subject may be based on an anatomical model, preferably a 3D morphable model. This may be advantageous in terms of the accuracy of the 3D model. The 3D morphable model may be advantageous in case the region of interest is a head or a face. The 3D morphable model is a generative model, which may be established on set of example faces or heads in a registration procedure. The 3D morphable model may be a statistical model of a distribution of the example faces or heads.
In an embodiment, the position of the region of interest of the subject may continuously be determined and provided for the motion correction of the MR data. The continuous execution of the method may require real time capable hardware components for calculation and data exchange. The calculation may be carried out by out by a work station, a FPGA, high-performance computer, a data center. The data exchange between first measuring device, second measuring device, control of the MRI system, and the calculation unit may be carried out by a third generation Bus system. Ethernet, and a Fast-Ethernet-Hub.
According to a further aspect, an apparatus for motion correction of MR data is provided, comprising: a generating unit, configured to generate a 3D model of a region of interest of a subject comprising at least one landmark; an obtaining unit, configured to obtain a 2D image of at least a part of the subject inside a MRI system, wherein the obtaining unit is arranged inside a bore of the MRI system; a first determining unit, configured to determine at least one landmark in the 2D image, wherein the at least one landmark in the 2D image corresponds to the at least one landmark of the 3D model; a second determining unit, configured to determine a position of the region of interest of the subject in the MRI system based on the determined at least one landmark in the 2D image; a providing unit, configured to provide the position region of interest of the subject for a motion correction of MR data. The generating unit, the first determining unit, the second determining unit, the providing unit may be separate hardware units or separate software units that run on one or more hardware units, or combinations thereof. The hardware unit may be a controller, a computer, a server, a workstation. The data exchange between the one or more hardware units may wired (e.g. Ethernet, Profinct) or wireless (e.g. WIFI, WLAN).
According to a further aspect, a system for medical imaging is provided, comprising: an apparatus described above; an MRI system; a first camera, configured to obtain a 2D image inside a MRI system; and optionally a second camera, configured to obtain an image outside the MRI system. The second camera may be depth camera. The first camera may be a RGB camera.
According to a last aspect a computer program element is provided, which when executed by a processor is configured to carry out the steps of a method described above. The processor may be part of the medical imaging system, or may be provided separately in another computer device. The computer program element might be stored on a computer unit, which might also be part of an embodiment. This computing unit may be configured to perform or induce performing of the steps of the method described above. Moreover, it may be configured to operate the components of the above described device. The computing unit can be configured to operate automatically and/or to execute the orders of a user. A computer program may be loaded into a working memory of a data processor. The data processor may thus be equipped to carry out the method according to one of the preceding embodiments. This exemplary embodiment of the invention covers both, a computer program that right from the beginning uses the invention and computer program that by means of an update turns an existing program into a program that uses invention. Further on, the computer program element might be able to provide all necessary steps to fulfill the procedure of an exemplary embodiment of the method as described above. According to a further exemplary embodiment of the present invention, a computer readable medium, such as a CD-ROM, USB stick or the like, is presented wherein the computer readable medium has a computer program element stored on it which computer program element is described by the preceding section. A computer program may be stored and/or distributed on a suitable medium, such as an optical storage medium or a solid state medium supplied together with or as part of other hardware, but may also be distributed in other forms, such as via the internet or other wired or wireless telecommunication systems. However, the computer program may also be presented over a network like the World Wide Web and can be downloaded into the working memory of a data processor from such a network. According to a further exemplary embodiment of the present invention, a medium for making a computer program element available for downloading is provided, which computer program element is arranged to perform a method according to one of the previously described embodiments of the invention.
It is noted that the above embodiments may be combined with each other irrespective of the aspect involved. Accordingly, the method may be combined with structural features of the device and/or system of the other aspects and, likewise, the device and the system may be combined with features of each other, and may also be combined with features described above with regard to the method.
These and other aspects of the present invention will become apparent from and elucidated with reference to the embodiments described hereinafter.
Exemplary embodiments of the invention will be described in the following drawings.
The system 10 for medical imaging comprises an apparatus 11 (see
In another exemplary embodiment, a computer program or computer program element is provided that is configured to execute the method steps of the method according to one of the preceding embodiments, on an appropriate device or system.
The computer program element might therefore be stored on a data processing unit, which might also be part of an embodiment. This data processing unit may be configured to perform or induce performing of the steps of the method described above. Moreover, it may be configured to operate the components of the above described device and/or system. The computing unit can be configured to operate automatically and/or to execute the orders of a user. A computer program may be loaded into a working memory of a data processor. The data processor may thus be equipped to carry out the method according to one of the preceding embodiments.
Further, the computer program element might be able to provide all necessary steps to fulfill the procedure of an exemplary embodiment of the method as described above.
According to a further exemplary embodiment of the present invention, a computer readable medium, such as a CD-ROM, USB stick or the like, is presented wherein the computer readable medium has a computer program element stored on it which computer program element is described by the preceding section.
A computer program may be stored and/or distributed on a suitable medium, such as an optical storage medium or a solid state medium supplied together with or as part of other hardware, but may also be distributed in other forms, such as via the internet or other wired or wireless telecommunication systems.
However, the computer program may also be presented over a network like the World Wide Web and can be downloaded into the working memory of a data processor from such a network. According to a further exemplary embodiment of the present invention, a medium for making a computer program element available for downloading is provided, which computer program element is arranged to perform a method according to one of the previously described embodiments of the invention.
It is noted that embodiments of the present disclosure are described with reference to different subject matter. In particular, some embodiments are described with reference to method type claims whereas other embodiments are described with reference to the device type claims. However, a person skilled in the art will gather from the above and the following description that, unless otherwise notified, in addition to any combination of features belonging to one type of subject matter also any combination between features relating to different subject matters is considered to be disclosed with this application. However, all features can be combined providing synergetic effects that are more than the simple summation of the features.
While the invention has been illustrated and described in detail in the drawings and the foregoing description, such illustration and description are to be considered illustrative or exemplary and not restrictive. The invention is not limited to the disclosed embodiments. Other variations to the disclosed embodiments can be understood and effected by those skilled in the art in practicing a claimed invention, from a study of the drawings, the disclosure, and the dependent claims.
In the claims, the word “comprising” does not exclude other elements or steps, and the indefinite article “a” or “an” does not exclude a plurality. A single processor or other unit may fulfill the functions of several items re-cited in the claims. The mere fact that certain measures are re-cited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage. Any reference signs in the claims should not be construed as limiting the scope.
| Number | Date | Country | Kind |
|---|---|---|---|
| 21180516.3 | Jun 2021 | EP | regional |
| Filing Document | Filing Date | Country | Kind |
|---|---|---|---|
| PCT/EP2022/066701 | 6/20/2022 | WO |
