Myocardial strain imaging is sensitive and prognostic for the assessment of heart disease, with potential advantages over imaging of ejection fraction (EF)1. Among various strain imaging methods, cine displacement encoding with stimulated echoes (DENSE)2-4 magnetic resonance imaging (MRI) uniquely measures heart motion by encoding myocardial displacement into the signal phase, which can facilitate high measurement accuracy5, high reproducibility of global and segmental strain6,7, and rapid computation of displacement and strain5,8. These properties translate to benefits in clinical performance. For example, cine DENSE shows superiority over late gadolinium enhanced (LGE) MRI and feature tracking in predicting major adverse cardiac events after myocardial infarction9 and predicting outcomes of heart failure (HF) patients treated with cardiac resynchronization therapy10. Cine DENSE also detects contractile dysfunction in childhood obesity11 and adult type 2 diabetes even when EF is normal12.
While low-rank13,14 and reduced field-of-view15 methods have been developed recently to accelerate data acquisition for DENSE, there remains a need and opportunity to accelerate DENSE strain analysis and to eliminate all steps that require user assistance. Currently, LV segmentation of DENSE is typically performed using motion-guided segmentation8, which can require manual segmentation of the LV epicardial and endocardial borders at a single cardiac phase, followed by automated propagation of these borders to all other phases (guided by the measured myocardial displacements). User intervention is sometimes needed to adjust the segmentation results. Identification of the anterior RV insertion point is currently performed manually by a user. Also, phase unwrapping is typically performed using a path-following method5, and this method requires user selection of seed points placed in regions known to not have phase wrapping.
It is with respect to these and other considerations that the various aspects of the present disclosure as described below are presented.
In one aspect, the present disclosure relates to a method of strain analysis of a cardiac region of interest of a subject from displacement encoded magnetic resonance image (MRI) data. In one embodiment, the method includes the steps of acquiring displacement encoded MRI data corresponding to the cardiac region of interest of the subject and generating a phase image for each frame of the displacement encoded MRI data, wherein the phase image includes potentially phase-wrapped measured phase values corresponding to pixels of the frame. The method includes training a convolutional neural network (CNN) to compute a wrapping label map for the phase image, wherein the wrapping label map includes a respective number of phase wrap cycles present at each pixel in the phase image, allowing for computing, by the trained CNN, the wrapping label map. Computing an unwrapped phase image includes adding a respective phase correction to each of the potentially-wrapped measured phase values of the phase image, wherein the phase correction is based on the number of phase wrap cycles present at each pixel. Computing myocardial strain follows by using the unwrapped phase image for strain analysis of the subject.
In one embodiment, the strain analysis includes quantification of global and segmental strain associated with the heart of the subject.
In one embodiment, the displacement encoded MRI data corresponds to displacement encoded stimulated echo (DENSE) cine frames of MRI image data.
In one embodiment, a U-Net structured CNN is used to compute the wrapping label map.
In one embodiment, at least one additional CNN is configured for epicardial and endocardial segmentation, and the at least one additional CNN assigns one of three classes to each pixel, wherein the three classes are the blood pool, the myocardium, and the background.
In one embodiment, computing the wrapping label map includes labeling each pixel as belonging to one of three classes, the classes including no-wrap, −2π wrapped, and +2π wrapped.
In one embodiment, the method includes displaying a visual representation of the phase image according to the respective class and label.
In one embodiment, at least one trained CNN is trained at least in part from augmented test data from previously verified test images produced by phase unwrapping the previously verified test image, multiplying a phase unwrapped verified test image by a constant, and phase wrapping a product test image within a range of −π to +π to generate a new wrap test image.
In one embodiment, the method further includes using at least one additional CNN to (a) identify the left-ventricular (LV) epicardial and endocardial borders; and (b) identify the interior right ventricular-LV insertion point.
In one embodiment, the method further includes using at least one additional CNN to generate (a) segmentation of the LV myocardium; (b) identification of the anterior right-ventricular (RV) insertion point into the LV; and (c) an unwrapped phase image by unwrapping of the potentially-wrapped displacement encoded phase values of the myocardium.
In one embodiment, the method further includes (d) computing the spatiotemporal displacement field of the unwrapped phase image.
In one embodiment, the potentially-wrapped measured phase values correspond to pixel (i, j) of the frame, and the wrapping label map includes values of respective wrapping constants kij for each pixel (i, j) in the phase image. The respective phase correction for each pixel (i, j) is computed by multiplying each value kij by 2π, and the unwrapped phase image is computed by adding the phase correction for each pixel (i, j) to each of the potentially-wrapped measured phase values of the phase image.
In one embodiment, the frames of the displacement encoded MRI data include image frames having displacement encoded data generated with multiple cycles of phase wrapping.
In one embodiment, the method uses the trained CNN to estimate the number of cycles of wrapping corresponding to the phase image during displacement encoding that produced the displacement encoded MRI data.
In one embodiment, the method includes converting the unwrapped phase image to a respective displacement array.
In one aspect, the present disclosure relates to a method of using a convolutional neural network (CNN) to calculate a wrapping label map for unwrapping an array of potentially-wrapped measured phase values from frames of magnetic resonance image (MRI) data. In one embodiment, the method includes calculating a phase image for each frame of the displacement encoded MRI data, the phase image including potentially-wrapped measured phase values corresponding to pixels (i, j) of the frame. The method further includes training the convolutional neural network with augmented test data to label each pixel (i, j) as belonging to one of three classes, wherein the classes include a no-wrap label, a −2π label, and a +2π label. The method includes storing the respective labels in a wrapping label map.
In one embodiment, the method further includes generating the augmented test data from previously verified test images by phase unwrapping the previously verified test image, multiplying a phase unwrapped verified test image by a constant, and phase wrapping a product test image within a range of −π to +π to generate a new wrap test image.
In one embodiment, the method applies randomly chosen serial image operations to the new wrap test image to develop additional test images for training the CNN.
In one embodiment, the serial image operations include at least one of deformations, rotations, and noise addition.
In one embodiment, the frames of MRI data may be image frames having displacement encoded data generated by applying multiple cycles of phase wrapping operations.
In one aspect, the present disclosure relates to a system which, in one embodiment, includes a data acquisition device configured to acquire displacement encoded magnetic resonance image (MRI) data corresponding to a cardiac region of interest of a subject. The system also includes a computer-implemented convolutional neural network (CNN), and one or more processors coupled to the data acquisition device and the CNN. The processor(s) are configured to cause the system to perform functions that include generating a phase image for each frame of the displacement encoded MRI data, wherein the phase image includes potentially phase-wrapped measured phase values corresponding to pixels of the frame; training a convolutional neural network (CNN) to compute a wrapping label map for the phase image, wherein the wrapping label map incorporates a respective number of phase wrap cycles present at each pixel in the phase image; and computing, by the trained CNN, the wrapping label map. Computing an unwrapped phase image includes adding a respective phase correction to each of the potentially-wrapped measured phase values of the phase image, wherein the phase correction is based on the number of phase wrap cycles present at each pixel. The system computes myocardial strain using the unwrapped phase image for strain analysis of the subject.
In one aspect, the present disclosure relates to a non-transitory computer-readable medium storing instructions which, when executed by one or more processors, cause one or more computing devices to perform functions for strain analysis of a cardiac region of interest of a subject from displacement encoded magnetic resonance image (MRI) data. In one embodiment, the performed functions include acquiring displacement encoded MRI data corresponding to the cardiac region of interest of the subject; generating a phase image for each frame of the displacement encoded MRI data, wherein the phase image includes potentially phase-wrapped measured phase values corresponding to pixels of the frame; and training a convolutional neural network (CNN) to compute a wrapping label map for the phase image, wherein the wrapping label map includes a respective number of phase wrap cycles present at each pixel in the phase image. The instructions are further configured for computing, by the trained CNN, the wrapping label map and computing an unwrapped phase image by adding a respective phase correction to each of the potentially-wrapped measured phase values of the phase image, wherein the phase correction is based on the number of phase wrap cycles present at each pixel. The product allows for computing myocardial strain using the unwrapped phase image for strain analysis of the subject.
Other aspects and features according to the example embodiments of the present disclosure will become apparent to those of ordinary skill in the art, upon reviewing the following detailed description in conjunction with the accompanying figures.
Reference will now be made to the accompanying drawings, which are not necessarily drawn to scale. The patent or application file contains at least one drawing executed in color. Copies of this patent or patent application publication with color drawing(s) will be provided by the Office upon request and payment of the necessary fee.
In some aspects, the present disclosure relates to systems, methods, and computer-readable medium for phase unwrapping for displacement encoding with stimulated echoes (DENSE) MRI using deep learning. Although example embodiments of the disclosed technology are explained in detail herein, it is to be understood that other embodiments are contemplated. Accordingly, it is not intended that the disclosed technology be limited in its scope to the details of construction and arrangement of components set forth in the following description or illustrated in the drawings. The disclosed technology is capable of other embodiments and of being practiced or carried out in various ways.
It must also be noted that, as used in the specification and the appended claims, the singular forms “a,” “an” and “the” include plural referents unless the context clearly dictates otherwise. Ranges may be expressed herein as from “about” or “approximately” one particular value and/or to “about” or “approximately” another particular value. When such a range is expressed, other exemplary embodiments include from the one particular value and/or to the other particular value.
By “comprising” or “containing” or “including” is meant that at least the named compound, element, particle, or method step is present in the composition or article or method, but does not exclude the presence of other compounds, materials, particles, method steps, even if the other such compounds, material, particles, method steps have the same function as what is named.
In describing example embodiments, terminology will be resorted to for the sake of clarity. It is intended that each term contemplates its broadest meaning as understood by those skilled in the art and includes all technical equivalents that operate in a similar manner to accomplish a similar purpose. It is also to be understood that the mention of one or more steps of a method does not preclude the presence of additional method steps or intervening method steps between those steps expressly identified. Steps of a method may be performed in a different order than those described herein without departing from the scope of the disclosed technology. Similarly, it is also to be understood that the mention of one or more components in a device or system does not preclude the presence of additional components or intervening components between those components expressly identified.
Some references, which may include various patents, patent applications, and publications, are cited in a reference list and discussed in the disclosure provided herein. The citation and/or discussion of such references is provided merely to clarify the description of the disclosed technology and is not an admission that any such reference is “prior art” to any aspects of the disclosed technology described herein. All references cited and discussed in this specification are incorporated herein by reference in their entireties and to the same extent as if each reference was individually incorporated by reference.
As discussed herein, a “subject” (or “patient”) may be any applicable human, animal, or other organism, living or dead, or other biological or molecular structure or chemical environment, and may relate to particular components of the subject, for instance specific organs, tissues, or fluids of a subject, may be in a particular location of the subject, referred to herein as an “area of interest” or a “region of interest.”
Throughout the description, the following abbreviations may be used:
DENSE—Displacement Encoding with Stimulated Echoes;
MRI—Magnetic Resonance Imaging;
CNN—Convolutional Neural Network;
LV—Left Ventricular;
RV—Right Ventricular;
HF—Heart Failure;
EF—Ejection Fraction;
DL—Deep Learning;
MSD—Mean Surface Distance;
MSE—Mean Squared Error;
SNR—Signal to Noise Ratio;
NW—No Wrap; and
Ecc—Circumferential Strain.
A detailed description of aspects of the present disclosure, in accordance with various example embodiments, will now be provided with reference to the accompanying drawings. The drawings form a part hereof and show, by way of illustration, specific embodiments and examples. In referring to the drawings, like numerals represent like elements throughout the several figures.
Embodiments of the present disclosure include DL-based fully-automated methods for global and segmental strain analysis of short-axis DENSE MRI from a multicenter dataset. U-Nets were designed, trained and found to be effective for LV segmentation, identification of the anterior RV-LV insertion point, and phase unwrapping. Steps involving displacement and strain calculations can be automated, thus, with the DL methods, the entire DENSE analysis pipeline for global and segmental strain can be fully automated. Identification of the anterior RV insertion point, and phase unwrapping, and remaining steps to compute displacement and strain can also be performed automatically without user assistance, as described herein4,5,17,18
Embodiments of the present disclosure include a fully-automated post-processing approach for cine displacement encoding with stimulated echoes (DENSE). Deep learning (DL) methods, particularly convolutional neural networks (CNN), can be used segmentation and analysis of various CMR techniques19,20,29,21-28. Some embodiments of the present disclosure include a pipeline for fully-automated analysis of cine DENSE data using four CNNs to (a) identify the LV epicardial border, (b) identify the LV endocardial border, (c) identify the anterior RV-LV insertion point, and (d) after LV segmentation, perform phase unwrapping of the LV myocardium. Embodiments of the present disclosure include a pipeline that can eliminate all user intervention and can reduce the time for image analysis.
Embodiments of the present disclosure include a fully-automatic DENSE analysis pipeline. Some embodiments of the present disclosure include the following general steps: (a) LV segmentation, (b) identification of the anterior RV-LV insertion point, (c) phase unwrapping, and (d) displacement and strain analysis. Steps (a)-(c) can utilize CNNs, and step (d) can use other fully-automatic methods5,31.
With reference to
In some embodiments, the MRI images may be subject to segmentation operations, including but not limited to those set forth in U.S. patent application Ser. No. 16/295,939 filed on Mar. 7, 2019, and published as United States Pub. No. 2019/0279361, which is incorporated by reference herein. This disclosure utilizes segmented images of the epicardial contour and endocardial contour, such as the segmented images illustrated in
At step 104, a phase image, which may be stored in a computer as a phase image or phase image array or matrix, is generated for each frame, including a phase value corresponding to the pixels of each frame. The method can include generating a phase image for each frame of the displacement encoded MRI data. A chart showing a non-limiting example of the labels used for different types of wrapping is shown in
At step 106, a convolutional neural network (CNN) is trained to compute a wrapping label map for the phase image, where the wrapping label map includes a number of phase wrap cycles present at each pixel in the phase image. The wrapping label map can, for example use the labels shown in
At step 108, the CNN is used to compute a wrapping label map as shown in
An unwrapping factor can be calculated for each region classified by the CNN, based on the classification of each region. As a non-limiting example, in some embodiments of the present disclosure, every “cycle” of wrapping corresponds to the phase being 2π off from the “true” phase value. Therefore, based on the classification of each pixel as being wrapped or not, and in which direction the phase is wrapped (i.e. in the positive or negative direction), the appropriate unwrapping factor can be calculated for each pixel.
At step 110, therefore, the method includes computing an unwrapped phase image by adding a respective phase correction to each of the potentially-wrapped measured phase values of the phase image, wherein the phase correction is based on the number of phase wrap cycles present at each pixel.
In phase-reconstructed MR images, the phase value is inherently confined to the range (−2π, 2π). However, in cardiac DENSE in order to balance displacement sensitivity, signal-to-noise ratio, and suppression of artifact-generating signals, displacement-encoding frequencies that lead to phase shifts of greater than 2π are typically used, and ±1 cycle of phase wrapping typically occurs during systole 5. Thus, phase unwrapping can be required to convert phase to displacement.
The unwrapped phase ψij can be estimated from the potentially-wrapped measured phase φij as follows:
ψij=φij+2πkij
where kij is an integer and where −2π<φij<2π. According to some embodiments of the present disclosure phase unwrapping problem requires determining kij for each pixel indexed by i and j. Thus, the phase unwrapping can be defined as a semantic segmentation problem35, and the network can label each pixel as belonging to one of at least three classes (no wrap, −2π wrapped, or +2π wrapped) as shown in
At step 112, and with the unwrapping complete, the method of this disclosure may be used to compute myocardial strain using the unwrapped phase image for strain analysis of the subject.
To create the ground truth for unwrapped phase images, a highly accurate but very slow phase unwrapping method based on multiple phase prediction pathways and region growing can be used36. Additionally, a user can also check the results of this method, frame by frame, and discard all frames with unwrapping errors. The same dilated U-Net structure with three output classes was trained using a pixel-wise cross-entropy loss function. The network's input was the segmented phase-reconstructed DENSE image and the output was the wrapping label map. With this design, after applying the CNN, the value of kij is known for each pixel. Then by multiplying kij by 2π and adding the result to the input wrapped image, the unwrapped phase is computed .
Based on whether there is +2π wrapping or −2π wrapping, the appropriate +2π or −2π phase correction can be added to the image to com, to compute 110 an accurate output image, as shown in
The CNN can be used to generate a more accurate wrapping label map than path-following approaches. As shown in
Similarly,
Additionally, embodiments of the present disclosure can perform phase unwrapping for images with more than one “cycle” of phase wrapping. For example, with reference to
Optionally, these 5 types of wrapping can correspond to the following classifications: 1—no wrap (k=0), 2—(−2π) wrap (k=−1), 3—(+2π) wrap (k=+1), 4—(−4π) wrap (k=−2), and 5—(+4π) wrap (k=+2). It should be understood that these classifications are intended only as non-limiting examples, and that different numbers of classifications and different systems for naming, labeling, and organizing classifications are contemplated by the present disclosure. Similarly, it should be understood that in embodiments of the present disclosure capable of performing more than two cycles of phase unwrapping, that more than 5 classifications can be used.
With reference to
At step 152, Phase encoded MRI data corresponding to the cardiac region of interest of the subject is acquired. The MRI data can be acquired using a Cine DENSE image acquisition protocol. Optionally, segmentation can be performed including LV-epicardial segmentation 154, LV-endocardial segmentation 156, and LV-myocardial segmentation 158.
LV Segmentation 154, 156, 158 can be performed using a convolutional neural network. Embodiments of the present disclosure implement a 2D U-Net approach to LV segmentation [e.g. 19-22,24,26,28], LGE27, T1-weighted MRI25 and phase contrast23. Three-dimensional convolutions may have advantages for segmentation of cine MRI data through time; however, they can be less well studied for cardiac cine MRI than 2D and can present unique issues (e.g. they can require a constant number of cardiac phases). For cine MRI, to date most studies use a 2D model and achieve very good results26,28,41. Since 2D models work well and DICE values can be reasonably good using a 2D approach, a 2D U-Net can be used. Also, values for HD and MSD can be similar to the mean contour distance of 1.14 mm and HD of 3.16-7.25 mm for myocardial segmentation reported by others19, and to the average perpendicular distance of 1.1±0.3 mm also reported by others26. Embodiments of the present disclosure use two separate U-Nets for epicardial and endocardial segmentation, although in some applications training one network for myocardial segmentation based on the proposed network architecture can result in the same performance. Optionally, three classes of the blood pool can be defined, myocardium and background and to assign class weights of 3, 5 and 1, respectively, which can overcome the imbalanced classes problem.
To create the ground-truth LV segmentation data, manual image annotation can be performed for DENSE magnitude-reconstructed images. The LV endocardial and epicardial borders can be manually traced for all frames using DENSEanalysis software17. To automatically segment the LV from DENSE magnitude images, one U-Net was trained to extract the epicardial border, and another to extract the endocardial border, and the myocardial pixels can be identified by performing a logical XOR between the two masks. The 2D U-Net networks utilized the structure presented by Ronneberger32 with modifications to get the best results for the proposed application. Specifically, in the contracting path, each encoding block can contain two consecutive sets of dilated convolutional layers with filter size 3×3 and dilation rate 2, a batch normalization layer and a rectified linear activation layer. Compared with traditional convolutions, dilated convolutions can increase the receptive field size without increasing the number of parameters and showed improved performance in our experiments. Padding can be used in each convolutional operation to maintain the spatial dimension. Between each encoding block, pooling layers with step size of 3×3 and stride 2 were applied to reduce the spatial dimension in all directions. The number of features can be doubled for the next encoding block.
Four symmetric encoding and decoding blocks were used in the contracting and expanding path, respectively. Each decoding block can contain two consecutive sets of deconvolutional layers with filter size 3×3, a batch normalization layer and a rectified linear activation layer. The output of each encoding block in the contracting path was concatenated with those in the corresponding decoding block in the expanding path via skip-connections. The final segmentation map can include two classes: background and endocardium or epicardium. The loss function can be the summation of the weighted pixel-wise cross entropy and soft Dice loss. The assigned class weights were 1 for background, 2 for endocardium in the endocardial network and 3 for the epicardial network. During training, data augmentation on-the-fly was performed by applying random translations, rotations and scaling followed by a b-spline-based deformation to the input images and to the corresponding ground-truth label maps at each iteration. This type of augmentation has the advantage that the model sees different data at each iteration. The use of other network configurations, including networks with different numbers of layers, different filter sizes, stride numbers and dilation rates, is contemplated by the present disclosure, and the above are intended only as non-limiting examples of network parameters that can be used for segmentation.
In one embodiment of the present disclosure, 400 epochs were used to train each network; therefore, each image was augmented 400 times. After applying the random transformations to the label maps, a threshold value of 0.5 was applied to the interpolated segmentation to convert back to binary values33. To improve the accuracy and smoothness of the segmented contours, during testing, each image can be rotated 9 times at an interval of 40 degrees and the corresponding output probability maps were rotated back and averaged34. Hereafter, this testing process is described in the present disclosure as “testing augmentation”. It should be understood that the number of rotations (9), the interval (of 40 degrees), the number of epochs (400), and the threshold value (0.5) as well as this order and selection of steps for testing augmentation, are included only as non-limiting examples of ways to improve the accuracy of he described network, and that the use of other training techniques is contemplated.
Based on the segmentation 104106108 the RV-LV insertion point can be identified 110. The anterior RV-LV insertion point is the location of the attachment of the anterior RV wall to the LV, and its location defines the alignment of the American Heart Association 16-segment model16 which can be used for segmental strain analysis of the LV. As the first frame of cine DENSE images can have poor blood-myocardium contrast, a U-Net is trained to detect the anterior RV-LV insertion point on early-systolic frames (e.g. frames 5 and 6), where the insertion point is reliably well visualized. To create the ground-truth data, an expert user can identify one point in these frames from magnitude-reconstructed DENSE images. During network training, instead of using that point as an absolute ground-truth, which only provides very limited information to the network to learn and suffers from severe class imbalance, a circle with a six-pixel radius around that point can be defined as the network target. The network's inputs were the DENSE magnitude image and the segmented LV binary mask obtained by the aforementioned myocardial segmentation networks as an additional input channel. The network's output is the probability map of a circle for which the center of mass is defined to be the detected RV-LV insertion point. The same aforementioned U-Net structure can be used. The loss function was the combination of the absolute difference and the soft Dice between the target and the output probability map computed using a Sigmoid function. The same on-the-fly data augmentation can be applied during training, and optionally testing augmentation may not be used in the network.
At step 162, phase unwrapping can be performed, for example according to the method illustrated in
At step 164, the unwrapped phase image can be used to perform strain analysis, based on the relationship between the phases in the unwrapped phase image and displacement. This can include determining correlation of the unwrapped phase image to strain values for strain analysis of the subject.
Optionally, the method can include testing and/or training augmentation. As shown in
For data augmentation, segmented and phase unwrapped data obtained by applying segmentation and phase unwrapping methods, can be used. Using simple manipulations of these data, as shown in
Example Implementations and Corresponding Results
The following description includes discussion of example implementations of certain aspects of the present disclosure described above, and corresponding results. Some experimental data are presented herein for purposes of illustration and should not be construed as limiting the scope of the disclosed technology in any way or excluding any alternative or additional embodiments.
An embodiment of the present disclosure including a semantic-segmentation phase-unwrapping network was compared to path-following for low-SNR data. To validate one embodiment of the present disclosure, each new step was be compared with expert-user or ground-truth methods and the end-to-end processing of global and segmental strains were compared to previously-validated user-assisted conventional DENSE analysis methods17.
An embodiment of the present disclosure was tested using Cine DENSE image acquisition parameters including a pixel size of 1.56×1.56 mm2-2.8×2.8 mm2, FOV=200 mm2 (using outer volume suppression) to 360 mm2, slice thickness=8 mm, a temporal resolution of 17 msec (with view sharing), 2D in-plane displacement encoding using the simple three-point method30, displacement-encoding frequency=0.1 cycles/mm, ramped flip angle with final flip angle of 15°, echo time=1.26-1.9 msec, and a spiral k-space trajectory with 4-6 interleaves.
Short-axis cine DENSE MRI data from 38 heart-disease patients and 70 healthy volunteers were used for network training and testing of a non-limiting example of the present disclosure. Twenty-six datasets were acquired using 1.5T systems (Magnetom Avanto or Aera, Siemens, Erlangen, Germany) and 82 were acquired using 3T systems (Magnetom Prisma, Skyra, or Trio, Siemens, Erlangen, Germany). The types of heart disease included dilated cardiomyopathy, hypertrophic cardiomyopathy, coronary heart disease, hypertension, acute coronary syndrome and heart failure with left bundle branch block. For each subject, 1-5 short-axis slices were acquired, each with 20-59 cardiac phases. Training data included 12,415 short-axis DENSE images from 64 randomly selected subjects, and 20% of all training data were used for model validation. Forty-four datasets, including 25 healthy volunteers and 19 patients imaged at both field strengths, were selected for the test data (10,510 total 2D images, including those with displacement encoded in both the x- and y-directions).
In the experimental embodiment described herein, the final model of each network was trained using data from 64 subjects. Network training was performed on an Nvidia Titan Xp GPU with 12 GB RAM over 400 epochs using an Adam optimizer at a learning rate of 5E-4 and a mini batch size of 10. The times to train the myocardial segmentation networks (endocardium and epicardium), identifying the RV-LV insertion point network, and using the myocardial segmentation for the phase unwrapping network were 34, 48, and 30 hours, respectively. The networks were implemented using Python (version 3.5; Python Software Foundation, www.python.org) with the Tensorflow machine-learning framework (version 1.12.0)37.
To quantitatively evaluate the results of myocardial segmentation, the DICE similarity coefficient38 was computed. This metric measures the overlap between the ground-truth segmentation (A) and the CNN's segmentation (B) as follow:
DICE coefficient is normalized between 0 and 1, where “0” indicates complete dissimilarity and “1” indicates complete agreement.
In addition, to measure the maximum and average distances between the myocardial ground-truth and the CNN-generated contours, the Hausdorff distance (DH) and the mean surface distance (MDS) were computed as follows. Given two sets of points A=(a1, . . . , an) and B=(b1, . . . , bm), and an underlying distance d(a, b) which is defined as the Euclidean distance d(a, b)=∥a−b∥, DH and MDS are given by:
To assess the accuracy of identifying the RV-LV insertion point position, the Euclidean distance between the expert-selected point and the centroid of the automatically-selected region was calculated.
To evaluate the phase-unwrapping CNN, it was compared with the widely-used path-following method5 using mean squared error (MSE). The ground-truth unwrapped phase was computed using the phase-unwrapping method based on multiple phase prediction pathways and region growing36.
For images with SNR typical of routine DENSE protocols15,39 (phase SNR of approximately 22), MSE referenced to ground truth were evaluated for the proposed U-Net and the path-following method. Similar to the phase SNR of velocity-encoded phase contrast imaging40, the DENSE phase SNR was calculated as
where the mean unwrapped phase of an end-systolic region of interest (ROI) measures the DENSE phase in the region with greatest displacement (representing the signal of interest), and the standard deviation of the phase of the end-diastolic myocardium provides a measure of the standard deviation of phase at a cardiac frame where the mean phase is essentially zero. Because SNR can be lower than typical in some circumstances (such as when imaging patients with implanted devices), the two methods were also analyzed for lower SNR data generated by adding noise to our datasets. For low-SNR data, if no ground truth data is available, low-SNR data (with phase SNR=5-10) can be synthetically created from the test data by adding noise with zero mean and with standard deviation of 0.75. Adding noise to the original wrapped phase data could change the wrapping class of any image pixel. As the label of the pixel may not be the same as the corresponding pixel in the original data, for the low-SNR data the U-Net was compared with the path-following method by calculating the MSE between the unwrapped phase and the typical-SNR unwrapped ground truth.
To evaluate the full pipeline shown in
This example focused on results for circumferential strain and not for radial strain. There are fewer pixels radially across the LV wall in short-axis images than circumferentially. For this reason, methods like DENSE and tagging can be less accurate and reproducible for the estimation of radial strain compared to circumferential strain, and many clinical applications of short-axis DENSE (and tagging) find that circumferential strain is diagnostically or prognostically useful, whereas radial strain may not perform as well.
In this non-limiting example implementation, all cardiac phases were segmented, with good results, although it is also contemplated that manually drawn-contours could be used for segmentation. Further, the DL methods described herein provide a superset of the contours needed for the simplified method, and a DL-based simplified method is contemplated.
While other strain imaging methods may provide reliable and reproducible global strain values and are well-suited to automatic DL-based analysis20,28,29, cine DENSE has shown excellent reproducibility of segmental strain7. The example described herein shows excellent agreement of DL-based fully-automated segmental strain with user-assisted semi-automatically computed segmental strain. The limits of agreement for DL automatic vs. user-assisted segmental circumferential strain are better than those for DL vs. user-assisted analysis of myocardial-tagging-based global circumferential strain29. A potential explanation for the substantially better results for DENSE is that for tag analysis, DL is used to perform motion tracking, and even when trained using data from thousands of subjects, there is error in motion tracking29. In contrast, for DENSE, DL is used only for segmentation and phase unwrapping, but DL is not used for automatic motion estimation. For DENSE, during data acquisition displacement is encoded directly into the pixel phase, thus there is no need to learn motion estimation from image features. In essence, the motion estimation problem for DENSE is much simpler than for methods like tagging and feature tracking, and the demands for DL to accomplish full automation are much less.
Evaluation of the U-Nets for LV segmentation using 5,255 test images resulted in a DICE coefficient of 0.87±0.04, a Hausdorff distance of 2.7±1 pixel (equivalent to 5.94±2.2 mm), and a mean surface distance of 0.41±0.29 pixels (0.9±0.6 mm). The computation times for determining the epicardial and endocardial contours for a single DENSE image, including test augmentation, were 0.16±0.02 s, 0.15±0.01 s, respectively. The typical semi-automatic LV segmentation time for DENSE is 3-5 minutes for all cardiac phases, which corresponds to about 6 s per frame. The RV-LV insertion point was detected within 1.38±0.9 pixels compared to the manually annotated data. The computation time for detecting the RV-LV insertion point was 2.4±0.15 s for all cardiac phases. An expert reader uses approximately 20 seconds to manually define the point.
The phase-unwrapping U-Net performed well on both typical-SNR and low-SNR DENSE phase images. The MSE values for the semantic-segmentation U-Net and the standard path-following method are provided in Table 2. MSE was similar for typical-SNR data using the U-Net and conventional path following, and was lower for low-SNR data using the U-Net (p<0.05). The time for DL phase unwrapping for all cardiac phases was 3.52±0.21 s, which was similar to path following method of 3.50±0.65 s.
Fully-automated DL methods described herein were used to compute global and segmental circumferential strain for all test data and compared the results with user-assisted DENSE analysis methods17.
The performance of each individual step of an embodiment of the present disclosure was validated, including segmentation, identification of the RV-LV insertion point, and phase unwrapping, and also validated the end-to-end performance of the entire pipeline by showing excellent correlation and agreement of whole-slice and segmental strain with well-established user-assisted semi-automatic methods.
Embodiments of the present disclosure were evaluated for short-axis cine DENSE data from multiple centers and different field strengths (1.5T and 3T). However, it is contemplated that the networks may be trained using long-axis cine DENSE data to compute longitudinal strain and using data from any machine that can provide the DENSE pulse sequence. It is also contemplated that any number of readers can be used to manually contour the data, and the neural networks can be trained or retrained for use with different numbers of readers. Additionally, while the example embodiment described herein was tested using a phase unwrapping neural network trained for one cycle of phase wrap, it should be understood that the methods disclosed herein can be used to perform an arbitrary number of cycles of phase unwrapping (e.g. 2 cycles of phase unwrap). Further, the data augmentation method for phase manipulation can be particularly useful for training with more than one cycle of phase unwrap, as comparatively few real datasets have two cycles of phase wrap. Additionally, it should be understood that the network can be trained on images with respiratory motion, other types of motion, or where the image is off-center, for example by performing further training using images with these qualities. Furthermore, it should be understood that the size of dataset in the present example is intended only as a nonlimiting example and that embodiments of the present disclosure can perform phase unwrapping with an arbitrary amount of training data.
The computerized methods, systems, and products of this disclosure are set forth herein as applied to individual frames of MRI data. This disclosure, however, also encompasses using these phase unwrapping techniques in three dimensional image analyses involving multiple frames of data of higher dimensionality, such as a set of frames of image data gathered over time.
The present study trained CNNs to perform LV segmentation, phase unwrapping, and identification of the anterior RV-LV insertion point for short-axis cine DENSE images, providing for fully-automatic global and segmental DENSE strain analysis with excellent agreement with conventional user-assisted methods. DL-based automatic strain analysis for DENSE may facilitate greater clinical use of DENSE for the assessment of global and segmental strain in heart disease patients.
The area of interest “A” corresponds to a region associated with one or more physiological activities in patient “P”. The area of interest shown in the example embodiment of
It should be appreciated that any number and type of computer-based medical imaging systems or components, including various types of commercially available medical imaging systems and components, may be used to practice certain aspects of the present disclosure. Systems as described herein with respect to example embodiments are not intended to be specifically limited to magnetic resonance imaging (MRI) implementations or the particular system shown in
One or more data acquisition or data collection steps as described herein in accordance with one or more embodiments may include acquiring, collecting, receiving, or otherwise obtaining data such as imaging data corresponding to an area of interest. By way of example, data acquisition or collection may include acquiring data via a data acquisition device, receiving data from an on-site or off-site data acquisition device or from another data collection, storage, or processing device. Similarly, data acquisition or data collection devices of a system in accordance with one or more embodiments of the present disclosure may include any device configured to acquire, collect, or otherwise obtain data, or to receive data from a data acquisition device within the system, an independent data acquisition device located on-site or off-site, or another data collection, storage, or processing device.
As shown, the computer 300 includes a processing unit 302 (“CPU”), a system memory 304, and a system bus 306 that couples the memory 304 to the CPU 302. The computer 300 further includes a mass storage device 312 for storing program modules 314. The program modules 314 may be operable to perform functions associated with one or more embodiments described herein. For example, when executed, the program modules can cause one or more medical imaging devices, localized energy producing devices, and/or computers to perform functions described herein for implementing the data acquisition used in the methods of
The mass storage device 312 is connected to the CPU 302 through a mass storage controller (not shown) connected to the bus 306. The mass storage device 312 and its associated computer-storage media provide non-volatile storage for the computer 300. Although the description of computer-storage media contained herein refers to a mass storage device, such as a hard disk, it should be appreciated by those skilled in the art that computer-storage media can be any available computer storage media that can be accessed by the computer 300.
The specific configurations, choice of materials and the size and shape of various elements can be varied according to particular design specifications or constraints requiring a system or method constructed according to the principles of the disclosed technology. Such changes are intended to be embraced within the scope of the disclosed technology. The presently disclosed embodiments, therefore, are considered in all respects to be illustrative and not restrictive. The patentable scope of certain embodiments of the present disclosure is indicated by the appended claims, rather than the foregoing description.
This application claims priority to and the benefit of U.S. provisional patent application No. 62/969,342, filed on Feb. 3, 2020, and titled “System and Method for Phase Unwrapping for DENSE MRI using Deep Learning”, the disclosure of which is hereby incorporated by reference herein in its entirety.
This invention was made with government support under grant number HL147104 awarded by the National Institutes of Health. The government has certain rights in the invention.
Number | Date | Country | |
---|---|---|---|
62969342 | Feb 2020 | US |