Patent Application
20250104228
This application claims the benefit of EP 23465538.9, filed on Sep. 26, 2023, which is hereby incorporated by reference in its entirety.
Various examples of the disclosure generally relate to angiography. Various examples specifically relate to processing multiple cardiac images depicting a portion of coronary arteries within an anatomical region of interest to adjust one or more segments of a contour of a given lumen segmentation of the portion of the coronary arteries.
Extraction or segmentation of vessel lumen from cardiac angiography images, e.g., X-ray angiography images or computed tomography angiography (CTA) images, is an integral part of angiography processing pipelines for extracting determining or calculating measurements such as minimal lumen diameter, stenosis severity, etc., or more advanced ones such as virtual fractional flow reserve (FFR), where a contour of a lumen segmentation is input to a computational fluid dynamics solver or similar models.
For such measurements to be clinically relevant, they should be as accurate as possible and thereby the segmentation of coronary artery lumen also needs to be as precise as possible. Currently, segmentation of the coronary artery lumen can be constructed by either fully manual annotation, e.g., drawing a contour of the coronary artery lumen manually, or semi-automated methods, i.e., a combination of automated algorithms with human review and maybe editing.
Semi-automated methods typically automatically determine one or more proposals of the contour of the lumen segmentation for an experienced user such as a cardiologist to review. One indicator of the quality of the one or more proposals of the contour is the qualitative fit of the contour of the lumen segmentation to the image data. This can be verified by the experienced user. However, due to various uncertainties including imaging artifacts, segmentation algorithm inaccuracies, etc., a user may disagree with a fully automatic lumen segmentation proposal and thus requires tools for correction or editing.
Standard contour editing tools may comprise dragging individual points or a group of points on the contour with mouse interaction, adding or removing individual points or a group of points, and so on. However, the disagreement of the experienced user may be localized to specific regions, e.g., regions where abnormalities such as stenoses are present or image quality is low. Consequently, such complex user input for generating or manipulating the contour of the lumen segmentation may be incompatible with an interventional setting, e.g., settings of percutaneous coronary intervention (PCI). This is due to inability to use normal editing devices such as a computer mouse or time constraints that do not allow for tedious editing acts.
Further, either fully manual annotation of the contour or semi-automated determination of the contour operates on a single cardiac image, e.g., a frame of an X-ray angiogram. Further, either fully manual annotation of the contour or semi-automated determination of the contour operates on or relies on a single cardiac image, e.g., a frame of an X-ray angiogram. Even when the single frame is selected by either an experienced user or by an elaborate algorithm and taking factors like the presence and severity of foreshortening, vessel overlaps, image artifacts/quality, panning, vessel out-of-view, and/or sub-optimal contrast agent, the existing methods still cannot construct a segmentation of the coronary artery lumen precisely and reliably because they intrinsically do not consider lumen information from multiple time points of a cardiac cycle (or heartbeat, or heart cycle) in which the lumen diameter varies along the time points of the cardiac cycle.
Therefore, a need exists for advanced techniques for editing contours of coronary arteries. Specifically, a need exists for advanced techniques of finely and precisely adjusting or manipulating a contour of a given lumen segmentation of a portion of the coronary arteries. A need exists for precisely and reliably determining a contour of a given lumen segmentation of a portion of the coronary arteries for a specific patient.
This need is met by the features described and/or claimed herein.
A computer-implemented method for processing multiple cardiac images is provided. The method includes obtaining multiple cardiac images. Each of the multiple cardiac images depicts a portion of coronary arteries within an anatomical region of interest. The method further includes determining, based on the multiple cardiac images, respective magnitudes of one or more local segmentation uncertainties. Each of the one or more local segmentation uncertainties is associated with a respective segment of a contour of a given lumen segmentation of the portion of the coronary arteries. The method also includes adjusting the respective segments of the contour based on the respective magnitudes of the one or more local segmentation uncertainties.
According to various examples, the multiple cardiac images may include multiple frames of an angiogram which are acquired within a (single) cardiac cycle.
A computing device (computer) including a processor and a memory is provided. Upon loading and executing program code from the memory, the processor is configured to perform a method for processing multiple cardiac images. The method includes obtaining multiple cardiac images. Each of the multiple cardiac images depicts a portion of coronary arteries within an anatomical region of interest. The method further includes determining, based on the multiple cardiac images, respective magnitudes of one or more local segmentation uncertainties. Each of the one or more local segmentation uncertainties is associated with a respective segment of a contour of a given lumen segmentation of the portion of the coronary arteries. The method also includes adjusting the respective segments of the contour based on the respective magnitudes of the one or more local segmentation uncertainties.
An angiography device (system or scanner) including a computing device is provided. The computing device includes a processor and a memory. Upon loading and executing program code from the memory, the processor is configured to perform a method for processing multiple cardiac images. The method includes obtaining multiple cardiac images. Each of the multiple cardiac images depicts a portion of coronary arteries within an anatomical region of interest. The method further includes determining, based on the multiple cardiac images, respective magnitudes of one or more local segmentation uncertainties. Each of the one or more local segmentation uncertainties is associated with a respective segment of a contour of a given lumen segmentation of the portion of the coronary arteries. The method also includes adjusting the respective segments of the contour based on the respective magnitudes of the one or more local segmentation uncertainties.
A further computer-implemented method for processing multiple cardiac images is provided. The method includes obtaining multiple cardiac images. Each of the multiple cardiac images depicts a portion of coronary arteries within an anatomical region of interest. The method further includes determining, based on each of the multiple cardiac images, a respective set of lumen radius measurements including multiple lumen radius measurements respectively associated with multiple locations of the portion of the coronary arteries. The method also includes determining, based on the respective sets of lumen radius measurements, a maximum stenosis severity profile and a minimum stenosis severity profile associated with the portion of the coronary arteries. The method still further includes determining, based on the maximum stenosis severity profile and the minimum stenosis severity profile, respective magnitudes of one or more local segmentation uncertainties. Each of the one or more local segmentation uncertainties is associated with a respective segment of a contour of a given lumen segmentation of the portion of the coronary arteries.
A computing device including a processor and a memory is provided. Upon loading and executing program code from the memory, the processor is configured to perform the further computer-implemented method for processing multiple cardiac images.
An angiography device including a computing device is provided. The computing device includes a processor and a memory. Upon loading and executing program code from the memory, the processor is configured to perform the further computer-implemented method for processing multiple cardiac images.
It is to be understood that the features mentioned above and those yet to be explained below may be used not only in the respective combinations indicated, but also in other combinations or in isolation without departing from the scope of the invention.
Some examples of the present disclosure generally provide for a plurality of circuits or other electrical devices. All references to the circuits and other electrical devices and the functionality provided by each are not intended to be limited to encompassing only what is illustrated and described herein. While particular labels may be assigned to the various circuits or other electrical devices disclosed, such labels are not intended to limit the scope of operation for the circuits and the other electrical devices. Such circuits and other electrical devices may be combined with each other and/or separated in any manner based on the particular type of electrical implementation that is desired. It is recognized that any circuit or other electrical device disclosed herein may include any number of microcontrollers, a graphics processor unit (GPU), integrated circuits, memory devices (e.g., FLASH, random access memory (RAM), read only memory (ROM), electrically programmable read only memory (EPROM), electrically erasable programmable read only memory (EEPROM), or other suitable variants thereof), and software which co-act with one another to perform operation(s) disclosed herein. In addition, any one or more of the electrical devices may be configured to execute a program code that is embodied in a non-transitory computer readable medium programmed to perform any number of the functions as disclosed.
In the following, embodiments will be described in detail with reference to the accompanying drawings. It is to be understood that the following description of embodiments is not to be taken in a limiting sense. The scope of the invention is not intended to be limited by the embodiments described hereinafter or by the drawings, which are taken to be illustrative only.
The drawings are to be regarded as being schematic representations and elements illustrated in the drawings are not necessarily shown to scale. Rather, the various elements are represented such that their function and general purpose become apparent to a person skilled in the art. Any connection or coupling between functional blocks, devices, components, or other physical or functional units shown in the drawings or described herein may also be implemented by an indirect connection or coupling. A coupling between components may also be established over a wireless connection. Functional blocks may be implemented in hardware, firmware, software, or a combination thereof.
Hereinafter, techniques for adjusting or editing respective segments of a contour of a (given) lumen segmentation of a portion of coronary arteries are described. The (given) lumen segmentation of the portion of the coronary arteries may be determined using any one of the existing methods. Such methods may include any one disclosed or referred to in a non-patent literature—Gharleghi, Ramtin, et al. “Towards automated coronary artery segmentation: A systematic review.” Computer Methods and Programs in Biomedicine (2022): 107015, or another non-patent literature—Kerkeni, Asma, et al. “A coronary artery segmentation method based on multiscale analysis and region growing.” Computerized Medical Imaging and Graphics 48 (2016): 49-61.
According to this disclosure, techniques for processing multiple cardiac images, each of which depicts the same portion of coronary arteries within an anatomical region of interest, are disclosed. For example, each of the multiple cardiac images may depict the same portion of either the left coronary artery or the right coronary artery of the same patient.
The multiple cardiac images may be obtained either during or after an angiography exam of the portion of coronary arteries within the anatomical region of interest. The processing may take place either during or after the angiography exam of the portion of coronary arteries, i.e., either real-time processing or post-processing is possible.
The multiple cardiac images, each of which depicts a portion of coronary arteries, i.e., the same portion of coronary arteries, within an anatomical region of interest, are obtained either during or after an angiography exam. Respective magnitudes of one or more local segmentation uncertainties are determined based on the multiple cardiac images. Each of the one or more local segmentation uncertainties is associated with a respective segment of a contour of a (given) lumen segmentation of the portion of the coronary arteries. The respective segments of the contour are adjusted (edited or manipulated) based on the respective magnitudes of the one or more local segmentation uncertainties.
In general, acquisition of cardiac images is inherently noisy and has limitations related to the achievable resolution as well as artifact, motion, and aliasing errors. Consequently, determination or construction of a contour of a lumen segmentation of a portion of the coronary arteries from cardiac images may be affected by image uncertainty.
For example, the size of the same stenosis may look different from different views and thereby in different frames acquired from different acquisition angles.
The techniques disclosed herein can leverage segmentation uncertainties derived from image processing and lumen segmentation procedures. In particular, inconsistencies and variabilities in lumen radius profiles extracted separately from different frames visualizing the same target vessel at different time points can be exploited to extract quantifiable localized uncertainty measures or magnitudes, which can facilitate smart, precise, and efficient editing, adjusting, or manipulating of a proposed contour of a (given) lumen segmentation of a portion of coronary arteries.
According to various examples, the multiple cardiac images may be acquired at different time points of a cardiac cycle. For example, the multiple cardiac images may be acquired during a cardiac cycle of 1 second using a C-arm X-ray machine with a frame rate of 10. I.e., the multiple cardiac images encapsulate the lumen information of the portion of the coronary arteries from multiple time points of a single cardiac cycle, e.g., changes in respective lumen diameter values at different locations of the portion of the coronary arteries.
By processing the multiple cardiac images using techniques disclosed herein, magnitudes of one or more local segmentation uncertainties associated with respective segments of a contour of a given lumen segmentation of a portion of coronary arteries can be precisely determined, and thereby the respective segments of the contour can be finely and precisely adjusted or edited based on the respective magnitudes of the one or more local segmentation uncertainties. For example, the respective magnitudes of the one or more local segmentation uncertainties can be determined taking lumen radius variability during an entire cardiac cycle into account.
In general, the cardiac cycle is the performance of the human heart from the beginning of one heartbeat to the beginning of the next. It consists of two periods: one during which the heart muscle relaxes and refills with blood, called diastole, following a period of robust contraction and pumping of blood, called systole. After emptying, the heart relaxes and expands to receive another influx of blood returning from the lungs and other systems of the body, before again contracting to pump blood to the lungs and those systems. A normally performing heart must be fully expanded before it can efficiently pump again. Assuming a healthy heart and a typical rate of 70 to 75 beats per minute, each cardiac cycle, or heartbeat, takes about 0.8 second to complete the cycle.
Events and details of cardiac cycles can be illustrated using a Wiggers diagram. In the Wiggers diagram, the X-axis is used to plot time subdivided into the cardiac phases, while the Y-axis typically contains the following on a single grid: blood pressure, aortic pressure, ventricular pressure, atrial pressure, ventricular volume, and electrocardiogram. Accordingly, a specific cardiac cycle can be determined using electrocardiography.
According to various examples, the multiple cardiac images may be obtained directly from an angiography device (system or scanner) or from a database for storing the multiple cardiac images acquired by the angiography device, e.g., a picture archiving and communication system (PACS).
Generally, the typical maximum frame rates found on an angiography device such as a C-arm X-ray machine are 8, 15, or 30. This means that during a single cardiac cycle of 0.8 second, 6, 12, or 24 frames can be acquired by a C-arm X-ray machine.
According to various examples, the angiography device may include a C-arm X-ray machine, an intravascular ultrasound machine, an optical coherence tomography machine, or a coronary computed tomography angiography machine.
As an exemplary angiography device,
The C arm 830 may be coupled to a C arm rotation unit (motor) 820. The C arm rotation unit 820 may be any motorized means (motor) configured to rotate the C arm 830 according to an angiography angel or a fluoroscopy angle. The C arm rotation unit 820 may be attached to and controlled by a C arm control until (controller) 810. The C arm control unit 810 may be any kind of circuitry capable of controlling C arm 830. For example, the C arm control unit 810 may include a computing device (computer).
The C-arm machine 800 may further include a control panel 850 mounted onto a side surface of patient surface support 841. The control panel 850 may be used to control C arm 830 in order to guide medical specialists to pathological vessels.
The C-arm machine 800 may also include a display 860. The display 860 may be used to display information to the medical specialist, such as real-time fluoroscopy image with an overlaid vessel roadmap image including a path to one or more pathological vessels, and/or a location of the pressure wire for measuring the at least one measurement associated with the at least one hemodynamic index. Further, the display 860 may be used to display vessel segmentation data included in overlaid vessel roadmap images, including labels for various vessel segments. In some examples, display 860 may be a touch screen, which may be used to toggle the display of the vessel segmentation data on and off.
According to this disclosure, the display 860 may be used to display a contour of a given lumen segmentation of a portion of coronary arteries and provide to a user with adjusting or editing tools for adjusting or editing one or more segments of the contour. Optionally, the display 860 may visualize the respective magnitudes of the one or more local segmentation uncertainties together with the contour, and the user can adjust or edit the respective segments of the contour based on the respective magnitudes of the one or more local segmentation uncertainties, e.g., by touching and/or dragging on the display.
The C-arm machine 800 may be connectable to a database (not shown in
According to various examples, a name (e.g., any one in the column “Abbreviation” in Table 1) of a specific segment of a coronary artery or of the entire coronary artery can be determined based on an anatomical region of interest of the angiography exam. For example, such an anatomical region of interest may include the left or right branch of the coronary vessel tree or a specific vessel section. The anatomical region of interest may be determined based on user input, or on a patient's electronic medical record included in a health information system. Additionally, or optionally, the anatomical region of interest may be determined based on multiple predefined seed points.
For example, the portion of coronary arteries within the anatomical region of interest may be one or more segments of the coronary arteries of the heart.
In addition to the vessel segments listed in Table 1,
Block 3100: obtaining multiple cardiac images, each of the multiple cardiac images depicting a portion of coronary arteries within an anatomical region of interest.
According to various examples, the multiple cardiac images may include cardiac images acquired under different acquisition angles.
For example, the multiple cardiac images may be obtained directly from an angiography device, e.g., the C-arm machine 800 of
Alternatively, the multiple cardiac images may be obtained from a database for storing the multiple cardiac images acquired by the angiography device, e.g., a PACS.
According to various examples, the multiple cardiac images may include cardiac images acquired using multi-phase coronary computed tomography angiography. I.e., the multiple cardiac images may be obtained from imaging data acquired using computed tomography angiography.
Hereinafter, techniques will be explained using frames of X-ray angiograms as an example of cardiac images. I.e., a single cardiac image may correspond to a frame of an angiogram.
For example, the multiple frames f1-f10 may be acquired during a single cardiac cycle of 1 second using a C-arm X-ray machine with a frame rate of 10. I.e., the multiple frames f1-f10 encapsulate the lumen information of the portion C of the coronary arteries from multiple time points of a single cardiac cycle, e.g., changes in respective lumen diameter or radius values at different locations of the portion C of the coronary arteries.
To improve the accuracy of determination of the respective magnitudes of one or more local segmentation uncertainties associated with respective segments of a contour of a given lumen segmentation of the portion C of the coronary arteries, it is possible to automatically and precisely select or determine, among frames of an angiogram, multiple appropriate frames, e.g., the multiple frames f1-f10, where the entire portion C of the coronary arteries is visible.
According to various examples, said obtaining of the multiple cardiac images may include obtaining an angiogram acquired during an angiography exam of the anatomical region of interest and selecting, based on at least one pre-defined criterion, the multiple cardiac images among frames of the angiogram.
Such pre-defined criterion may include at least one of the following:
For example, the multiple cardiac images may be selected from frames acquired in a single cardiac cycle, e.g., selecting 5 frames from the 10 frames f1-f10 acquired in a single cardiac cycle.
Alternatively, or optionally, the multiple cardiac images may be selected from frames acquired in different cardiac cycles. For example, an acquired angiogram may include 100 frames acquired in 10 cardiac cycles, e.g., first to tenth cardiac cycles, and 10 frames, e.g., first to tenth frames, are acquired in each cardiac cycle. A first cardiac image may be selected from respective first frames acquired in respective cardiac cycles. A further or second cardiac image may be selected from respective second (or any one of the third to tenth) frames acquired in respective cardiac cycles.
As a further example, 10 cardiac images may be selected from frames acquired in different cardiac cycles. Each one of the 10 cardiac images may be selected from respective frames acquired at the same time point within respective cardiac cycles. E.g., the first cardiac image, e.g., f1, among the 10 cardiac images may be selected from all the first frames respectively acquired in the first to tenth cardiac cycles; the second cardiac image, e.g., f2, among the 10 cardiac images may be selected from all the second frames respectively acquired in the first to tenth cardiac cycles; and so on.
According to various examples, after obtaining the multiple cardiac images, the method 3000 may further include determining the anatomical region of interest based on multiple predefined seed points.
For example, proximal seed points may include the catheter tip, and distal seed points may include one or more points selected on the main vessels or on side branches with significant stenoses.
Optionally, the portion of coronary arteries may be determined based on angulation information for acquiring the multiple cardiac images.
Optionally, or additionally, the method 3000 may further include registering the multiple cardiac images.
For example, the proximal and distal seed points may be co-registered across the multiple cardiac images to ensure that they represent the same anatomical region of interest.
According to various examples, said registration or co-registration may include applying a diffeomorphic registration algorithm, e.g., as disclosed in a non-patent literature—Ashburner, John. “A fast diffeomorphic image registration algorithm.” Neuroimage 38.1 (2007): 95-113, on the vesselness maps extracted from each frame; using the obtained deformation fields to track each of the seed points from each frame onto all of the other frames; and aggregating the tracked seed points via a density-based clustering algorithm such as DB Scan (i.e., Density-based spatial clustering of applications with noise), e.g., as disclosed in a non-patent literature—Hu, Lihua, et al. “KR-DBSCAN: A density-based clustering algorithm based on reverse nearest neighbor and influence space.” Expert Systems with Applications 186 (2021): 115763.
According to various examples, the method 3000 may further include segmenting the anatomical region of interest across the multiple cardiac images, i.e., separating the anatomical region of interest from the background in each of the multiple cardiac images.
Block 3200: determining, based on the multiple cardiac images, respective magnitudes of one or more local segmentation uncertainties, each of the one or more local segmentation uncertainties being associated with a respective segment of a contour of a given lumen segmentation of the portion of the coronary arteries.
According to various examples, the respective magnitudes of the one or more local segmentation uncertainties may be determined based on variabilities in respective lumen radii of the portion of the coronary arteries, e.g., variabilities in lumen radii respectively determined based on the multiple cardiac images.
For example, the respective magnitudes of the one or more local segmentation uncertainties may be determined based on variabilities in respective lumen radii of the portion C of the coronary arteries as shown in
According to various examples, the respective magnitudes of the one or more local segmentation uncertainties may be determined using the method 4000 as shown in
Block 4100: obtaining multiple cardiac images, each of the multiple cardiac images depicting a portion of coronary arteries within an anatomical region of interest.
According to this disclosure, block 4100 may correspond to block 3100 of the method 3000 as described above and shown in
Block 4200: determining, based on each of the multiple cardiac images, a respective set of lumen radius measurements including multiple lumen radius measurements respectively associated with multiple locations of the portion of the coronary arteries.
According to various examples, the lumen radius measurements may be determined or calculated using any one of the existing techniques or algorithms related to quantitative coronary angiography. For example, non-patent literature—Garrone, Paolo, et al. “Quantitative coronary angiography in the current era: principles and applications.” Journal of interventional cardiology 22.6 (2009): 527-536.—provides a comprehensive and updated viewpoint on quantitative coronary angiography. It is also conceivable that various existing machine learning algorithms or image processing techniques can be used to determine the lumen radius measurements.
For example, at location point 1107, two lumen radii r1 and r2 may be determined or measured. The respective lumen radius associated with location point 1107 may be determined as an average of r1 and r2. It is also possible to determine or select the lumen radius associated with location point 1107 as either r1 or r2. Optionally, r1 and r2 may be determined based on different frames acquired using different acquisition angles.
According to various examples, for each of the location points 1101-1124, it is also possible to determine multiple, e.g., 3, 4, 8, or even more, lumen radii based on multiple frames acquired using different acquisition angles, and determine an average of the multiple lumen radii as the respective lumen radius for the respective set of lumen radius measurements.
Similarly, for each one of the multiple frames f1-f10 of
To facilitate the determination of the respective sets s1-s10 of the lumen radius measurements, a segmentation algorithm may be applied to the ten frames f1-f10, respectively, to separate the portion C of the coronary arteries from the background of the respective frame.
According to various examples, said determining of the respective set of lumen radius measurements may include segmenting the portion C of the coronary arteries from the respective cardiac image of the multiple cardiac images f1-f10 and determining the respective set s1-s10 of the lumen radius measurements based on the respective segmented portion of the coronary arteries.
According to various examples, after segmenting the portion C of the coronary arteries from the respective cardiac image of the multiple cardiac images f1-f10, the respective segmented portion of the coronary arteries may be co-registered along the centerline 1100 and/or based on landmarks detected in the respective frames f1-f10, e.g., bifurcations, stenoses, calcifications, and etc.
Block 4300: determining, based on the respective sets of lumen radius measurements, a maximum stenosis severity profile and a minimum stenosis severity profile associated with the portion of the coronary arteries.
According to various examples, the maximum stenosis severity profile and the minimum stenosis severity profile may be respectively determined based on local concavities and/or local convexities of multiple curves and each of the multiple curves may depict a respective relationship between a respective set of the lumen radius measurements and the multiple locations of the portion of the coronary arteries.
For example, as shown in
At a location point of a curve, if the second derivative is positive the curve is concave up (also referred to as convex) at such a location point, meaning that the tangent line at such a location point lies below the curve; similarly, if the second derivative is negative the curve is concave down (also simply called concave) at such a location point, meaning that the tangent line at such a location point lies above the curve; if the second derivative changes sign, the curve will switch from concave down to concave up, or vice versa, and a point where this occurs is called an inflection point.
According to various examples, the second derivatives may be computed via central finite differences.
In practice, due to vessel deformations caused by heart motion, the respective sets s1-s10 of the lumen radius measurements (or the radius profiles/curves p1-p10) may be not aligned. To more precisely determine the respective values of the at least one hemodynamic indices, the respective sets s1-s10 of the lumen radius measurements (or the radius profiles/curves p1-p10) may be warped before determining the maximum stenosis severity profile and the minimum stenosis severity profile.
For example, a dynamic time warping algorithm, e.g., as disclosed in a non-patent literature—Salvador, Stan, and Philip Chan. “Toward accurate dynamic time warping in linear time and space.” Intelligent Data Analysis 11.5 (2007): 561-580.—may be used for warping the respective sets s1-s10 of the lumen radius measurements. Alternatively, it is also possible to use an optimal subsequence bijection algorithm such as that disclosed in a non-patent literature—Latecki, Longin Jan, et al. “Optimal subsequence bijection.” Seventh IEEE International Conference on Data Mining (ICDM 2007). IEEE, 2007.
Optionally or additionally, said determining of the maximum stenosis severity profile and the minimum stenosis severity profile may include determining a reference set of the lumen radius measurements, aligning each of the respective sets of the lumen radius measurements with the reference set of the lumen radius measurements, and determining the maximum stenosis severity profile and the minimum stenosis severity profile based on the respective aligned sets of the lumen radius measurements.
For example, the reference set of the lumen radius measurements may be randomly or arbitrarily selected from the ten sets s1-s10 of the lumen radius measurements. For instance, s1 is selected as the reference set of the lumen radius measurements and the other sets s2-s10 can be respectively aligned with the reference set s1 using dynamic time warping algorithm, e.g., the dtw function of Matlab, i.e., executing dtw (s1, s2), dtw (s1, s3), dtw (s1, s4), . . . dtw (s1, s10).
According to various examples, said determining of the reference set of the lumen radius measurements may include selecting the set of the lumen radius measurements having the longest length as the reference set of the lumen radius measurements. Optionally or additionally, the set of the lumen radius measurements having the longest length may be determined by sorting the respective sets of the lumen radius measurements based on respective lengths of the respective sets in descending or ascending order.
For example, each of the ten sets s1-s10 of the lumen radius measurements may be indicated as a row or column vector, and the length of a respective set corresponds to the number of elements or entries of the respective vector.
According to various examples, said aligning of each of the respective sets of the lumen radius measurements with the reference set of the lumen radius measurements may include, for each of the respective unaligned sets of the lumen radius measurements, iteratively performing the following acts:
For example, after sorting the respective sets s1-s10 of the lumen radius measurements based on respective lengths of the respective sets s1-s10 in descending order, the first sorted set, i.e., the longest set, e.g., s5, may be selected as the initial reference set and then the second longest set, e.g., s2, may be aligned with the initial reference set s5 to obtain the aligned set s2′ of s2. Next, the reference set is updated based on the initial reference set s5 and the aligned set s2′ of s2. For example, the reference set may be determined or calculated based on either an element-wise average or an element-wise weighted average of s5 and s2′. Similarly, the third longest set may be aligned with the updated reference set and then the reference set may be updated again by further considering the aligned set of the third longest set. I.e., said aligning and updating may be iteratively performed until the shortest set has been aligned.
Optionally or additionally, said aligning may be based on one or more anatomical landmarks within the anatomical region of interest.
For example, the one or more anatomical landmarks may be associated with the portion of coronary arteries, e.g., bifurcations, stenoses, calcifications, and etc.
According to various examples, the maximum stenosis severity profile and the minimum stenosis severity profile may be respectively determined based on local concavities and/or local convexities of the multiple aligned curves p1′-p10′.
In general, coronary stenosis refers to an abnormally narrowed coronary artery. The stenosis is usually discrete but may sometimes be diffuse and involve a long segment of an artery. In coronaries, stenoses are usually located in the large epicardial arteries with diameters of more than 2 mm, but they may also occur in smaller arteries. In the majority of cases, a stenosis is caused by atherosclerosis, but other causes such as systemic inflammatory diseases may be involved. Coronary stenoses are usually chronic fixed changes, but sometimes a transient, dynamic stenosis due to spasm of the artery may happen.
From a pathophysiological point of view, a coronary stenosis impairs flow to the subtending myocardium, as the perfusion pressure is decreased distal to the stenosis. The flow dynamics in a vessel is governed by Poiseuille's law, which states that resistance in a vessel has an inverse relationship with the radius of the vessel to the fourth power. For example, the diameter (and therefore the radius as well) of a coronary artery may be reduced by two-thirds (for example, an artery with a diameter of 3 mm with a fixed stenosis at a discrete segment may have a residual diameter of only 1 mm in that segment). In that case, the remaining diameter (and the radius) in the diseased segment will be one-third of the initial diameter/radius, and according to Poiseuille's law, the resistance within that stenotic segment will increase by 81-fold. If the perfusion pressure in that coronary artery remained constant, then the blood flow distal to this stenosis would decrease by 81-fold.
Based on these intrinsic characteristics of coronary artery stenoses, individual stenosis can be determined by a significant reduction followed by a significant increase in the lumen radius value or profiles. An individual stenosis generally starts at a location with a (local) maximum radius and ends at a location with a further (local) maximum radius. Between the two local maximum radius locations, there is a significant reduction in the radius value, i.e., the segment of the lumen radius profile between the two local maximum radius locations looks like a convex curve.
Act 1: determining, based on the aligned curves p1′-p10′, one or more local maximum lumen radius of the portion C of the coronary arteries and respective locations of the portion C of the coronary arteries with which the one or more local maximum lumen radius are associated; and determining, based on the aligned curves p1′-p10′, respective minimum lumen radii associated with the respective locations with which the one or more local maximum lumen radius are associated.
For example, points M1, M2, M3, and M4 respectively indicate four local maximum lumen radii. The points M1, M2, M3, and M4 are respectively associated with locations 11, 12, 13, and 14 of the portion C of the coronary arteries. After determining the locations 11, 12, 13, and 14, the respective minimum lumen radii associated with the four locations 11, 12, 13, and 14 can be determined, i.e., the respective radii indicated by points A1, A2, A3, and A4.
According to this disclosure, the maximum and minimum lumen radii associated with the same location of the portion C of the coronary arteries may be determined among different frames, e.g., f1-f10, of an angiogram.
Act 2: determining stenoses based on the local maximum lumen radii.
As described above, an individual stenosis generally starts at a location with a (local) maximum radius and ends at a location with a further (local) maximum radius. As shown in
Act 3: determining, based on the aligned curves p1′-p10′, a respective local minimum lumen radius for each of the determined stenoses, e.g., S1, S2 and S3, and respective locations of the portion C of the coronary arteries with which the respective local minimum lumen radii are associated; and determining, based on the aligned curves p1′-p10′, respective maximum lumen radii associated with the respective locations with which the respective local minimum lumen radii are associated.
As shown in
According to this disclosure, the maximum and minimum lumen radii associated with the same location of the portion C of the coronary arteries may be determined among different frames, e.g., f1-f10, of an angiogram.
Act 4: for each of the determined stenoses, e.g., S1, S2, and S3, determining respective segments of both the maximum severity profile 2100 and the minimum severity profile by interpolating lumen radius values segment-wise from the start to the end of the respective stenosis.
In general, for a specific stenosis such as any one of S1, S2, and S3, the respective segment of the maximum severity profile 2100 has, compared to the respective segment of the minimum severity profile 2200, a larger lumen radius at the start and end of the stenosis, respectively, and on the other hand, has a smaller radius at the location of the minimal lumen radius. This means that the maximum and minimum severity profiles 2100 and 2200 typically intersect with each other within one stenosis.
For example, for stenosis S1, the respective segment of the maximum severity profile 2100 may start from point M1, via point N1, and end at point M2 and the respective segment of the minimum severity profile 2200 may start from point A1, via point B1, and end at point A2; for stenosis S2, the respective segment of the maximum severity profile 2100 may start from point M2, via point N2, and end at point M3 and the respective segment of the minimum severity profile 2200 may start from point A2, via point B2, and end at point A3; for stenosis S3, the respective segment of the maximum severity profile 2100 may start from point M3, via point N3, and end at point M4 and the respective segment of the minimum severity profile 2200 may start from point A3, via point B3, and end at point A4.
According to various examples, for an individual stenosis determined in act 2 explained above, multiple segments of the stenosis may be determined based on one or more local maximum lumen radii between the start and end of the stenosis. For each segment of an individual stenosis, respective segments of the maximum and minimum severity profiles 2100 and 2200 can be determined or revised more precisely by performing the acts 1-4.
Additionally, or optionally, each segment of an individual stenosis may be divided into multiple sub-segments based on one or more local maximum lumen radii between the start and end of the respective segment. For each sub-segment, the maximum and minimum severity profiles 2100 and 2200 can be determined or revised more precisely by performing the acts 1-4, too.
In summary, acts 1-4 can be applied to different segments of the portion C of the coronary arteries and thereby the maximum and minimum severity profiles 2100 and 2200 can be determined. The more segments, the more precise the two profiles 2100 and 2200. The techniques disclosed herein can ensure that different local severity values are obtained for each stenosis, while at the same time, the radius profile displays no discontinuities.
Block 4400: determining, based on the maximum stenosis severity profile and the minimum stenosis severity profile, respective magnitudes of one or more local segmentation uncertainties, each of the one or more local segmentation uncertainties being associated with a respective segment of a contour of a given lumen segmentation of the portion of the coronary arteries.
According to various examples, for each location of the portion C of the coronary arteries, e.g., any one of location points 1101-1124 of
For example, referring to
Similarly, the respective magnitudes of the respective local segmentation uncertainties associated with any other locations of the portion C of the coronary arteries, e.g., locations 12-17, can be determined as well.
For an individual segment of the portion C of the coronary arteries, e.g., 11-15, 15-12, 15-16, 12-14, and so on, the respective magnitude of the local segmentation uncertainty associated with the individual segment can be determined based on respective magnitude associated with respective locations of multiple or even all locations of the individual segment. It is also possible to use a magnitude associated with a single location of the individual segment to represent the magnitude associated with the individual segment.
For instance, multiple, e.g., 5-20, location points can be selected or sampled for segment 11-15, and the magnitude associated with the segment 11-15 can be determined by averaging respective magnitudes associated with the multiple location points. Alternatively, it is possible to determine the magnitude associated with the segment 11-15 based on either a weighted sum or a weighted average of the respective magnitudes associated with the multiple location points.
Referring back to
Block 3300: adjusting the respective segments of the contour based on the respective magnitudes of the one or more local segmentation uncertainties.
According to this disclosure, the (given) lumen segmentation of the portion of the coronary arteries may be determined using any one of the existing methods. For example, an existing machine-learning-based method may be applied to one or more of the multiple cardiac images obtained at block 3100 to determine or generate the contour of the (given) lumen segmentation of the portion of the coronary arteries.
As shown in
For example, during a PCI, a cardiologist may review the contour 1200 displayed on the screen of the display 860 of the C-arm machine 800 of
According to various examples, as shown in
For example, as shown in
The user touches or clicks a point on the centerline 1100, then drags to the left/right or orthogonal to the centerline 1100 to adjust the radius at this location and pull or push respective points of the contour 1200 corresponding to this location in the orthogonal direction to the centerline. When underlying uncertainty at this location is high, this could automatically enable larger adjustment acts or less constrained adjustment, and finer acts or more constrained adjustment for lower uncertainty regions. The editing or adjusting could be localized to contours corresponding to that particular touch or click point or extended to edit contour points corresponding to neighboring points on the centerline 1100. In the latter case, the magnitude of adjustments for neighboring points could consider the uncertainty at each point and/or weighted by a factor corresponding to the inverse distance to the original touch or click point.
According to various examples, said adjusting of the respective segments of the contour may include for each one of the one or more local segmentation uncertainties, determining whether the respective magnitude is greater than a predefined threshold and upon determining the respective magnitude is greater than the predefined threshold, adjusting the respective segment of the contour.
According to various examples, said adjusting of the respective segments of the contour may be further based on a specific use case (or subsequent processing) of the given lumen segmentation and/or an anatomical structure of the portion of the coronary arteries.
For example, specific use cases may include computation of at least one hemodynamic index such as FFR, Quantitative Coronary Analysis (QCA), and PCI planning.
According to various examples, respective magnitudes of local segmentation uncertainties may be weighted given a main scope of the lumen segmentation based on the following specific use cases or scenarios:
FFR computation (hemodynamic index computation): the FFR values are most sensitive with respect to the minimum radius values inside the coronary lesions. Other sensitive locations are the start and the end locations of the lesions. Hence, start and end stenosis markers and minimum radius locations would have a larger weight, while other locations, e.g., healthy segments, and stenosis regions with decreasing/increasing radius, would have a smaller weight.
QCA: As for FFR only the start and end locations of the stenoses and the minimum radius locations would have a larger weight. In the case of FFR, all lesions are of interest, whereas in the case of QCA, typically the focus lies on the main stenosis. Hence, the weights for a secondary or tertiary lesion would be small.
PCI planning: To correctly choose the stent size, it is important to assess the vessel diameter in the proximal and distal healthy segments (these locations would have large weights). In this case, the locations inside the stenosis, including the minimal radius location would have a small weight. Hence, as part of the user interaction workflow, the user would specify the main scope of the segmentation, and then the local uncertainty would be adapted. Additional input may be provided by a lesion detection module, to determine the start and end markers of the stenoses, the healthy segments, etc.
According to various examples, said adjusting of the respective segments of the contour may be performed using a trained machine learning algorithm.
The adjusting or editing operations described above could be designed as a machine learning based interactive editing or adjusting. A user can still provide input(s) by clicking on the point on the centerline and/or on the true lumen boundary. A machine learning algorithm can be trained which takes intensity images, initial segmentation results, and the user input, for example as a multi-channel input for a convolutional neural network, and outputs updated segmentation. The user input could be heatmaps or distance maps generated using the user click(s) and the computed uncertainty at that location (for example uncertainty could be used as the variance of the heatmap).
For example, during a PCI, a cardiologist may review the contour 1200 displayed on the screen of the display 860 of the C-arm machine 800 of
As shown in
According to various examples, the machine learning algorithm 5000 may include multiple encoders 5100, 5200, 5300, and 5400, which respectively take one or more frames of the multiple cardiac images obtained at blook 3100, an image of the contour 1200, an uncertainty map indicating respective magnitudes of local segmentation uncertainties associated with the portion C of the coronary arteries, and a heatmap indicating the specific location where the contour 1200 may need to be adjusted as inputs. The respective outputs of the multiple encoders 5100, 5200, 5300, and 5400 may be combined or concatenated as input to a single decoder 5500, and the decoder 5500 outputs an image 5600 depicting an adjusted or updated contour of an updated lumen segmentation of the portion of the coronary arteries. The image 5600 may be displayed on the screen of the display 860 of the C-arm machine 800 of
According to various examples, if the user such as a cardiologist is still not satisfied with the adjusted or updated contour depicting in the image 5600. The user can manually adjust or edit the adjusted contour. Such manual adjustment can be used to fine-tune parameters of the machine learning algorithm 5000 through a training process. In such a way, localized versions for individual clinicians or hospitals could be generated.
According to various examples, each of the multiple encoders 5100, 5200, 5300, and 5400 may be based on a convolutional neural network or a transformer.
In general, the machine learning algorithm 5000 can be trained using supervised learning, semi-supervised learning, or un-supervised learning. When using supervised learning, training examples may be represented as tuples (x, y). Here, x may include various inputs like original cardiac images, initial segmentation outcomes, uncertainty maps, and user inputs. The goal is to predict updated segmentation results such as the image 5600, which may be manually annotated as y. The training process may employ common optimizers like Adam or SGD along with predefined hyperparameters.
According to various examples, like those shown in
According to this disclosure, the multiple cardiac images may include cardiac images acquired under different acquisition angles. If the correspondence between certain landmarks on the target coronary vessel, e.g., the portion C of coronary arteries, is given in the multiple cardiac images or views via user input or an adequate multi-view vessel matching algorithm, the techniques described herein may further take into account the radius profiles from multiple views, where each view contributes one or more frames to the co-registration/registration procedure. More comprehensive co-registration methodologies may be employed to compensate for e.g., different lengths of the target vessel (due to projection geometry).
The computing device 9000 may be embedded in or connected with an angiography device such as the C-arm machine 800. I.e., the angiography device including the computing device 9000 may be configured to perform the method 3000 and/or the method 4000.
Referring to
According to this disclosure, the method 4000 can be executed independently to determine respective magnitudes of one or more local segmentation uncertainties by processing multiple cardiac images. Additionally, or optionally, the method 3000 can utilize the method 4000 as an exemplary realization of block 3200.
Summarizing, techniques have been described that facilitate efficient editing, adjusting, or manipulating of a proposed contour of a (given) lumen segmentation of a portion of coronary arteries by taking lumen information from multiple time points of a cardiac cycle into account. By processing multiple cardiac images, e.g., acquired in a cardiac cycle, magnitudes of one or more local segmentation uncertainties associated with respective segments of a contour of a given lumen segmentation of a portion of coronary arteries can be precisely determined, and thereby the respective segments of the contour can be finely and precisely adjusted or edited based on the respective magnitudes of the one or more local segmentation uncertainties. For example, the respective magnitudes of the one or more local segmentation uncertainties can be determined taking lumen radius variability during an entire cardiac cycle into account.
According to various examples, by leveraging information extracted from multiple cardiac images, the adjusting or editing process can be guided and thus accelerated. Additionally, or optionally, by leveraging uncertainty stemming from an automatic processing and segmentation pipeline, a robust smart local editing model can be built.
According to the disclosure, a simple user interface can be designed for editing a contour of a lumen segmentation. Editing can be performed via a simple, touch-friendly user interface as indicated in
According to various examples, the techniques disclosed herein can be integrated into a multi-stage editing process: e.g., first use proposed uncertainty-guided editing for fast local and global adjustment of contours, then “normal” manual editing for fine details, that were not captured by the proposed uncertainty-guided editing.
The techniques disclosed herein may further increase transparency. Instead of only using the localized uncertainty as a means to guide the editing, it could be displayed to the user, e.g., via color-coding of the contour or centerline, and thus visually indicate regions along the target vessel, where the automatic contours are more likely to require user editing because aggregated contours may have low confidence.
Although the disclosure has been shown and described with respect to certain preferred embodiments, equivalents and modifications will occur to others skilled in the art upon the reading and understanding of the specification. The present disclosure includes all such equivalents and modifications and is limited only by the scope of the appended claims.
For illustration, the disclosure is explained in detail based on uncertainty derived from variabilities across segmentation from multiple frames. However, other sources of uncertainty can be further taken into account. For example, the uncertainty could be derived from localized image quality measures or even input by the user.
According to various examples, different use cases or utilizations of the contour of the lumen segmentation, e.g., computation of FFR, QCA, or PCI planning, may be not impacted equally by different local segments or sections of the coronary artery tree. Focusing human review and editing on regions that will most affect the subsequent processing of the contour can significantly accelerate the review process.
For illustration, the disclosure is explained in detail based on X-ray angiography, the techniques disclosed herein can be also applied to angiography that may be done using scans instead of X-rays, such as CT (computed tomography) angiography or MRI (Magnetic resonance imaging) angiography.
Further, the approaches described herein may be used also when certain parts of the region of interest are not well visible on a certain frame, e.g., due to foreshortening, vessel overlap, etc.
In addition, the approaches described herein may be used also when the multiple cardiac images are obtained from different angiography views/acquired under different acquisition angles. Certain regions of interest may be recommended depending on the angulation of the acquisition (e.g. LM/pLAD/proximal LCx for the spider view). By combining the recommended regions of interest from the various angulations, a more accurate representation of the entire region of interest may be obtained.
Further, the approaches described herein may be used also when the multiple cardiac images are obtained from different imaging modalities: OCT/IVUS: highly accurate anatomical information on a single vessel/vessel segment or CCTA: entire coronary arterial tree modeled in 3D.
For example, if intravascular imaging (IVUS, OCT) is available for certain coronary segments and the data can be co-registered, the uncertainty may be reduced inside those segments. High-resolution CTA data may also be used to reduce uncertainty within vessel segments with e.g., overlap, low image quality, etc.
Wherever not already described explicitly, individual embodiments, or their individual aspects and features, described in relation to the drawings can be combined or exchanged with one another without limiting or widening the scope of the described invention, whenever such a combination or exchange is meaningful and in the sense of this invention. Advantages which are described with respect to a particular embodiment or with respect to a particular figure are, wherever applicable, also advantages of other embodiments.
| Number | Date | Country | Kind |
|---|---|---|---|
| 23465538.9 | Sep 2023 | EP | regional |
