This disclosure relates to medical instruments and procedures and more particularly to monitoring fluid flow in tissue using photo-plethysmography (PPG) information.
During cardiac bypass surgery, a surgeon uses one or more vessels from the body to bypass an atherosclerotic narrowing in coronary arteries. This is performed with the intention to increase blood flow in the coronary arteries, which supply blood to myocardial tissue of the heart. During minimally invasive (MI) cardiac bypass surgery, elongated instruments are inserted into small incisions in the chest of a patient, and an endoscope provides visualization. In robotic guided MI cardiac bypass surgery, one or more of the instruments (or endoscope) are controlled by a robotic device. Cardiac bypass surgery can be performed with the patient using cardiopulmonary bypass and the heart in cardiac arrest, or with the heart beating.
Graft patency after bypass is validated using X-ray angiography or Ultrasound intraoperatively or using volumetric imaging (computed tomography (CT), magnetic resonance imaging (MRI)) postoperatively. Both intraoperative techniques validate the flow in the graft and in coronary arteries downstream from the graft. However, a successful revascularization of arteries may not necessarily lead to a successful perfusion in the heart muscle, since the flow in smaller vessels, which are not visible in conventional imaging, may also be obstructed. Additionally, these imaging modalities are often not available to surgeons.
In accordance with an exemplary embodiment of the present invention, a system for evaluating patency is described that includes a light sensor positionable relative to a vessel to plethysmography (PPG) interpretation module is configured to receive the image signal and output pixel values in an image representing PPG information. An image generation module is coupled to the PPG interpretation module to receive the pixel values and generate a PPG map to be output to a display for analysis.
For example, the light sensor can include a camera mounted on an endoscope and/or over an open incision to permit light to reach the camera from the blood vessel. The light sensor positionable relative to the blood vessel can be positioned using a robot. The light sensor can be positionable relative to the blood vessel at a first time and positioned in a same position at a second time such that PPG signals can be compared to determine PPG changes in the blood vessel. The system can further include a robot configured to permit repeatable positioning of the light sensor at different times. It is also possible that the image generation module can generate a subtraction image of the PPG signals at different times. The subtraction image can be overlaid on an image collected by the light sensor. It is also possible that the PPG map can be overlaid on an image, such as, e.g., an X-ray image, collected by the light sensor. Further, the blood vessel can include a bypass graft and the PPG map can indicate blood flow through the graft at different times.
In accordance with another exemplary embodiment of the present invention, a system for evaluating patency is described that includes an endoscope including a camera, the endoscope being positionable inside a body relative to a blood vessel to receive light from the blood vessel and convert the light into an image signal. A photo-plethysmography (PPG) interpretation module can be configured to receive the image signal, decipher PPG information from the image signal and output pixel values in an image representing PPG information. An image generation module can be coupled to the PPG interpretation module to receive the pixel values and generate PPG maps corresponding to a plurality of times. A such that differences in the PPG maps represent differences in blood flow through the blood vessel.
For example, the blood vessel can include a bypass graft and the endoscope can be positionable in a same position before and after revascularization to permit a comparison of blood flow using the PPG maps. The system can further include a robot configured to permit repeatable positioning of the endoscope at different times. It is also possible that the image generation module can generate a subtraction image of the PPG information at different times. The subtraction image can be overlaid on an image collected by the camera. The PPG maps can be overlaid on an image, such as, e.g., an X-ray image, collected by the camera. Further, the blood vessel can include a bypass graft and the PPG maps can indicate blood flow through the graft at different times.
In accordance with yet another exemplary embodiment of the present invention, a method for evaluating patency is described that includes positioning a light sensor relative to a blood vessel to receive light from the blood vessel; converting the light into an image signal; interpreting the image signal to determine a photo-plethysmography (PPG) signal and output pixel values in an image representing PPG information; generating a PPG map to be output to a display for analysis from the pixel values; and analyzing at least one PPG map to determine patency of the blood vessel.
For example, the light sensor can include a camera mounted on an endoscope and/or over an open incision. Positioning the light sensor can include positioning the light sensor at a same position at different times such that PPG signals can be compared to determine PPG changes in the blood vessel. It is also possible that positioning the light sensor can include employing a robot to permit repeatable positioning of the light sensor at different times. The method can further include generating a subtraction image from PPG maps at different times. by the light sensor. It is also possible that the blood vessel includes a bypass graft and the PPG map indicates blood flow through the graft at different times. The method can further include accounting for motion in the blood vessel to register PPG maps. The accounting can include providing a grid overlaid on the image signal of the blood vessel, where each portion of the grid includes a tracked point to follow to determine the motion of the blood vessel, for example. It is also possible that the accounting includes providing an electro-cardiogram (ECG) signal, and triggering a PPG measurement at corresponding positions along the ECG graph to account for motion of the blood vessel.
These and other objects, features and advantages of the present disclosure will become apparent from the following detailed description of illustrative embodiments thereof, which is to be read in connection with the accompanying drawings.
This disclosure will present in detail the following description of preferred embodiments with reference to the following figures wherein:
In accordance with the present principles, system and methods are provided that determine fluid flow in a region of interest using light emitted or reflected from tissue. In one embodiment, photo-plethysmography (PPG) is employed to evaluate blood flow in tissue. PPG uses light reflectance or transmission to detect cardio-vascular pulse waves travelling through the body. PPG is based on the principle that blood absorbs light more than the surrounding tissue so variations in blood volume affect transmission or reflectance accordingly. PPG signals may be employed to detect respiration and heart rate using only a CCD camera and ambient light illumination. The systems and methods described herein may extract, e.g., green and blue color pixel intensities from regions of interest on the CCD camera based image, and then measure their variation over time. Other information may be extracted and monitored as well. Higher amplitude signals correspond to a higher reflectance and thus a smaller blood volume at the location of the imaged pixels. PPG maps can be generated by plotting a PPG signal at each pixel. The use of PPG enables the extraction of information about heart and respiration rate from a video. The intensity of the signal or pixel on a PPG map is proportional to blood volume, at different phases during the heart beat cycle.
PPG may be employed to extract vital signs from subjects in a completely non-invasive and inexpensive way, using regular CCD-based cameras. Vital signs that can be measured with this technique (PPG) include heart rate, heart rate variability, respiration rate, arterial oxygen saturation (SpO2). PPG may also provide valuable physiological information, e.g., a shape of the plethysmographic waveform and the spatial distribution (across anatomical locations) of the PPG signals (i.e., PPG imaging) can be provided. during surgery changes the modulation depth of respiration on the PPG signal.
In one embodiment, the present principles provide a measure for the validation of the success of a bypass procedure using endoscope images of the heart and PPG, in which blood flow and volume can be compared directly before and after the bypass has been performed using the PPG signals, and a visualization method for the surgeon to evaluate blood volume in the myocardial tissue and vessels. PPG map variations before and after bypass, after processing endoscope images, can show the increased blood volume indicative of a successful revascularization. The present principles may be employed to check graft patency using PPG signals, e.g., in the case of beating heart surgery, using feature tracking. A method to check graft patency using PPG signals in the case of beating heart surgery using gating with ECG signals may also be employed. In another embodiment, a method to combine PPG images and X-ray images for a comprehensive validation framework is disclosed.
It should be understood that the present invention will be described in terms of medical instruments and procedures; however, the teachings of the present invention are much broader and are applicable to any structure or system where flow measurements can be made using light collected from a surface. In some embodiments, the present principles are employed in tracking or analyzing complex biological or mechanical systems. In particular, the present principles are applicable to internal tracking procedures of biological systems, procedures in all areas of the body such as the lungs, gastro-intestinal tract, excretory organs, blood vessels, etc. The elements depicted in the FIGS. may be implemented in various combinations of hardware and software and provide functions which may be combined in a single element or multiple elements.
The functions of the various elements shown in the FIGS. can be provided through association with appropriate software. When provided by a processor, the functions can be provided by a single dedicated processor, by a single shared processor, or by a plurality of individual processors, some of which can be shared. Moreover, explicit use of the term “processor” or “controller” should not be construed to refer exclusively to hardware capable of executing software, and can implicitly include, without limitation, digital signal processor (“DSP”) hardware, read-only memory (“ROM”) for storing software, random access memory (“RAM”), non-volatile storage, etc.
Moreover, all statements herein reciting principles, aspects, and embodiments of the invention, as well as specific examples thereof, are intended to encompass both structural and functional equivalents thereof. Additionally, it is intended that such equivalents include both currently known equivalents as well as equivalents developed in the future (i.e., any elements developed that perform the same function, regardless of structure). Thus, for example, it will be appreciated by those skilled in the art that the block diagrams presented herein represent conceptual views of illustrative system components and/or circuitry embodying the principles of the invention. Similarly, it will be appreciated that any flow charts, flow diagrams and the like represent various processes which may be substantially represented in computer readable storage media and so executed by a computer or processor, whether or not such computer or processor is explicitly shown.
Furthermore, embodiments of the present invention can take the form of a computer program product accessible from a computer-usable or computer-readable storage medium providing program code for use by or in connection with a computer or any instruction execution system. For the purposes of this description, a computer-usable or computer readable storage medium can be any apparatus that may include, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution electromagnetic, infrared, or semiconductor system (or apparatus or device) or a propagation medium. Examples of a computer-readable medium include a semiconductor or solid state memory, magnetic tape, a removable computer diskette, a random access memory (RAM), a read-only memory (ROM), a rigid magnetic disk and an optical disk. Current examples of optical disks include compact disk-read only memory (CD-ROM), compact disk-read/write (CD-R/W), Blu-Ray™ and DVD.
Referring now to the drawings in which like numerals represent the same or similar elements and initially to
In one embodiment, workstation 112 includes an image generation module 148 configured to receive feedback from the interpretation module 115 to display an image of pixel data (e.g., variation of color components or intensities of color components, e.g., blue and green) collected from the images through the light capturing device 128. In one embodiment, the colored pixel data includes pixel intensity. The image generation module 148 may output, e.g., blue and green pixel data accumulated over time to generate a graph or graphs. The graph or graphs may be displayed on a display 118 to provide a reading or information on blood flow through tissue. The image generation module 148 may be employed to generate grids or other overlaid information in images. For example, the grids may be employed to track many points in the image to account for movement in the image over time (described below with respect to
In particularly useful embodiments, internal tissue 130 of a body 132 (illustratively depicted on an operating table 125) is analyzed in accordance with the present principles. In one embodiment, the internal tissue 130 includes a coronary graft during or after bypass surgery. The internal tissue 130 is accessed for photo imaging using an endoscope 102 with the light capturing device 128. An image or images 134 of the internal tissue 130 is/are collected over time and recorded from the light sensor 128 or camera. The endoscope 102 includes a light or lights 126 configured to illuminate the tissue 130. The light or lights 126 may produce light at particular wavelengths (e.g., red or infra-red, although other measurement. It should be understood that the heat or other electromagnetic signature may be employed without light or other radiation source. The PPG signals may be generated from data collected from multiple channels (e.g., more than one sensor).
Images 134 are collected and interpreted by the interpretation module 115 to determine pixel values of interest. Once the pixel values are determined, the pixel values are output to the image generation module 148 to be displayed on the display device 118. Workstation 112 includes the display 118 for viewing internal images of a subject (patient) or tissue 130. Internal images 144 may include preoperative or intra-operative images of the body 132. These images may be collected by another imaging modality or system 110. PPG maps 136 may be overlaid on images 134 and/or 144. The image generation module 148 may overlay the data on the image 134 to create an overlay, map 136 or other rendering of the collected data. Display 118 may also permit a user to interact with the workstation 112 and its components and functions, or any other element within the system 100. This is further facilitated by an interface 120 which may include a keyboard, mouse, a joystick, a haptic device, or any other peripheral or control to permit user feedback from and interaction with the workstation 112.
A comparison of maps 136 at different times provides a preferable mode of operation for determining patency. The position of the light sensing device 128 (in the endoscope 102 for internal applications) is maintained or returned to a same position at two different times (e.g., before and after a graft has been installed). In one embodiment, the same position is manually achieved. In another embodiment, a robot 124 is controlled by a robot controller 138 to return to the same before and after position. In yet another embodiment, PPG light measurements are timed by movement cycles (e.g., heartbeat, or breathing cycles). In this way, the measurements are made at the same time/position of the anatomy to account for cycle measurements, etc.
The PPG principle is employed to ensure that the bypass has been successful during the interventional procedure. Since the endoscope 102 may include a camera (e.g., light collecting device 128) during the minimally invasive procedure, the images 134 from the camera can be used to extract information about the blood flow and volume before and after the bypass has been performed. This imaging can be used adjuvant to other imaging by, e.g., imaging device 110 or as a stand-alone imaging modality even in operating rooms not equipped with X-ray devices. Imaging device 110 is illustratively depicted as a C-arm X-ray system; however, other imaging modes may be employed instead or in addition to X-rays.
Advantageously, evaluating graft patency using only endoscope images without any additional hardware may be provided. Images for graft patency evaluation are subjected to PPG algorithms/methods 117 in module 115 to output PPG maps 136. While the present example describes graft patency, the present principles are applicable to not only coronary artery bypass grafting but to any other surgical procedure where revascularization is performed or blood flow needs to be evaluated. In addition, any procedure where changes in blood volume and flow are expected as a result can be validated using the present principles, for example, tumor ablation, focused ultrasound ablation, etc.
In the example, blood flow and volume can be compared directly before and after the bypass has been performed using PPG signals. This can be employed to check graft patency using the PPG signals in the case of beating heart surgery by also employing feature tracking in the image of the heart or blood vessel. The graft patency can also be checked using the PPG signals in the case of beating heart surgery using gating with ECG signals.
Feature tracking may include comparing two or more PPG signals over time; the signals are extracted from the same anatomical position each time. A salient or distinctive motion of the tissue. One way on doing this is to divide the image into a grid and track points in each of the cells of the grid to ensure that PPG signals are extracted from the same set of anatomical points at two different time points. This can also be achieved by other tracking methods.
Imaging device 110 may include a fluoroscopy (X-ray) imaging system, a computed tomography (CT) system, a magnetic resonance imaging (MRI) system, and Ultrasonic (US) system, etc. In an open surgery, a small CCD camera (e.g., externally mounted camera) can be located above a surgical field to perform similar analysis to the internal procedures performed with, e.g., a camera on an endoscope.
Referring to
If the robot 202 is used to guide the endoscope 102, joint positions of the robot 202 when the endoscope 102 is positioned correctly over the heart 204 can be stored, to be referred to later after the bypass has been completed so that the position can be reproduced for imaging the heart 204. With the endoscope 102 in place, endoscope images can be taken over a period of time and PPG maps can then be generated for each image.
Once the bypass has been completed, the endoscope 102 is located in a same position as before the bypass. In the case of the robot guided endoscopy, the robot 202 can return to maps are generated again. The PPG signals before and after the bypass should show a strong difference, given that the vessels downstream from the bypass location, as well as the myocardial tissue should be receiving higher quantities of blood from the bypassed vessel. If this were not the case, then the signals would be very similar.
For the surgeon, visualization of the PPG maps can be presented in several ways to facilitate the evaluation. The before and after PPG maps can be shown side by side, as well with a subtraction image. These images and maps may be generated by the image generation module 148 (
Referring to
PPG signals are very sensitive to motion. If certain pixels move in the image, then the PPG signal will be very different and may not correspond to the expected values. In the case where the patient is not on a cardiopulmonary bypass machine, the beating heart may cause sufficient motion to render the PPG signal useless.
Referring to
Referring to
Referring to
Referring to
In block 708, the light is converted into an image signal. In block 710, the image signal is interpreted to determine photo-plethysmography (PPG) signals and output pixel values in an image representing PPG information. In block 712, a PPG map is generated to be output to a display for analysis from the pixel values. In block 714, a subtraction image of the PPG signals generated at different times may be provided. In block 716, the subtraction image or PPG map may be overlaid on an image collected by the light sensor (e.g., over the endoscope image) or other image. It should be noted that other operations may be performed between the PPG maps to provide a comparison.
In block 718, at least one PPG map is analyzed to determine patency of the blood vessel. In block 720, the PPG map may be overlaid on an endoscope image, an X-ray image, an ultrasound image, etc. In block 722, the blood vessel may include a bypass graft, and the PPG map may indicate blood flow through the graft at different times.
In block 724, motion of the area of interest is accounted for. Two examples follow although other methods may be employed. In block 726, a grid overlay may be generated on the image signal of the blood vessel. Each portion of the grid may include a tracked point to is provided (e.g., an electro-cardiogram (ECG) signal). In block 730, the timing signal is employed for triggering a PPG measurement at corresponding positions along the timing signal to account for motion of the blood vessel.
In interpreting the appended claims, it should be understood that:
Having described preferred embodiments for evaluation of patency using photo-plethysmography on endoscope images (which are intended to be illustrative and not limiting), it is noted that modifications and variations can be made by persons skilled in the art in light of the above teachings. It is therefore to be understood that changes may be made in the particular embodiments of the disclosure disclosed which are within the scope of the embodiments disclosed herein as outlined by the appended claims. Having thus described the details and particularity required by the patent laws, what is claimed and desired protected by Letters Patent is set forth in the appended claims.
This application is the U.S. National Phase application under 35 U.S.C. §371 of International Application No. PCT/I132013/055071, filed on Jun. 20, 2013, which claims the benefit of U.S. Provisional Patent Application No. 61/665,335, filed on Jun. 28, 2012. These applications are hereby incorporated by reference herein.
Filing Document | Filing Date | Country | Kind |
---|---|---|---|
PCT/IB2013/055071 | 6/20/2013 | WO | 00 |
Publishing Document | Publishing Date | Country | Kind |
---|---|---|---|
WO2014/001981 | 1/3/2014 | WO | A |
Number | Name | Date | Kind |
---|---|---|---|
7881777 | Docherty | Feb 2011 | B2 |
20050182295 | Soper | Aug 2005 | A1 |
20100160798 | Banet | Jun 2010 | A1 |
Number | Date | Country |
---|---|---|
19741982 | Oct 1998 | DE |
2012035492 | Mar 2012 | WO |
2012046182 | Apr 2012 | WO |
Entry |
---|
Hulsbusch: “Ein bildgestutztes, funktionelles Verfahren zur optpelektronischen Erfassung der Hautperfusion”, Dissertation Technischen Hochschule Aachen, Jan. 28, 2008, pp. 1-145. XP007913039. |
Wim Verkruysse et al, “Remote plethysmographic imaging using ambient light”, Optics Express, vol. 16, No. 26, Dec. 22, 2008, p. 21434, XP055065281, ISSN: 1094-4087, DOI: 10.1364/0E.16.021434. |
Kirk H. Shelley et al, “The Use of Joint Time Frequency Analysis to Quantify the effect of Ventilation on the Pulse Oximeter Waveforn”, Journal of Clinical Monitoring and Computing (2006) 20: 81-87, DOI: 10.1007/s10877-006-9010-7. |
Number | Date | Country | |
---|---|---|---|
20150164592 A1 | Jun 2015 | US |
Number | Date | Country | |
---|---|---|---|
61665335 | Jun 2012 | US |