1. Field of Invention
The field of the currently claimed embodiments of this invention relates to optical coherence tomography systems; and more particularly to real-time, three-dimensional optical coherence tomography systems.
2. Discussion of Related Art
Optical coherence tomography (OCT) has been viewed as an “optical analogy” of ultrasound sonogram (US) imaging since its invention in early 1990's (D. Huang, E. A. Swanson, C. P. Lin, J. S. Schuman, W. G. Stinson, W. Chang, M. R. Hee, T. Flotte, K. Gregory, C. A. Puliafito, and J. G. Fujimoto, “Optical coherence tomography,” Science, vol. 254, pp. 1178-1181, 1991). Compared to the conventional image-guided interventions (IGI) using modalities such as MRI, X-ray CT and US (T. Peters and K. Cleary, Image-Guided Interventions: Technology and Applications, Springer, 2008), OCT has much higher spatial resolution and therefore possesses great potential for applications in a wide range of microsurgeries, such as vitreo-retinal surgery, neurological surgery and otolaryngologic surgery.
As early as the late 1990's, interventional OCT for surgical guidance using time domain OCT (TD-OCT) at a slow imaging speed of hundreds of A-scans/s has been demonstrated (S. A. Boppart, B. E. Bouma, C. Pitris, G. J. Tearney, J. F. Southern, M. E. Brezinski, J. G. Fujimoto, “Intraoperative assessment of microsurgery with three-dimensional optical coherence tomography,” Radiology, vol. 208, pp. 81-86, 1998). Thanks to the technological breakthroughs in Fourier domain OCT (FD-OCT) during the last decade, ultrahigh-speed OCT is now available at >100,000 A-scan/s. For example, see the following:
For a spectrometer-based SD-OCT, an ultrahigh speed CMOS line scan camera based system has achieved up to 312,500 line/s in 2008 (Potsaid et al.); while for a swept laser type OCT, >20,000,000 line/s rate was achieved by multi-channel FD-OCT using a Fourier Domain Mode Locking (FDML) laser in 2010 (Wieser et al.).
For a clinical interventional imaging system, high-speed image acquisition, reconstruction and visualization are all essential. While the acquisition speed has satisfied the real-time multi-dimensional imaging requirement, current FD-OCT systems generally suffers from shortcomings in the last two stages:
(1) Slow image reconstruction of A-scans which is generally computational intensive due to the huge volume of numerical interpolation and fast Fourier transform (FFT) involved. Moreover, for the complex-conjugate-free full-range FD-OCT, a widely used phase-modulation approach requires a modified Hilbert transform (MHT) [10-15], which is even more time-consuming. See, for example:
(2) Slow comprehensively visualization of a 3D OCT data set using a volume rendering technique such as ray-casting (M. Levoy, “Display of Surfaces from Volume Data,” IEEE Comp. Graph. & Appl., vol. 8, pp. 29-37, 1988) adds a heavy computation load. Therefore, most high-speed OCT systems usually cannot operate in real time and typically operate in the “post-processing” mode, which limits their applications.
Current real-time video-rate OCT display is generally limited to 2D (B-scan) images. The most common way of dealing with huge amounts of volumetric data (C-scan) is to “capture and save” and then perform post-processing at a later time. The post-processing of 3D data usually includes two stages, FD-OCT signal processing and volumetric visualization, both of which are heavy-duty computing tasks due to the huge data size. Therefore, real-time signal processing and volumetric visualization has two bottlenecks for an ultra-high speed FD-OCT system that could be a useful system for clinical applications such as surgical intervention and instrument guidance, which usually requires a real-time 4D imaging capability.
To overcome the signal processing bottleneck, several solutions have recently been proposed and demonstrated: Multi-CPU parallel processing has been implemented and achieved 80,000 line/s processing rate on nonlinear-k system (G. Liu, J. Zhang, L. Yu, T. Xie, and Z. Chen, “Real-time polarization-sensitive optical coherence tomography data processing with parallel computing,” Appl. Opt. 48, 6365-6370 (2009)) and 207,000 line/s on linear-k system for 1024-OCT (J. Probst, P. Koch, and G. Huttmann, “Real-time 3D rendering of optical coherence tomography volumetric data,” Proc. SPIE 7372, 73720Q (2009)); A linear-k Fourier-domain mode-locked laser (FDML) with direct hardware frequency demodulation method enabled real-time en face image by yielding the analytic reflectance signal from one depth for each axial scan (B. R. Biedermann, W. Wieser, C. M. Eigenwillig, G. Palte, D. C. Adler, V. J. Srinivasan, J. G. Fujimoto, and R. Huber, “Real time en face Fourier-domain optical coherence tomography with direct hardware frequency demodulation,” Opt. Lett. 33, 2556-2558 (2008)). More recently, a graphics processing unit (GPU) has been utilized for processing FD-OCT data (Y. Watanabe and T. Itagaki, “Real-time display on Fourier domain optical coherence tomography system using a graphics processing unit,” J. Biomed. Opt. 14, 060506 (2009)) using linear-k spectrometer. However, the methods cited above are limited to highly-special linear-k FD-OCT systems to avoid interpolation for λ-to-k spectral re-sampling. Therefore, they are not applicable to the majority of nonlinear-k FD-OCT systems. Moreover, a linear-k spectrometer is not completely linear over the whole spectrum range (Watanabe et al.; Z. Hu and A. M. Rollins, “Fourier domain optical coherence tomography with a linear-in-wavenumber spectrometer,” Opt. Lett. 32, 3525-3527 (2007)) so re-sampling would still be required for a wide spectrum range which is essential for achieving ultra-high axial resolution.
For the volumetric visualization issue, multiple 2D slice extraction and co-registration is the simplest approach, while volume rendering offers a more comprehensive spatial view of the whole 3D data set, which is not immediately available from 2D slices. However, volume rendering such as ray-casting is usually very time-consuming for CPUs. There thus remains a need for OCT systems.
A real-time, three-dimensional optical coherence tomography system according to some embodiments of the current invention includes an optical interferometer configured to illuminate a target with light and to receive light returned from the target; an optical detection system arranged in an optical path of light from the optical interferometer after being returned from the target, the optical detection system providing output data signals; and a data processing system adapted to communicate with the optical detection system to receive the output data signals. The data processing system includes a parallel processor configured to process the output data signals to provide real-time, three-dimensional optical coherence tomography images of the target.
Further objectives and advantages will become apparent from a consideration of the description, drawings, and examples.
Some embodiments of the current invention are discussed in detail below. In describing embodiments, specific terminology is employed for the sake of clarity. However, the invention is not intended to be limited to the specific terminology so selected. A person skilled in the relevant art will recognize that other equivalent components can be employed and other methods developed without departing from the broad concepts of the current invention. All references cited anywhere in this specification, including the Background and Detailed Description sections, are incorporated by reference as if each had been individually incorporated.
The term “light” as used herein is intended to have a broad meaning that can include both visible and non-visible regions of the electromagnetic spectrum. For example, visible, near infrared, infrared and ultraviolet light are all considered as being within the broad definition of the term “light.” The term “real-time” is intended to mean that the OCT images can be provided to the user during use of the OCT system. In other words, any noticeable time delay between detection and image displaying to a user is sufficiently short for the particular application at hand. In some cases, the time delay can be so short as to be unnoticeable by a user.
Since A-scan OCT signals are acquired and processed independently, the reconstruction of an FD-OCT image is inherently ideal for parallel processing methods, such as multi-core CPU parallelization (G. Liu, J. Zhang, L. Yu, T. Xie, and Z. Chen, “Real-time polarization-sensitive optical coherence tomography data processing with parallel computing,” Appl. Opt., vol. 48, pp. 6365-6370, 2009) and FPGA hardware acceleration (T. E. Ustun, N. V. Iftimia, R. D. Ferguson, and D. X. Hammer, “Real-time processing for Fourier domain optical coherence tomography using a field programmable gate array,” Rev. Sci. Instrum., vol. 79, pp. 114301, 2008; A. E. Desjardins, B. J. Vakoc, M. J. Suter, S. H. Yun, G. J. Tearney, B. E. Bouma, “Real-time FPGA processing for high-speed optical frequency domain imaging,” IEEE Trans. Med. Imaging, vol. 28, pp. 1468-1472, 2009). Recently, cutting-edge general purpose computing on graphics processing units (GPGPU) technology has been gradually utilized for ultra-high speed FD-OCT imaging. See, for example:
Compared to FPGAs and multi-core processing methods, GPGPU acceleration is more cost-effective in terms of price/performance ratio and convenience of system integration: one or multiple GPUs can be directly integrated into the FD-OCT system in the popular form of a graphics card without requiring any optical modifications. Moreover, as with its original purpose, GPUs are also highly suitable for implementing volume rendering algorithms on reconstructed 3D data sets, which provides a convenient unified solution for both reconstruction and visualization.
Real-time rendering for a large data volume can be provided through the use of a GPU according to some embodiments of the current invention. A complete 3D data set has to be ready prior to any volumetric visualization (B. R. Biedermann, W. Wieser, C. M. Eigenwillig, G. Palte, D. C. Adler, V. J. Srinivasan, J. G. Fujimoto, and R. Huber, “Real time en face Fourier-domain optical coherence tomography with direct hardware frequency demodulation,” Opt. Lett. 33, 2556-2558 (2008)).
Some embodiments of the current invention provide GPU-based, real-time, three-dimensional signal processing and visualization on a regular FD-OCT system with nonlinear k-space. (Since time provides another dimension, this is sometimes also referred to as 4D or real-time 4D, i.e., time plus three spatial dimensions.) An ultra-high-speed, linear spline interpolation (LSI) method for λ-to-k spectral re-sampling can be implemented in a GPU architecture according to some embodiments of the current invention. The complete FD-OCT signal processing, including interpolation, fast Fourier transform (FFT) and post-FFT processing can all be implemented on a GPU according to some embodiments of the current invention. Three-dimensional data sets can be continuously acquired in real time, immediately processed and visualized by either en face slice extraction or ray-casting based volume rendering from 3D texture mapped in graphics memory. For some embodiments, no optical modifications are needed. Such embodiments can be highly cost-effective and can be easily integrated into most ultrahigh speed FD-OCT systems to overcome the 3D data processing and visualization bottlenecks. However, the general concepts of the current invention are not limited to only these particular applications.
The parallel processor 116 can be one or more graphics processing units (GPUs) according to an embodiment of the current invention. However, the broad concepts of the current invention are not limited to only embodiments that include GPUs. However, GPUs can provide advantages of cost and speed according to some embodiments of the current invention. In some embodiments, a single GPU can be used. In other embodiments, two GPUs can be used. However, the broad concepts of the current invention are not limited to the use of only one or two GPUs. Three, four or more GPUs can be used in other embodiments.
The parallel processor 116 can be installed on a computer 118, for example, but not limited to, one or more graphics cards. The computer can communicate with the detection system 108 but direct electrical or optical connections, or by wireless connections, for example. real-time, three-dimensional optical coherence tomography system 100 can also include one or more display devices, such as monitor 120, as well as any suitable input or output devices depending on the particular application.
Further additional concepts and embodiments of the current invention will be described by way of the following examples. However, the broad concepts of the current invention are not limited to these particular examples.
The lateral resolution is estimated to be 9.5 μm assuming Gaussian beam. An 8-core Dell T7500 workstation was used to obtain and display images, and a GPU (NVIDIA Quadro FX5800 graphics card) with 240 stream processors (1.3 GHz clock rate) and 4 GBytes graphics memory was used to perform OCT signal processing and 3D visualization such as en face slice extraction or volume rendering.
Start from the LSI equation:
where k[n]=2π/λ[n] is the nonlinear k-space value series and λ[n] is the calibrated wavelength values of the FD-OCT system. S[n] is the spectral intensity series corresponding to k[n]. k′[n] is the linear k-space series covering the same frequency range as k[n]. Linear spline interpolation requires a proper interval [k[i], k[i+1]] for each k′[j], that is:
k[i]<k′[j]<k[i+1]. (1.2)
Let a series E[n] present the lower ends for each element of k[n], then Eq. (1.1) can be written as:
E[n] can be easily obtained before interpolation by comparing k[n] and k′[n]. From Eq. (1.3), one would notice that S′[j] is independent of other values in the series S′[n], therefore this LSI algorithm is highly suitable for the parallel computation.
Volume rendering is a numeric simulation of the eye's physical vision process in the real world, which provides better presentation of the entire 3D image data than the 2D slice extraction (J. Kruger and R. Westermann, “Acceleration techniques for GPU-based volume rendering,” in Proceedings of the 14th IEEE Visualization Conference (VIS'03) (IEEE Computer Society, Washington, D.C., 2003), pp. 287-292; A. Kaufman and K. Mueller, “Overview of Volume Rendering,” in The Visualization Handbook, C. Johnson and C. Hansen, ed. (Academic Press, 2005); M. Levoy, “Display of Surfaces from Volume Data,” IEEE Comp. Graph. and. Appl. 8, 29-37 (1988)). Ray-casting is the simplest and most straightforward method for volume rendering, shown as
C(λ)out(uj)=C(λ)in(uj)*(1−α(xi))+C(λ)(xi)*α(xi). (1.4)
αout(uj)=αin(uj)*(1−α(xi))+α(xi). (1.5)
where C(λ)(xi) and α(xi) stands for the color and opacity values of a single voxel at the spatial position xi. C(λ)out(uj), αout(uj), C(λ)in(uj) and αin(uj) are the color and opacity values on a particular eye ray in, and out of this voxel. The eye ray corresponds to a pixel position ui on the image plane, and voxels along the ray will be taken into account for color and opacity accumulation.
The principle of ray-casting demands heavy computing duty, so in general real-time volume rendering requires the use of hardware acceleration devices such as a GPU.
To test the GPU's OCT data processing ability, we processed a series of large numbers of A-scan lines in one batch. The complete processing time is recorded in milliseconds from the interval between the data transfer-in (host memory to graphics memory) and data transfer-out (graphics memory to host memory), and the time for interpolation is also recorded. Here both 2048-pixel and 1024-pixel OCT modes were tested and the 1024-pixel mode was enabled by the CMOS camera's area-of-interest (AOI) output feature. The processing time versus one-batch line number is shown as
We then tested the actual imaging speed by performing the real-time acquisition and display of 2-D B-scan images. The target used is an infrared sensing card, as in
To demonstrate the higher acquisition speed case and evaluate the possible bus and memory contention issue, for each frame the raw data transferring-in and processing were repeated for 4 times within each frame period, while achieving the same frame rate for both OCT modes. Therefore the minimum effective processing speeds of 512,000 A-scan/s for 1024-OCT and 280,000 A-scan/s for 2048-OCT can be expected. These speeds represent more than double the currently highest acquisition speed using a CMOS camera, which is 215,000 A-scan/s for 1024-OCT (J. Probst, P. Koch, and G. Huttmann, “Real-time 3D rendering of optical coherence tomography volumetric data,” Proc. SPIE 7372, 73720Q (2009)) and 135,000 A-scan/s for 2048-OCT (I. Grulkowski, M. Gora, M. Szkulmowski, I. Gorczynska, D. Szlag, S. Marcos, A. Kowalczyk, and M. Wojtkowski, “Anterior segment imaging with Spectral OCT system using a high-speed CMOS camera,” Opt. Express 17, 4842-4858 (2009), http://www.opticsinfobase.org/oe/abstract.cfm?uri=oe-17-6-4842).
We further tested the real-time volumetric data processing and en face image reconstruction by running the OCT at 1024-pixel mode. The line scan rate was set to 100,000 line/second for the convenience of the synchronization. A Naval orange juice sac was used as the sample. Three different volume sizes are tested: 250×160×512 voxels (40,000 A-scans/volume); 250×80×512 voxels (20,000 A-scans/volume); 125×80×512 voxels (10,000 A-scans/volume); corresponding to a volume rate of 2.5, 5 and 10 volume/second, respectively.
Here it is necessary to compare an en face FD-OCT imaging with another en face OCT imaging technology—time-domain transverse-scanning OCT/OCM (TD-TS-OCT/OCM) which acquires only one resolution element per A-scan. A typical TD-TS-OCT/OCM system can achieve a large en face image size (250,000 pixels) at 4 frame/s (A. D. Aguirre, P. Hsiung, T. H. Ko, I. Hartl, and J. G. Fujimoto, “High-resolution optical coherence microscopy for high-speed, in vivo cellular imaging,” Opt. Lett. 28, 2064-2066 (2003)), giving 1,000,000 transverse points per second. In contrast, en face FD-OCT has less transverse scan rate (typically <500,000 A-scan/s) because a whole spectrum has to be acquired for each A-scan. However, en face FD-OCT provides a complete 3D data set so multiple en face images at different depth of the volume can be extracted simultaneously, which is not available by TD-TS-OCT/OCM.
Next we implemented the real-time volume rendering of continuous acquired data volume and realized the 10 volume per second 4D FD-OCT “live” image. The acquisition line rate is set to be 125,000 line/s at 1024-OCT mode. The acquisition volume size is set to be 12,500 A-scans, providing 125(X)×100(Y)×512(Z) voxels after the signal processing stage, which takes less than 10 ms and leaves more than 90 ms for each volume interval at the volume rate of 10 volume/s. As noticed from
First we tested the real-time visualization ability by imaging non-biological samples. Here the half volume rendering is applied and the real volume size is approximately 4 mm×4 mm×0.66 mm. The dynamic scenarios are captured by free screen-recording software (BB FlashBack Express).
We next implemented the in vivo real-time 3D imaging of a human fingertip.
Finally, to make full use of the ultrahigh processing speed and the whole 3D data, we implemented multiple 2D frames real-time rendering from the same 3D data set with different modelview matrix, including side-view (
The processing bandwidth in the example above is much higher than most of the current FD-OCT system's acquisition speed, which indicates a huge potential for improving the image quality and volume speed of real-time 3D FD-OCT by increasing the acquisition bandwidth. The GPU processing speed can be increased even higher by implementing a multiple-GPU architecture using more than one GPU in parallel. Therefore the bottleneck for 3D FD-OCT imaging would now lie in the acquisition speed.
For all the experiments described above, the only additional device required to implement the real-time high speed OCT data processing and display for most cases is a high-end graphics card which cost far less compared to most optical setups and acquisition devices. The graphics card is plug-and-play computer hardware without the need for any optical modifications. And it is much simpler than adding a prism to build a linear-k spectrometer or developing a linear-k swept laser. Both are complicated to build and will change the overall physical behavior of the OCT system.
In conclusion, we realized GPU based real-time 4D signal processing and visualization on a regular FD-OCT system with nonlinear k-space for the first time to the best of our knowledge. An ultra-high speed linear spline interpolation (LSI) method for interpolation for λ-to-k spectral re-sampling is implemented in GPU architecture. The complete FD-OCT signal processing including interpolation for λ-to-k spectral re-sampling, fast Fourier transform (FFT) and post-FFT processing have all been implemented on a GPU. 3D Data sets are continuously acquired in real time, immediately processed and visualized by either en face slice extraction or ray-casting based volume rendering from 3D texture mapped in graphics memory. For standard FD-OCT systems, a GPU is the only additional hardware needed to realize this improvement and no optical modification is needed. This technique is highly cost-effective and can be easily integrated into most ultrahigh speed FD-OCT systems to overcome the 3D data processing and visualization bottlenecks.
As mentioned above, for most conventional FD-OCT systems, the raw data is acquired in real-time and saved for post-processing. For microsurgeries, such imaging protocol provides valuable “pre-operative/post-operative” images, but is incapable of providing real-time, “inter-operative” imaging for surgical guidance and visualization. In addition, standard FD-OCT systems suffer from spatially reversed complex-conjugate ghost images that could severely misguide the users. As a solution, the complex full-range FD-OCT (C-FD-OCT) has been utilized, which removes the complex-conjugate image by applying a phase modulation on interferogram frames. See, for example:
A 140 k line/s 2048-pixel C-FD-OCT has been implemented for volumetric anterior chamber imaging (Subhash et al). However, the complex-conjugate processing is even more time-consuming and presents an extra burden when providing real-time images during surgical procedures.
Several methods have been implemented to improve data processing and visualization of FD-OCT images: Field-programmable gate array (FPGA) has been applied to both spectrometer and swept source-based systems (T. E. Ustun, N. V. Iftimia, R. D. Ferguson, and D. X. Hammer, “Real-time processing for Fourier domain optical coherence tomography using a field programmable gate array,” Rev. Sci. Instrum. 79, 114301 (2008); A. E. Desjardins, B. J. Vakoc, M. J. Suter, S. H. Yun, G. J. Tearney, B. E. Bouma, “Real-time FPGA processing for high-speed optical frequency domain imaging,” IEEE Trans. Med. Imaging 28, 1468-1472 (2009)); multi-core CPU parallel processing has been implemented and achieved 80,000 line/s processing rate on nonlinear-k polarization-sensitive OCT system and 207,000 line/s on linear-k systems, both with 1024-point/A-scan (G. Liu, J. Zhang, L. Yu, T. Xie, and Z. Chen, “Real-time polarization-sensitive optical coherence tomography data processing with parallel computing,” Appl. Opt. 48, 6365-6370 (2009); J. Probst, D. Hillmann, E. Lankenau, C. Winter, S. Oelckers, P. Koch, G. Hüttmann, “Optical coherence tomography with online visualization of more than seven rendered volumes per second,” J. Biomed. Opt. 15, 026014 (2010)). Moreover, recent progress in general-purpose computing on graphics processing units (GPGPU) makes it possible to implement heavy-duty OCT data processing and visualization on a variety of low-cost, many-core graphics cards.
For standard half-range FD-OCT, GPU-based data processing has been implemented on both linear-k and non-linear-k systems (Y. Watanabe and T. Itagaki, “Real-time display on Fourier domain optical coherence tomography system using a graphics processing unit,” J. Biomed. Opt. 14, 060506 (2009); K. Zhang and J. U. Kang, “Real-time 4D signal processing and visualization using graphics processing unit on a regular nonlinear-k Fourier-domain OCT system,” Opt. Express 18, 11772-11784 (2010), http://www.opticsinfobase.org/abstract.cfm?uri=oe-18-11-11772; S. V. Jeught, A. Bradu, and A. G. Podoleanu, “Real-time resampling in Fourier domain optical coherence tomography using a graphics processing unit,” J. Biomed. Opt. 15, 030511 (2010)). Real-time 4D OCT imaging has also been achieved up to 10 volume/s through GPU-based volume rendering (Probst et al.; Zhang et al., id.). We have found that, based on a 1024-pixel standard FD-OCT system using NVIDIA's GTX 480 GPU, the maximum processing line rate in this example is >3,000,000 line/s (effectively >1,000,000 line/s under data transfer limit), which will be shown in detail below.
For complex full-range FD-OCT, the processing workload is more than 3 times the standard OCT, since each A-scan requires three fast Fourier transforms (FFT) in different dimensions of the frame, a band-pass filtering, and performing a necessary matrix transpose (Y. Yasuno, S. Makita, T. Endo, G. Aoki, M. Itoh, and T. Yatagai, “Simultaneous B-M-mode scanning method for real-time full-range Fourier domain optical coherence tomography,” Appl. Opt. 45, 1861-1865 (2006)). In a separate work, we realized a real-time processing of 1024-pixel C-FD-OCT at >500,000 line/s (effectively >300,000 line/s under data transfer limit), and a real-time camera-limited display speed of 244,000 line/s (J. U. Kang and K. Zhang, “Real-time complex optical coherence tomography using graphics processing unit for surgical intervention,” to appear on IEEE Photonics Society Annual 2010, Denver, Colo., USA, November, 2010). Very recently, a 27,900 line/s 2048-pixel C-FD-OCT system was reported by Watanabe et al during the preparation of this manuscript (Y. Watanabe, S. Maeno, K. Aoshima, H. Hasegawa, and H. Koseki, “Real-time processing for full-range Fourier-domain optical-coherence tomography with zero-filling interpolation using multiple graphic processing units,” Appl. Opt. 49, 4756-4762 (2010)).
In most FD-OCT systems, the signal is sampled nonlinearly in k-space, which will seriously degrade the image quality if the FFT is directly applied to such signal. So far there have been both hardware and software solutions to the nonlinear-k issue. Hardware solutions such as linear-k spectrometer (Z. Hu and A. M. Rollins, “Fourier domain optical coherence tomography with a linear-in-wavenumber spectrometer,” Opt. Lett. 32, 3525-3527 (2007)), linear-k swept laser (C. M. Eigenwillig, B. R. Biedermann, G. Palte, and R. Huber, “K-space linear Fourier domain mode locked laser and applications for optical coherence tomography,” Opt. Express 16, 8916-8937 (2008), http://www.opticsinfobase.org/oe/abstract.cfm?URI=oe-16-12-8916) and k-triggering (D. C. Adler, Y. Chen, R. Huber, J. Schmitt, J. Connolly, and J. G. Fujimoto, “Three-dimensional endomicroscopy using optical coherence tomography,” Nat. Photonics 1,709-716 (2007)) have been successfully implemented, but these methods generally increase the system complexity and cost. Software solutions include various interpolation methods such as simple linear interpolation, oversampled linear interpolation, zero-filling linear interpolation, and cubic spline interpolation. Different GPU-based interpolation methods have also been implemented and compared (Y. Watanabe and T. Itagaki, “Real-time display on Fourier domain optical coherence tomography system using a graphics processing unit,” J. Biomed. Opt. 14, 060506 (2009); K. Zhang and J. U. Kang, “Real-time 4D signal processing and visualization using graphics processing unit on a regular nonlinear-k Fourier-domain OCT system,” Opt. Express 18, 11772-11784 (2010), http://www.opticsinfobase.org/abstract.cfm?uri=oe-18-11-11772; S. V. Jeught, A. Bradu, and A. G. Podoleanu, “Real-time resampling in Fourier domain optical coherence tomography using a graphics processing unit,” J. Biomed. Opt. 15, 030511 (2010)). Alternatively, the non-uniform discrete Fourier transform (NUDFT) has been proposed recently for both swept source OCT (S. S. Sherif, C. Flueraru, Y. Mao, and S. Change, “Swept Source Optical Coherence Tomography with Nonuniform Frequency Domain Sampling,” in Biomedical Optics, OSA Technical Digest (CD) (Optical Society of America, 2008), paper BMD86) and spectrometer-based OCT (K. Wang, Z. Ding, T. Wu, C. Wang, J. Meng, M. Chen, and L. Xu, “Development of a non-uniform discrete Fourier transform based high speed spectral domain optical coherence tomography system,” Opt. Express 17, 12121-12131 (2009),
http://www.opticsinfobase.org/oe/abstract.cfm?URI=oe-17-14-12121) through direct Vandermonde matrix multiplication with the spectrum vector. Compared with the interpolation-FFT (InFFT) method, NUDFT is simpler to implement and immune to the interpolation-caused errors such as increased background noise and side-lobes, especially at larger image depth (Sébastien Vergnole, Daniel Lévesque, and Guy Lamouche, “Experimental validation of an optimized signal processing method to handle non-linearity in swept-source optical coherence tomography,” Opt. Express 18, 10446-10461 (2010), http://wwvv.opticsinfobase.org/abstract.cfm?uri=oe-18-10-10446). Moreover, NUDFT has improved sensitivity roll-off than the InFFT (Wang et al., id.). However, NUDFT by direct matrix multiplication is extremely time-consuming, with a complexity of O(N2), where N is the raw data size of an A-scan. As an approximation to NUDFT, the gridding-based non-uniform fast Fourier transform (NUFFT) has been tried to process simulated (D. Hillmann, G. Huttmann, and P. Koch, “Using nonequispaced fast Fourier transformation to process optical coherence tomography signals,” Proc. SPIE 7372, 73720R (2009)) and experimentally acquired data (Vergnole et al) with reduced calculation complexity of ˜O(NlogN). To the best of our knowledge, NUDFT/NUFFT have yet to be utilized in ultra-high speed, real-time FD-OCT systems due to computational complexity and associated latency in data processing.
In this example according to an embodiment of the current invention, we implemented the fast Gaussian gridding (FGG)-based NUFFT on the GPU architecture for ultrafast signal processing in a general FD-OCT system. The Vandermonde matrix-based NUDFT as well as the linear/cubic InFFT methods are also implemented on GPU as comparisons of image quality and processing speed. GPU-NUFFT provides a very close approximation to GPU-NUDFT in terms of image quality while offering >10 times higher processing speed. Compared with the GPU-InFFT methods, we have also observed improved sensitivity roll-off, a higher local signal-to-noise ratio, and absence of side-lobe artifacts in GPU-NUFFT. Using a high speed CMOS line-scan camera, we demonstrated the real-time processing and display of GPU-NUFFT-based C-FD-OCT at a camera-limited speed of 122 k line/s (1024 pixel/A-scan).
The FD-OCT system used in this work is spectrometer-based, as shown in
In this section, the implementation of both GPU-NUDFT and GPU-NUFFT in a standard FD-OCT system is described. For the implementation, the wavenumber-pixel relation of the system k[i]=2π/λ[i] is pre-calibrated accurately, where refers to the pixel index.
After pre-calibrating the k[i] relation, the depth information A[zm] can be implemented through discrete Fourier transform over non-uniformly distributed data I[ki], as in (S. S. Sherif, C. Flueraru, Y. Mao, and S. Change, “Swept Source Optical Coherence Tomography with Nonuniform Frequency Domain Sampling,” in Biomedical Optics, OSA Technical Digest (CD) (Optical Society of America, 2008), paper BMD86.
K. Wang, Z. Ding, T. Wu, C. Wang, J. Meng, M. Chen, and L. Xu, “Development of a non-uniform discrete Fourier transform based high speed spectral domain optical coherence tomography system,” Opt. Express 17, 12121-12131 (2009), http://wwvv.opticsinfobase.org/oe/abstract.cfm?URI=oe-17-14-12121; Sébastien Vergnole, Daniel Lévesque, and Guy Lamouche, “Experimental validation of an optimized signal processing method to handle non-linearity in swept-source optical coherence tomography,” Opt. Express 18, 10446-10461 (2010), http://www.opticsinfobase.org/abstract.cfm?uri=oe-18-10-10446),
where N is the total pixel number of a spectrum, zm refers to the depth coordinate with the pixel index m. Δk=kN-1−k0 is the wavenumber range.
For standard FD-OCT, where I[ki] are real-values, Eq. (2.1) can be reduce to half-range as,
For C-FD-OCT, where I[kj] are complex-values after Hilbert transform (Y. Yasuno, S. Makita, T. Endo, G. Aoki, M. Itoh, and T. Yatagai, “Simultaneous B-M-mode scanning method for real-time full-range Fourier domain optical coherence tomography,” Appl. Opt. 45, 1861-1865 (2006)), Eq. (1) can be modified to full-range as,
here the index m is shifted by N/2 to set the DC component to the center of A[zm]. Considering a frame with M A-scans, Equations (2.2) and (2.3) can be written in matrix form for processing the whole frame as,
where the subscript of A[zm] and I[ki] denotes the index of A-scan within one frame, the complex factor pi=exp[j2π/Δk*(ki−k0)], Dhalf and Dfull are the Vandermonde matrix, which can be pre-calculated from k[i].
To realize C-FD-OCT mode, a phase modulation φ(x)=βx is applied to each B-scan's 2D interferogram frame I(k,x) by slightly displacing the probe beam off the galvanometer's pivoting point, as shown in
where Es (k,x) and Er (k) are the electrical fields from the sample and reference arms, respectively. Γu{ } is the correlation operator. The first three terms on the right hand of Eq. (2.12) present the DC noise, autocorrelation noise, and complex-conjugate noise, respectively. The last term can be filtered out by a proper band-pass filter in the u domain and then convert back to x domain by applying an inverse Fourier transform along x direction. Here, to implement the standard Hilbert transform, we use the Heaviside step function as the band-pass filter and the more delicate filters such as super Gaussian filter can also be designed to optimize the performance (Y. Watanabe, S. Maeno, K. Aoshima, H. Hasegawa, and H. Koseki, “Real-time processing for full-range Fourier-domain optical-coherence tomography with zero-filling interpolation using multiple graphic processing units,” Appl. Opt. 49, 4756-4762 (2010)). Finally, the OCT image is obtained by NUDFT in k domain and logarithmically scaled for display.
The GPU-CPU hybrid processing flowchart for standard/complex FD-OCT using GPU-NUDFT is shown in
The direct GPU-NUDFT presented above has a computation complexity of O(N2), which greatly limits the computation speed and scalability for real-time display even on a GPU, as is shown experimentally below. Alternative to direct GPU-NUDFT, here we implemented fast Gaussian gridding-based GPU-NUFFT to approximate GPU-NUDFT: the raw signal I[ki] is first oversampled by convolution with a Gaussian interpolation kernel on a uniform grid, as [40],
where kτ[u] is uniform grid covering the same range as k[i], gτ[k] is the Gaussian interpolation kernel, Mr=R*N is the uniform gridding size, and R is the oversampling ratio. On each kτ[u] grid, the source data I[i] is selected such that kτ[u] is within the nearest 2*Msp grids to k[i], where Msp is the kernel spread factor. The calculation of Eq. (2.13) is illustrated in
Here it is worth noting that the Kaiser-Bessel function is found to be the optimal convolution kernel for the gridding-based NUFFT shown in recent works (Sébastien Vergnole, Daniel Lévesque, and Guy Lamouche, “Experimental validation of an optimized signal processing method to handle non-linearity in swept-source optical coherence tomography,” Opt. Express 18, 10446-10461 (2010), http://www.opticsinfobase.org/abstract.cfm?uri=oe-18-10-10446; D. Hillmann, G. Huttmann, and P. Koch, “Using nonequispaced fast Fourier transformation to process optical coherence tomography signals,” Proc. SPIE 7372, 73720R (2009)). The implementation of Kaiser-Bessel convolution on GPU is similar to the Gaussian kernel.
GPU Processing Line Rate for Different FD-OCT Methods
First we performed benchmark line rate test of different FD-OCT processing methods as follows:
All algorithms are tested on the GTX 480 GPU with 4096 lines of both 1024-pixel spectrum and 2048-pixel spectrum. Here the 2048-pixel mode is tested as reference only and we will use the 1024-pixel mode for the real-time imaging tests in this work. For each case, both the peak internal processing line rate and the reduced line rate considering the data transfer bandwidth of PCIE x16 interface are listed in
As in
For C-FD-OCT, the Hilbert transform, which is implemented by two Fast Fourier transforms, has the computational complexity of ˜O(M*logM), where M is the number of lines within one frame. Therefore, the processing line rate of C-FD-OCT is also influenced by the frame size M. To verify this, we tested the relation between processing line rate of NUFFT-C mode versus frame size, as shown in
The zero-filling interpolation with FFT is also effective in suppressing the side-lobe effect and background noise for FD-OCT. However, the zero-filling usually requires an oversampling factor of 4 or 8, and two additional FFT, which considerably increase the array size and processing time of the data (C. Dorrer, N. Belabas, J. Likforman, and M. Joffre, “Spectral resolution and sampling issues in Fourier-transform spectral interferometry,” J. Opt. Soc. Am. B 17, 1795-1802 (2000)). From
We compared the point spread function (PSF) and sensitivity roll-off of different FD-OCT processing methods, as shown in
We compared the real-time image quality of GPU-accelerated C-FD-OCT using different processing methods. Here a multi-layered polymer phantom is used as a sample. In the scanning protocol, each frame consists of 4296 A-scans in acquisition, but the first 200 lines are disposed before processing, since they are within the fly-back period of the galvanometer. Therefore each frame-size is 4096 pixel (lateral)×1024 pixel (axial).
First, a single frame is captured and GPU-processed using different methods, shown in
Then we screen-captured the real-time displayed scenarios using different GPU-accelerated C-FD-OCT, shown in Media 1 to 4. The image frames are rescaled to 1024 pixel (lateral)×512 pixel (axial) to accommodate the monitor display. LIFFT/CIFFT/NUFFT modes are running at 29.8 fps, corresponding to a camera-limited line rate of 122 k line/s, while the NUDFT mode is GPU-limited to 9.3 fps (38 k line/s). As shown in both
From these results, it is clear that the GPU-NUFFT method is a very close approximation of GPU-NUDFT while offering much higher processing speed. GPU-NUFFT can be achieved at a comparable processing speed to GPU-CIFFT and is immune to interpolation error-caused ghost images.
Finally, we conducted the in vivo human finger imaging using GPU-NUFFT-based C-FD-OCT, displayed at 29.8 fps with original frame size of 4096 pixel (lateral)×1024 pixel (axial).
In this example, we implemented and successfully demonstrated the FGG-based NUFFT on the GPU architecture for online signal processing in a general FD-OCT system. The Vandermonde matrix-based NUDFT as well as the linear/cubic InFFT methods were also implemented on GPU as comparisons of image quality and processing speed. GPU-NUFFT provides an accurate approximation to GPU-NUDFT in terms of image quality while offering >10 times higher processing speed. Compared to the GPU-InFFT methods, GPU-NUFFT has better sensitivity roll-off, a higher local signal-to-noise ratio, and is immune to the side-lobe artifacts caused by interpolation error. Using a high speed CMOS line-scan camera, we demonstrated the real-time processing and display of GPU-NUFFT-based C-FD-OCT at a camera-limited speed of 122 k line/s (1024 pixel/A-scan). The GPU processing speed can be increased even higher by implementing a multiple-GPU architecture using more than one GPU in parallel.
High speed Fourier domain OCT (FD-OCT) has been proposed as a new method of microsurgical intervention. However, conventional FD-OCT systems suffer from spatially reversed complex-conjugate ghost images that could severely misguide surgeons. As a solution, complex OCT has been proposed which removes the complex-conjugate image by applying a phase modulation on interferogram frames (Y. Yasuno, S. Makita, T. Endo, G. Aoki, M. Itoh, and T. Yatagai, “Simultaneous B-M-mode scanning method for real-time full-range Fourier domain optical coherence tomography,” Appl. Opt., 45, 1861-1865 (2006)). Due to its complexity, the signal processing of complex OCT takes several times longer than the standard OCT. Thus, complex OCT images are usually “captured and saved” and post-processed, which is not ideal for microsurgical intervention applications. In this example, we implemented an ultra-high-speed, complex OCT system for surgical intervention applications using a graphics processing unit (GPU) that performs real-time signal processing and display up to an effective line speed of 244,000 A-scan/s.
The system is schematically shown as
A phase modulation φ(x)=βx is applied to each B-scan's 2D interferogram frame s(k,x) by slightly displacing the probe beam off the galvanometer's pivoting point (B. Baumann, M. Pircher, E. Götzinger and C. K. Hitzenberger, “Full range complex spectral domain optical coherence tomography without additional phase shifters,” Opt. Express, 15, 13375-13387 (2007)). The x indicates A-scan index in each B-scan and k presents the wavenumber index in each A-scan. By applying Fourier transform along x direction, the following equation can be obtained (Y. Yasuno, S. Makita, T. Endo, G. Aoki, M. Itoh, and T. Yatagai, “Simultaneous B-M-mode scanning method for real-time full-range Fourier domain optical coherence tomography,” Appl. Opt., 45, 1861-1865 (2006)):
F
x→u
[S(k,x)]=|Er(k)|2δ(u)+Γu{Fx→u[Es(k,x)]}
+Fx→u[E*s(k,x)Er(k)]δ(u+β)+Fx→u[Es(k,x)E*r(k)]
δ(u−β) (3.1)
where Er(k,x) and Er(k) are the electrical fields from the sample and reference arms, respectively. Γu{ } is the correlation operator. The first three terms on the right hand of Eq. (3.1) present the DC noise, autocorrelation noise, and complex-conjugate noise, respectively. The last term can be filtered out by a proper band pass filter in the u domain and then back to x domain by applying an inverse Fourier transform along the x direction. Finally, the OCT image is obtained by Fourier transform in the k domain and logarithmically scaled for display as in standard FD-OCT.
The above signal processing was fully implemented using NVIDIA's CUDA language on a GPU (NVIDIA GeForce GTX 480) with 480 stream processors (1.4 GHz clock rate) and 1.5 Gigabyte graphics memory (K. Zhang and J. U. Kang, “Real-time 4D signal processing and visualization using graphics processing unit on a regular nonlinear-k Fourier-domain OCT system,” Opt. Express, 18, 11772-11784 (2010)). The developed software contains three synchronized threads for raw data acquisition, GPU processing, and display respectively. For each B-scan, 8592 lines are acquired as one frame but the first 400 lines are automatically disposed, since it contains the fly-back period of the galvanometer. The remaining 8192 lines are transferred-in and processed by the GPU at a frame rate of 29.8 Hz, which corresponds to an effective line speed of 244,000 A-scan/s. The total time for GPU processing, including data transfer-in and transfer-out, was measured to be about 25 ms, which calculates to be a processing speed of 328,000 A-scan/s for 1024 pixel complex OCT.
The real-time displayed images are directly captured from the screen and shown in
In this example, we implemented an ultra-high-speed, complex OCT system using a graphics processing unit and achieved real-time signal processing display up to an effective line speed of 244,000 A-scan/s. Some embodiments of this ultra-high-speed, real-time complex OCT system can be implemented for microsurgical guidance and intervention applications.
Microsurgeries require both physical and optical access to limited space in order to perform tasks on delicate tissue. The ability to view critical parts of the surgical region and work within micron proximity to the fragile tissue surface requires excellent visibility and precise instrument manipulation. The surgeon needs to function within the limits of human sensory and motion capability to visualize targets, steadily guide microsurgical tools and execute all surgical targets. These directed surgical maneuvers must occur intraoperatively with minimization of surgical risk and expeditious resolution of complications. Conventionally, visualization during the operation is realized by surgical microscopes, which limits the surgeon's field of view (FOV) to the en face scope (K. Zhang, W. Wang, J. Han and J. U. Kang, “A surface topology and motion compensation system for microsurgery guidance and intervention based on common-path optical coherence tomography,” IEEE Trans. Biomed. Eng. 56, 2318-2321 (2009)), with limited depth perception of micro-structures and tissue planes.
As a noninvasive imaging modality, optical coherence tomography (OCT) is capable of cross-sectional micrometer-resolution images and a complete 3D data set could be obtained by 2D scanning of the targeted region. Compared to other modalities used in image-guided surgical intervention such as MRI, CT, and ultrasound, OCT is highly suitable for applications in microsurgical guidance (K. Zhang, W. Wang, J. Han and J. U. Kang, “A surface topology and motion compensation system for microsurgery guidance and intervention based on common-path optical coherence tomography,” IEEE Trans. Biomed. Eng. 56, 2318-2321 (2009); Y. K. Tao, J. P. Ehlers, C. A. Toth, and J. A. Izatt, “Intraoperative spectral domain optical coherence tomography for vitreoretinal surgery,” Opt. Lett. 35, 3315-3317 (2010); Stephen A. Boppart, Mark E. Brezinski and James G. Fujimoto, “Surgical Guidance and Intervention,” in Handbook of Optical Coherence Tomography, B. E. Bouma and G. J Tearney, ed. (Marcel Dekker, New York, N.Y., 2001)). For clinical intraoperative purposes, a FD-OCT system should be capable of ultrahigh-speed, raw data acquisition as well as matching-speed data processing and visualization. In recent years, the A-scan acquisition rate of FD-OCT systems has generally reached multi-hundred-of-thousand line/second level (W-Y. Oh, B. J. Vakoc, M. Shishkov, G. J. Tearney, and B. E. Bouma, “>400 kHz repetition rate wavelength-swept laser and application to high-speed optical frequency domain imaging,” Opt. Lett. 35, 2919-2921 (2010); B. Potsaid, B. Baumann, D. Huang, S. Barry, A. E. Cable, J. S. Schuman, J. S. Duker, and J. G. Fujimoto, “Ultrahigh speed 1050 nm swept source/Fourier domain OCT retinal and anterior segment imaging at 100,000 to 400,000 axial scans per second,” Opt. Express 18, 20029-20048 (2010), http://www.opticsinfobase.org/oe/abstract.cfm→URI=oe-18-19-20029) and approaching the multi-million line/second level (W. Wieser, B. R. Biedermann, T. Klein, C. M. Eigenwillig, and R. Huber, “Multi-Megahertz OCT: High quality 3D imaging at 20 million A-scans and 4.5 GVoxels per second,” Opt. Express 18, 14685-14704 (2010), http://www.opticsinfobase.org/oe/abstract.cfm?URI=oe-18-14-14685; T. Bonin, G. Franke, M. Hagen-Eggert, P. Koch, and G. Hüttmann, “In vivo Fourier-domain full-field OCT of the human retina with 1.5 million A-lines/s,” Opt. Lett. 35, 3432-3434 (2010)). Recent developments of graphics processing unit (GPU) accelerated FD-OCT processing and visualization have enabled real-time 4D (3D+time) imaging at the speed up to 10 volume/second (K. Zhang and J. U. Kang, “Real-time 4D signal processing and visualization using graphics processing unit on a regular nonlinear-k Fourier-domain OCT system,” Opt. Express 18, 11772-11784 (2010), http://wwww.opticsinfobase.org/abstract.cfm?URI=oe-18-11-11772; M. Sylwestrzak, M. Szkulmowski, D. Szlag and P. Targowski, “Real-time imaging for Spectral Optical Coherence Tomography with massively parallel data processing,” Photonics Letters of Poland, 2, 137-139 (2010); J. Probst, D. Hillmann, E. Lankenau, C. Winter, S. Oelckers, P. Koch, G. Hüttmann, “Optical coherence tomography with online visualization of more than seven rendered volumes per second,” J. Biomed. Opt. 15, 026014 (2010)). However, these systems all work in the standard mode, and therefore suffer from spatially reversed complex-conjugate ghost images. During intraoperative imaging, for example, when long-shaft surgical tools are used, such ghost images could severely misguide the surgeons. As a solution according to an embodiment of the current invention, a GPU-accelerated full-range FD-OCT has been utilized and real-time B-scan images were demonstrated with effective complex-conjugate suppression and doubled imaging range (K. Zhang and J. U. Kang, “Graphics processing unit accelerated non-uniform fast Fourier transform for ultrahigh-speed, real-time Fourier-domain OCT,” Opt. Express 18, 23472-23487 (2010), http://www.opticsinfobase.org/abstract.cfm?URI=oe-18-22-23472; Y. Watanabe, S. Maeno, K. Aoshima, H. Hasegawa, and H. Koseki, “Real-time processing for full-range Fourier-domain optical-coherence tomography with zero-filling interpolation using multiple graphic processing units,” Appl. Opt. 49, 4756-4762 (2010)).
In this example according to an embodiment of the current invention, we implemented a real-time, 4D full-range complex-conjugate-free FD-OCT based on a dual-GPU architecture, where one GPU was dedicated to the FD-OCT data processing while the second one was used for the volume rendering and display. GPU-based, non-uniform, fast Fourier transform (NUFFT) (Y. Watanabe, S. Maeno, K. Aoshima, H. Hasegawa, and H. Koseki, “Real-time processing for full-range Fourier-domain optical-coherence tomography with zero-filling interpolation using multiple graphic processing units,” Appl. Opt. 49, 4756-4762 (2010)) is also implemented to suppress the side lobes of the point spread function and to improve the image quality. With a 128,000 A-scan/second OCT engine, we obtained 5 volumes/second 3D imaging and display. We have demonstrated the real-time visualization capability of the system by performing a micro-manipulation process using a vitro-retinal surgical tool and a phantom model. Multiple-volume rendering of the same 3D data set was performed and displayed with different view angles. This technology can provide the surgeons with comprehensive intraoperative imaging of the microsurgical region which could improve the accuracy and safety of microsurgical procedures.
The system configuration is shown in
A quad-core Dell T7500 workstation was used to host the frame grabber (PCIE-x4 interface), DAQ card (PCI interface), GPU-1 and GPU-2 (both PCIE-x16 interface), all on the same mother board. GPU-1 (NVIDIA GeForce GTX 580) with 512 stream processors, 1.59 GHz processor clock and 1.5 GBytes graphics memory is dedicated for raw data processing of B-scan frames. GPU-2 (NVIDIA GeForce GTS 450) with 192 stream processors, 1.76 GHz processor clock and 1.0 GBytes graphics memory is dedicated for the volume rendering and display of the complete C-scan data processed by GPU-1. The GPU is programmed through NVIDIA's Compute Unified Device Architecture (CUDA) technology (NVIDIA, “NVIDIA CUDA C Programming Guide Version 3.2,” (2010)). The software is developed under the Microsoft Visual C++ environment with National Instrument's IMAQ Win32 APIs.
The signal processing flow chart of the dual-GPU architecture is illustrated in
Compared to previously reported systems, this dual-GPU architecture separates the computing task of the signal processing and the visualization into different GPUs, which can provide the following advantages:
(1) Assigning different computing tasks to different GPUs makes the entire system more stable and consistent. For the real-time 4D imaging mode, the volume rendering is only conducted when a complete C-scan is ready, while B-scan frame processing is running continuously. Therefore, if the signal processing and the visualization are performed on the same GPU, competition for GPU resource will happen when the volume rendering starts while the B-scan processing is still ongoing, which could result in instability for both tasks.
(2) It will be more convenient to enhance the system performance from the software engineering perspective. For example, the A-scan processing could be further accelerated and the point spread function (PSF) could be refined by improving the algorithm with GPU-1, while more complex 3D image processing tasks such as segmentation or target tracking can be added to GPU-2.
In our experiment, the B-scan size is set to 256 A-scans with 1024 pixel each. Using the GPU based NUFFT algorithm, GPU-1 achieved a peak A-scan processing rate of 252,000 lines/s and an effective rate of 186,000 lines/s when the host-device data transferring bandwidth of PCIE-x16 interface was considered, which is higher than the camera's acquisition line rate. The NUFFT method was effective in suppressing the side lobes of the PSF and in improving the image quality, especially when surgical tools with metallic surface are used. The C-scan size is set to 100 B-scans, resulting in 256×100×1024 voxels (effectively 250×98×1024 voxels after removing of edge pixels due to fly-back time of galvanometers), and 5 volumes/second. It takes GPU-2 about 8 ms to render one 2D image with 512×512 pixel from this 3D data set using the ray-casting algorithm (K. Zhang and J. U. Kang, “Real-time 4D signal processing and visualization using graphics processing unit on a regular nonlinear-k Fourier-domain OCT system,” Opt. Express 18, 11772-11784 (2010), http://wvvw.opticsinfobase.org/abstract.cfm?URI=oe-18-11-11772).
First, we tested the optical performance of the system using a mirror as the target. At both sides of the zero-delay, PSFs at different positions are processed as A-scans using linear interpolation with FFT (
Then, the in vivo human finger imaging is conducted to test the imaging capability on biological tissue. The scanning range is 3.5 mm (X)×3.5 mm (Y) lateral and 3 mm (Z) for the axial full-range. The finger nail fold region is imaged as
Finally, we performed a real-time 4D full-range FD-OCT guided micro-manipulation using a phantom model and a vitreoretinal surgical forceps, with the same scanning protocol as
In this example, a real-time 4D full-range FD-OCT system is implemented based on the dual-GPUs architecture. The computing task of signal processing and visualization into different GPUs and real-time 4D imaging and display of 5 volume/second has been obtained. A real-time 4D full-range FD-OCT guided micro-manipulation is performed using a phantom model and a vitreoretinal surgical forceps. This technology can provide the surgeons with a comprehensive spatial view of the microsurgical site and can be used to guide microsurgical tools during microsurgical procedures effectively.
Compared to other imaging modalities such as CT, ultrasound and MRI, which have already been widely used in image-guided intervention (IGI), OCT has the disadvantage of shallower imaging penetration depth. As a solution, endoscopic OCT has been proposed for intra-body imaging (Z. Yaqoob, J. Wu, E. J. McDowell, X. Heng, and C. Yang, “Methods and application areas of endoscopic optical coherence tomography,” J. Biomed. Opt. 11, 063001 (2006)). A wide range of miniature endoscopic OCT probes have been developed to provide high resolution imaging while being flexible and integratable with medical devices such as rotary OCT imaging needles (X. Li, C. Chudoba, T. Ko, C. Pitris, and J. G. Fujimoto, “Imaging needle for optical coherence tomography,” Opt. Lett. 25, 1520-1522 (2000)) and balloon catheters (J. Xi, L. Huo, Y. Wu, M. J. Cobb, J. Ha Hwang, and X. Li, “High-resolution OCT balloon imaging catheter with astigmatism correction,” Opt. Lett. 34, 1943-1945 (2009)), paired-angle-rotation scanning (PARS) probes (J. Wu, M. Conry, C. Gu, F. Wang, Z. Yaqoob, and C. Yang, “Paired-angle-rotation scanning optical coherence tomography forward-imaging probe,” Opt. Lett. 31, 1265-1267 (2006)), polymer-based scanning cantilevers (Y. Wang, M. Bachman, G-P. Li, S. Guo, B. J. F. Wong, and Z. Chen, “Low-voltage polymer-based scanning cantilever for in vivo optical coherence tomography,” Opt. Lett. 30, 53-55 (2005)), piezoelectric scanning mirrors (K. H. Gilchrist, R. P. McNabb, J. A Izatt and S. Grego, “Piezoelectric scanning mirrors for endoscopic optical coherence tomography,” J. Micromech. Microeng. 19, 095012 (2009)), MEMS scanning mirrors (A. D. Aguirre, P. R. Hertz, Y. Chen, J. G. Fujimoto, W. Piyawattanametha, L. Fan, and M. C. Wu, “Two-axis MEMS scanning catheter for ultrahigh resolution three-dimensional and en face imaging,” Opt. Express 15, 2445-2453 (2007); K. H. Kim, B. H. Park, G. N. Maguluri, T. W. Lee, F. J. Rogomentich, M. G. Bancu, B. E. Bouma, J. F. de Boer, and J. J. Bernstein, “Two-axis magnetically-driven MEMS scanning catheter for endoscopic high-speed optical coherence tomography,” Opt. Express 15, 18130-18140 (2007)), piezoelectric (X. Liu, M. J. Cobb, Y. Chen, M. B. Kimmey, and X. Li, “Rapid-scanning forward-imaging miniature endoscope for real-time optical coherence tomography,” Opt. Lett. 29, 1763-1765 (2004)) and magnetic force-driven resonant fiber scanners (E. J. Min, J. Na, S. Y. Ryu, and B. H. Lee, “Single-body lensed-fiber scanning probe actuated by magnetic force for optical imaging,” Opt. Lett. 34, 1897-1899 (2009); E. J. Min, J. G. Shin, Y. Kim, and B. H. Lee, “Two-dimensional scanning probe driven by a solenoid-based single actuator for optical coherence tomography,” Opt. Lett. 36, 1963-1965 (2011)). With the rapid development in OCT technologies, in recent years endoscopic OCT has undergone a transition from traditional low-speed, time-domain mode to high-speed Fourier domain mode (L. Huo, J. Xi, Y. Wu, and X. Li, “Forward-viewing resonant fiber-optic scanning endoscope of appropriate scanning speed for 3D OCT imaging,” Opt. Express 18, 14375-14384 (2010); S. Moon, S-W. Lee, M. Rubinstein, B. J. F. Wong, and Z. Chen, “Semi-resonant operation of a fiber-cantilever piezotube scanner for stable optical coherence tomography endoscope imaging,” Opt. Express 18, 21183-21197 (2010)).
However, as in the regular bulky-scanner based systems, the FD-OCT imaging probes can also suffer from complex-conjugate ghost image that could severely misguide the users. For some cases, a transparent window can be used to keep the image target within one side of the zero-delay position [6], however, this method automatically sacrificed half of the image range and is not applicable for many other circumstances. In this work, we implemented the full-range, complex-conjugate-free FD-OCT with a forward-viewing miniature resonant fiber-scanning probe. A galvanometer-driven reference mirror provides a linear phase modulation to the A-scans within one frame and the simultaneous B-M-mode scanning are implemented. Inside the probe, a fiber cantilever is driven by a low-voltage magnetic transducer synchronized to the reference mirror scanning. Using a CCD line-scan camera-based spectrometer, we demonstrated real-time full-range FD-OCT imaging with doubled imaging range at 34 frame/s.
First we designed a forward-viewing miniature resonant fiber-scanning probe based on a low-voltage miniature magnetic transducer, as shown in
The probe is integrated into a spectrometer-based FD-OCT system, as shown in
To induce a phase modulation, as in
and λ0 is the central wavelength of the light source. N is the number of A-scans within each frame. In our setup, with λ0=825 nm, we set R=3 mm, Δα=2°, and N=1024.
As shown in
where Es (t|x,λ) and Er(λ) are the electric fields from the sample and reference arms, respectively. Γf|u { } is the correlation operator. The first three terms on the right hand of Eq. (5.2) present the DC noise, autocorrelation noise, and complex-conjugate noise, respectively. The last term can be filtered out by a proper band-pass filter in the f|u domain and then convert back to the t|x domain by applying an inverse Fourier transform along the f|u direction. The complete data processing can be divided into following steps:
In particular, here step (vi) is realized by adding a phase correction term
We then processed the M-scan data frame in
A single A-scan is extracted from the frame in
In the imaging test, the probe was operated under the single-direction scanning mode at 34 frame/s, and only the odd frames were acquired and processed with an image size of 1024 pixel lateral by 2048 pixel axial.
Finally, we conducted in vivo human finger imaging using the scanning probe with the same scanning protocol as depicted in
In this example, a full-range FD-OCT imaging probe with a magnetic-driven resonant fiber cantilever was developed. The simultaneous B-M-mode phase modulation was implemented by scanning the reference mirror synchronized to the sample arm. The complete image processing was presented and in vivo human finger images were obtained with the complex-conjugate artifact removal. This method can also be generally applied to other types of FD-OCT imaging probes to eliminate the complex-conjugate artifact and double the imaging range.
Optical dispersion mismatch is a common issue for all Michelson-type OCT systems, especially for an ultra-high resolution FD-OCT system using an extremely broadband light source. Therefore dispersion compensation is essential for such systems.
Hardware methods usually match the dispersion of the sample arms physically by putting dispersive optical components on the reference arm. One simple way is to use identical optical components. An alternative way is to use a dispersion matching prism pair. Both methods obviously cost extra and a perfect matching is usually difficult to realize in many cases especially when the dispersion mismatch is the result of the dispersion in the sample itself.
In comparison, numerical dispersion compensation is cost-effective and adaptable. Numerical dispersion compensation can be performed by adding a phase correction to the complex spectrum obtained via Hilbert transformation from the original spectrum (M. Wojtkowski, V. Srinivasan, T. Ko, J. G. Fujimoto, A. Kowalczyk, and J. Duker, “Ultrahigh-resolution, high-speed, Fourier domain optical coherence tomography and methods for dispersion compensation,” Opt. Express, vol. 12, pp. 2404-2422, 2004),
where a2 and a3 can be pre-optimized values according to the system properties. In most cases, the majority of the dispersion mismatch comes from the optical system itself and the contribution from the image target is usually small. Even for retinal imaging with more than 20 mm of vitreous humor, the region of interest in the depth is usually within 1 mm, therefore in an ultrahigh speed imaging mode, a2 and a3 can be applied to all A-scans of the image frame or volume.
The embodiments illustrated and discussed in this specification are intended only to teach those skilled in the art how to make and use the invention. In describing embodiments of the invention, specific terminology is employed for the sake of clarity. However, the invention is not intended to be limited to the specific terminology so selected. The above-described embodiments of the invention may be modified or varied, without departing from the invention, as appreciated by those skilled in the art in light of the above teachings. It is therefore to be understood that, within the scope of the claims and their equivalents, the invention may be practiced otherwise than as specifically described.
This application claims priority to U.S. Provisional Application No. 61/426,399; 61/426,403; and 61/426,406 each of which were filed Dec. 22, 2010, the entire contents of which are hereby incorporated by reference.
This invention was made with Government support of Grant No. R21 1R21NS0633131-01A1, awarded by the Department of Health and Human Services, The National Institutes of Health (NIH). The U.S. Government has certain rights in this invention.
Filing Document | Filing Date | Country | Kind | 371c Date |
---|---|---|---|---|
PCT/US11/66603 | 12/21/2011 | WO | 00 | 6/21/2013 |
Number | Date | Country | |
---|---|---|---|
61426399 | Dec 2010 | US | |
61426403 | Dec 2010 | US | |
61426406 | Dec 2010 | US |