The present invention relates to methods of quantitative phase imaging, and, more particularly, to tomographic imaging using white light illumination of substantially transparent specimens.
Objectives and techniques of quantitative phase imaging (QPI) are described in detail in Popescu, Quantitative Phase Imaging of Cells and Tissues, (McGraw-Hill, 2011), incorporated herein by reference. In QPI, the optical path length associated with substantially transparent specimens is measured and mapped, and translated into biomedically relevant information.
Off-axis phase imaging methods have provided the fastest acquisition rates by virtue of the fact that phase information, and thus optical path lengths, are extracted from a single recorded interferogram, as described, for example in Ikeda et al., Hilbert phase microscopy for investigating fast dynamics in transparent systems, Opt. Lett., vol. 30, pp. 1165-67 (2005), incorporated herein by reference. While diffraction-limited transverse resolution is intrinsically preserved in phase shifting methods, off-axis techniques may degrade transverse resolution. The highest temporal phase sensitivity (that is, the smallest frame-to-frame phase shift) is provided by common-path methods because they are the most stable. Finally, the highest spatial phase sensitivity (i.e., the smallest point-to-point phase change within the same frame) is obtained in the absence of speckles, which implies an incoherence in at least some respect—of the source of illumination.
Diffraction phase microscopy (DPM), as described in Popescu et al., Diffraction phase microscopy for quantifying cell structure and dynamics, Opt. Lett., vol. 31, pp. 775-77 (2006), incorporated herein by reference, is both off-axis and common-path, and thus combines the benefits of fast acquisition rates and high temporal sensitivity. These features enabled DPM to perform unprecedented biological studies, especially related to red blood cell membrane dynamics. Diffraction phase microscopy with white light is the subject of US Published Patent Application US 2014/0085715, which is incorporated herein by reference.
Spatial light interference microscopy (SLIM), described, for example, in U.S. Pat. No. 8,184,298 (hereinafter, “Popescu '298”), entitled “Spatial Light Interference Microscopy,” and by Wang et al., Spatial Light Interference Microscopy (SLIM), Opt. Exp., vol. 19, pp. 1016-26 (2011), both of which are incorporated herein by reference, combines Zernike's phase contrast microscopy and Gabor's holography, has been shown to be of great value in measuring nanoscale structures and dynamics in live cells by generating endogenous contrast and mapping quantitative optical path-lengths at each point in the image.
In accordance with embodiments of the invention, methods are provided for obtaining a tomographic phase image of a substantially transparent specimen. The methods have steps of:
In accordance with alternate embodiments of the invention, the specimen may be substantially transparent in a visible portion of the electromagnetic spectrum, or, alternatively, the specimen may be opaque in the visible portion of the electromagnetic spectrum. The specimen may be substantially transparent in an infrared portion of the electromagnetic spectrum. More particularly, the specimen may be a biological cell.
In accordance with further embodiments of the present invention, forming a scattered phase image of the specimen in an image plane may correspond to measuring a temporal cross-correlation function between a scattered field and a reference plane wave, which, more particularly, may traverse the specimen.
In accordance with yet further embodiments of the present invention, forming a scattered phase image may include combining a plurality of interferograms obtained with distinct phase retardation rings. Forming a phase image may include obtaining a single interferogram from an off-axis type interferometer, such as a diffraction phase microscopy, in particular.
The foregoing features of the invention will be more readily understood by reference to the following detailed description, taken with reference to the accompanying drawings, in which:
Further details regarding the invention described herein may be obtained by reference to Kim et al., White-light diffraction tomography of unlabeled live cells, Nature Photonics, DOI: 10.1038/NPHOTON.2013.350 (Jan. 19, 2014) (hereinafter, “Kim (2014)”), and in the Supplementary Information appended thereto, both of which are incorporated herein by reference.
The term “image” shall refer to any multidimensional representation, whether in tangible or otherwise perceptible form, or otherwise, whereby a value of some characteristic (amplitude, phase, etc.) is associated with each of a plurality of locations corresponding to dimensional coordinates of an object in physical space, though not necessarily mapped one-to-one thereonto. Thus, for example, the graphic display of the spatial distribution of some field, either scalar or vectorial, such as brightness or color, constitutes an image. So, also, does an array of numbers, such as a 3D holographic dataset, in a computer memory or holographic medium. Similarly, “imaging” refers to the rendering of a stated physical characteristic in terms of one or more images.
As used herein, “label-free” refers to a method of imaging a substantially transparent sample that does not require introduction of materials, such as fluorophores or contrast agents that are extrinsic to the imaged sample.
The terms “object,” “sample,” and “specimen” shall refer, interchangeably, to a tangible, non-transitory physical object capable of being rendered as an image.
The term “temporally incoherent” as applied to a source of irradiation shall refer to a source characterized by a coherence time τc that is no longer than 10 cycles of a central frequency of the light emitted by the source.
The term “substantially high spatial coherence” as applied to a beam of light shall denote a condition of coherence wherein if two slits were to be placed within the a coherence area exceeding the field of view in a plane transverse to the beam propagation axis, at least one interference null would result in the far field where the intensity is no greater than 10% of the peak intensity of the beam.
“High numerical aperture,” for purposes of the present description and claims, shall refer to NA>0.5, which for visible light insures a transverse resolution better than one micron.
Comment Regarding Notation:
For convenience of notation in the current description, the same symbol is used for a function and for its Fourier transform. In order to indicate the domain in which the function operates, all the arguments are carried explicitly, e.g., f(k⊥,z; ω) is the Fourier transform of f(r⊥, z; t) over r⊥ and t.
Embodiments of the present invention, significantly, measure a complex field at an image plane. Complex field measurements in an image plane have been shown, as in Ding et al., Fourier transform light scattering of inhomogeneous and dynamic structures, Phys. Rev. Lett., vol. 101, 238102 (2008), incorporated herein by reference, to yield higher sensitivity than scatter measurements in the far zone. Once a complex field has been measured at successive depths to form a z-stack of complex images, the complex field stack is processed using white light diffraction tomography (WDT) in accordance with the following teachings of the present invention.
A sample (not shown) is transilluminated, in microscope 102, by a broadband source 103. In one example, the broadband source is a halogen lamp with a temporal coherence length of 1.2 μm. The broadband source 103 (also referred to, herein, as a “lamp”) assumes a significant role in increasing resolution since it does not suffer from speckle, and thus, improves the sensitivity.
Insofar as the SLIM module 100 is preferably an add-on module to a commercial microscope, it is possible to overlay SLIM with other microscopy modalities (e.g. epi-fluorescence, or DIC), within the scope of the present invention. In particular, within the scope of the present invention, a phase image may be obtained from a single interferogram from an off-axis type interferometer, such as a diffraction phase microscope, for example.
With these features, SLIM is capable of performing multimodal and functional studies. Furthermore, with the short coherence length and a high numerical aperture objective, SLIM may advantageously provide excellent depth sectioning (1.2 μm depth sectioning for a 63×/1.4 NA objective), which plays a large role in the reconstruction method described herein.
It is known that the Rytov approximation is more appropriate for reconstructing smooth objects with respect to the wavelength of light, i.e., for low-resolution imaging, while the Born approximation works better for imaging finer structures. Thus, for purposes of resolving features of imaged cells, for example, the latter is appropriate, and is used for purposes of the discussion herein. Under the first-order Born approximation, illustrated in
For purposes of the present teachings, a temporal cross-correlation function, Γ12 (r,τ), is usefully defined as follows:
Γ12(r,τ)=Us(r,t)Ur*(r,t+τ), (1)
where Us and Ur are the scattered and the reference (unscattered) field, respectively. Γ12(r,τ) is a complex function that can be written as
Γ12(r,τ)=|Γ12(r,τ)|eiφ(r,τ), (2)
where φ(r,τ) is the phase associated with the image field, measured in SLIM using the four phase shifted interferograms, as discussed above. The generalized Wiener-Khintchine theorem (see Wiener, Generalized harmonic analysis, Acta Mathematica, vol. 55, pp. 117-258 (1930), incorporated herein by reference) allows Γ12(r,τ) to be related to the cross-spectral density through Fourier transform of
During the phase shifting measurement, Γ12(r,τ) is evaluated at τ=0, to yield:
in which the unknown quantity is the scattered field, Us. In the following, an analytical solution is derived for Us which establishes the relationship between the measurable quantity, Γ12, and the 3D object structure of interest.
The inhomogeneous Helmholtz equation, which describes the field U in a medium with index distribution of n(r) is
∇2U(r,ω)+n2(r,ω)β02(ω)U(r,ω)=0, (6)
where β0 (ω)=ω/c is the wavenumber in vacuum. Eq. (6) may be rearranged as
∇2U(r,ω)+β2(ω)U(r,ω)=−χ(r,ω)βo2(ω)U(r,ω), (7)
where β(ω)=
∇2Ui(r,ω)+β2(ω)Ui(r,ω)=0. (8)
Eq. (8) has a plane-wave solution Ui(r,ω)=A(ω)eiβ(ω)z where A(ω) is the spectral amplitude of the incident field. Subtracting Eq. (7) from Eq. (8) gives
∇2Us(r,ω)+β2(ω)Us(r,ω)=−χ(r)βo2(ω)U(r,ω). (9)
Eq. (9) makes clear that the driving term of the right hand side represents the interaction of the object scattering potential χ with the total field U. Under the first Born approximation, i.e., |Us(r,ω)|<<|Ui(r,ω)|, U(r,ω) on the right hand side can be approximated as Ui(r,ω), allowing Eq. (9) to be re-written as,
∇2Us(r,ω)+β2(ω)Us(r,ω)=−χ(r,ω)βo2(ω)A(ω)eiβ(ω)z. (10)
Instead of employing the traditional Green's function approach and the angular spectrum representation (Weyl's formula), the scattered field directly may be solved for directly in wavevector space, using the 3D Fourier transformation. First, a 3D Fourier transform of Eq. (10) is performed, which yields
[β2(ω)−k2]Us(k,ω)=−β02(ω)A(ω)χ[k⊥,kz−β(ω)]. (11)
In Eq. (11), we used the shift theorem of Fourier transforms, namely, χ(r,ω)eiβ(ω)z→χ[k⊥,kz−β(ω)], where the arrow indicates Fourier transformation. Note that, throughout the manuscript, we use the same symbol for a function and its Fourier transform but carry all the arguments explicitly, which clearly identifies the domain in which the function operates. For example, χ(k) is the Fourier transform of χ(r). From Eq. (1), the scattered field Us s obtained immediately as
Next, an expression is derived for the field as a function of axial distance z, i.e., we arrange the terms such that a 1D inverse Fourier transform with respect to kz can be easily performed. Toward this end, a k⊥-dependent propagation constant is defined, q=√{square root over (β2(ω)−k⊥2)}, Eq. (12) is rewritten as
In instances where the transmitted or forward-scattered field is acquired, the backscattered field (the second term in the right-most parentheses) may be ignored. Conversely, if the backscattered field is exclusively acquired, the forward scattering term may be ignored. However, implementation of the methods of the present invention to encompass the backscattered field is similarly within the capability of a person of ordinary skill in the art, once the present teachings have been appreciated. Solely for heuristic purposes, the teachings that follow will be expressed in terms of a forward scattered field, though their application to backscattered fields is within the scope of the present invention.
Performing an inverse Fourier transform on Eq. (13) with respect to kz in order to obtain the forward-scattered field (for example) as a function of transverse wavevector, k⊥, axial distance, z, and angular frequency, ω, one obtains:
In Eq. (14), {circle around (v)}z indicates convolution along the z-dimension,
can be easily seen that the convolution of a function with a complex exponential yields the Fourier transform of that function multiplied the complex exponential, which yields the simple result [χ(k⊥,z; ω)eiβ(ω)z]{circle around (v)}zeiqz=eiqzχ[k⊥,q−β(ω)].
In order to insert the result of Eq. (14) into Eq. (3), Γ12(r,τ) is Fourier transformed with respect to the transverse position vector, r⊥=(x,y). Since Ur(r,ω) is a plane wave propagating in z-direction, Eq. (a4) in k⊥-space is
W12(k⊥,z,ω)=Us(k⊥,z,ω)Ur*(z,ω). (15)
Using the solution of Us from Eq. (b9) and Ur(z,ω)=A(ω)eiβ(ω)z, one has
where S(ω)=|A(ω)|2 is the power spectrum of the illumination field. Using Eq. (3), one obtains the temporal cross correlation at zero-delay as a function of the frequency integral
With the relation β(ω)=
In practicing the present invention, what is measured is S(λ). So, to obtain the spectrum distribution for S(βc/
Substituting dβ for dQ, the Jacobian becomes
and Eq. (18) becomes:
In Eq. (19), Σ is the function that incorporates all the details of the instrument response. Note that the k⊥ coverage of Σ is limited to a maximum value k⊥max=β0NA, where NA is the numerical aperture of the objective. By measuring z-stacks in SLIM, it becomes possible to reconstruct the object's 3D distribution through deconvolution of Eq. (19). Alternatively, Eq. (19) may be written in the spatial frequency domain as a product, namely,
Calculating the Point Spread Function (PSF)
Since Eq. (21) gives the coherent transfer function, i.e., the Fourier transform of the point spread function (PSF) of the imaging system, therefore, by measuring the source spectrum and filtering according to the NA of the objective, the coherent transfer function of the system can be calculated directly. The PSF of the imaging system may also be referred to herein equivalently as an “instrument function.”
First, the optical spectrum of the lamp 103 is measured at a specified temperature, such as 3200K, using a fiber optic spectrometer, for example. Insofar as the spectrum SW is measured as in air, we perform the Jacobian transformation introduced in the previous section to obtain the spectrum in terms of the variable β=−(Q2+k⊥2)/2Q. Next through numerical calculations, this spectrum is resampled onto a 3D grid in spatial-frequency space (kx, ky, Q). For a 63×/1.4 NA objective, each pixel in space corresponds to 45 nm and each z-slice is separated by 200 nm. Because of the quadratic relationship between Q and β, the resampling yields two duplicates of the spectrum. Therefore, the second of the two duplicates which appears at high Q is removed by applying a spatial low-pass filter with cutoff Q=k⊥. Further filtering in Q is performed to incorporate the physical minimum and maximum value of Q and the maximum value, which is determined by the NA of the objective, k⊥max=β0NA. In order to avoid well known numerical artifacts due to sharp cutoffs in the frequency domain, we used a simple apodization procedure to smooth the edges of the filter function. The filter function may be convolved with a narrow Gaussian function, of a width that is 5% of the system's maximum transverse frequency. Finally, the coherent transfer function is Fourier transformed to obtain the PSF. The complex PSF is then used for the 3D reconstruction, as detailed below.
Deconvolution
As discussed above, SLIM measures the 3D complex field distribution, i.e. phase and amplitude, which can be expressed as a convolution between the point spread function (PSF) of the imaging system, Σ(r), and the structure of the object, χ(r)=n2(r)−
The function that is minimized, in accordance with a preferred embodiment of the invention, is
where σ2 is the noise variance, ρ is the regularization parameter, and R is the regularization functional. For simplification, these purely phase-dependent fields can be expressed in vector forms:
where g represents the measured field, exp[i
Phase contrast (PC) images are highly sensitive to the sharp object boundaries, but not to slow variations in the background region. For small scale objects, such as a biological cell, these characteristics of PC permit the use the sparse representation, described, for example, in Shechtman et al., Sparsity-based sub-wavelength imaging with partially incoherent light via quadratic compressed sensing, Opt. Express, vol. 19, pp. 14807-22 (2011), incorporated herein by reference. When an appropriate transform is applied to a phase image, only a few of the transform coefficients contain most of the signal energy while all the other coefficients become very small. This situation is known as transform sparsity. In a preferred embodiment of the present invention, the first- and second-order directional difference operators, [−1 1] and [−1 2 −1], along with 45° and −45° first-order derivative filters,
are used as transforms. For each plane in the image, x-y, y-z and z-x, these 2D transforms are applied and total of 12 transforms are generated to hold the spatial variations. Therefore, by applying the sparse deconvolution principle with these filters, the problem in Eq. (23) can be expressed as
where αki are the weighting coefficients for each plane, i, Dk are the transform matrices, and Ak are diagonal matrices with αki in the diagonal. This problem is solved by an alternating iterative minimization scheme where only one unknown is estimated at a time while others are fixed. As a result, the complex image f is estimated by taking derivative of Eq. (24) and setting it to zero. Therefore, the solution is of the structure is,
and the weighting coefficients are estimated to be,
where ε is a small number 10−9, in a preferred embodiment) used to avoid the trivial solution.
Application to Tomographic Reconstruction
It is known that It is known that the Rytov approximation is more appropriate for reconstructing smooth objects with respect to the wavelength of light, i.e., for low-resolution imaging, and the Born approximation works better for imaging finer structures. Thus, for purposes of resolving features of imaged cells, for example, the latter is appropriate, and is used for purposes of the discussion herein. Under the first-order Born approximation, with an incident plane wave, Ui=A(ω)eiβ(ω)z, the forward scattered field Us is solved in the wavevector space, as described above, instead of using the traditional Green's function and Weyl's formula approach. In the transverse wavevector domain, k⊥, Us can be expressed, per Eq. (13), as
with symbols as defined above. The dispersion in the object may be neglected, because most biological samples of interest here are weakly absorbing. This is true even for single red blood cells. Even though the haemoglobin absorbs strongly in blue, the overall absorption of visible light through a single red blood cell is very small. This is so because the absorption length of haemoglobin in a normal red blood cell is around 10 μm in blue (averaged over 400-500 nm wavelength) and around 3 mm in red (averaged over 600-750 nm wavelength), while the thickness of the cell is only 2-3 μm.
In conventional phase shifting interferometry, the cross-correlation of the scattered and reference fields is measured as Γ12(r⊥,z,τ)=Us(r⊥,z,t)Ur*(z,t+τ) at τ=0 which is equivalent to integrating the spectrum over ω. Knowledge of the spatial frequency response of our instrument, or coherent transfer function (see Supplementary Information), Σ(kx, ky, kz), allows us to write the main result of our calculation, i.e., the solution to the inverse scattering problem, in terms of the measured data, Γ12, and the instrument function (or the coherent transfer function), Σ, in the wave-vector domain as,
In practice, the operation in Eq. (28) requires regularization, as discussed above. Σ(k) is given by
In Eq. (29), S is the optical spectrum of the imaging field as a function of the wavenumber and Q=√{square root over (β2−k⊥2)}−β. The 3D point spread function can be obtained, as shown above, through an inverse Fourier transform of Eq. (29).
Qualitatively, Σ has a physically intuitive behaviour, of which salient features are depicted in
Most importantly, in methods in accordance with the present invention, the coherence gating works in synergy with the high numerical aperture (NA) optics and, thus, allows for high resolution tomography. In other words, in WDT, coherence gating by itself would not work at zero NA and, conversely, high NA gating would not work with monochromatic light.
Use of broadband light from source 103 and high numerical aperture objectives results in optical sectioning capabilities highly advantageous for high resolution tomography. The function Σ(kx, ky, kz) for an imaging system in accordance with embodiments of the present invention is illustrated in
The approach described herein in accordance with the present invention was applied to obtain three-dimensional images of Escherichia coli (E. coli) cells. A z-stack consisting of 17 slices with a step size of 280 nm was obtained.
WDT was also employed to obtain 3D images of speculated red blood cells, shown in Kim (2014), to which the interested reader is directed.
In order to study more complex subcellular structures, WDT was used to image human colon adenocarcinoma cells (HT29), using a 63×/1.4 NA oil immersion objective. The z-stack consisting of 140 frames was acquired in 150 nm z-steps. The results obtained on a cell that has recently divided are summarized in
In preferred embodiments of the present invention, the disclosed methods for tomographic quantitative phase imaging of transparent structures using white light diffraction phase microscopy may be implemented as a computer program product for use with a computer system. Such implementations may include a series of computer instructions fixed either on a tangible medium, such as a computer readable medium (e.g., a diskette, CD-ROM, ROM, or fixed disk) or transmittable to a computer system, via a modem or other interface device, such as a communications adapter connected to a network over a medium. The medium may be either a tangible medium (e.g., optical or analog communications lines) or a medium implemented with wireless techniques (e.g., microwave, infrared or other transmission techniques). The series of computer instructions embodies all or part of the functionality previously described herein with respect to the system. Those skilled in the art should appreciate that such computer instructions can be written in a number of programming languages for use with many computer architectures or operating systems. Furthermore, such instructions may be stored in any memory device, such as semiconductor, magnetic, optical or other memory devices, and may be transmitted using any communications technology, such as optical, infrared, microwave, or other transmission technologies. It is expected that such a computer program product may be distributed as a removable medium with accompanying printed or electronic documentation (e.g., shrink wrapped software), preloaded with a computer system (e.g., on system ROM or fixed disk), or distributed from a server or electronic bulletin board over the network (e.g., the Internet or World Wide Web). Of course, some embodiments of the invention may be implemented as a combination of both software (e.g., a computer program product) and hardware. Still other embodiments of the invention are implemented as entirely hardware, or entirely software (e.g., a computer program product).
The embodiments of the invention described above are intended to be merely exemplary; numerous variations and modifications will be apparent to those skilled in the art. All such variations and modifications are intended to be within the scope of the present invention as defined in any appended claims.
This application is a continuation-in-part of U.S. Ser. No. 13/727,875, and, through that application, claims priority from U.S. Provisional Application Ser. No. 61/582,599, filed Jan. 3, 2012. This application is also a continuation-in-part of U.S. Ser. No. 13/776,232, filed Feb. 25, 2013, and, through that application, claims priority from U.S. Provisional Patent Application Ser. No. 61/704,005, filed Sep. 21, 2012. All of the aforesaid US patent applications are incorporated herein by reference.
This invention was made with government support under Grants CBET0846660, CBET0939511, and CBET1040462, all awarded by the National Science Foundation. The Government has certain rights in the invention.
Number | Name | Date | Kind |
---|---|---|---|
6002480 | Izatt et al. | Dec 1999 | A |
8184298 | Popescu et al. | May 2012 | B2 |
20030065268 | Chen et al. | Apr 2003 | A1 |
20060158660 | Luttge et al. | Jul 2006 | A1 |
20090125242 | Choi | May 2009 | A1 |
20130088723 | Feldkhun | Apr 2013 | A1 |
20140219050 | Park et al. | Aug 2014 | A1 |
Entry |
---|
Choi et al., “Extended depth of focus in tomographic phase microscopy using a propagation algorithm,” Opt. Lett., vol. 33, No. 2, pp. 171-173 (Jan. 2008). |
Cuche et al., “Digital holography for quantitative phase-contrast imaging,” Opt. Lett., vol. 24, No. 5, pp. 291-293 (Mar. 1999). |
Ding et al., “Instantaneous spatial light interference microscopy,” Opt. Express, vol. 18, No. 2, pp. 1569-1575 (Jan. 2010). |
Halder et al., “Label-Free High-Resolution Imaging of Live Cells with Deconvolved Spatial Light Interference Microscopy,” 32nd Annual International Conference of the IEEE EMBS Buenos Aires, Argentina, pp. 3382-3385 (Aug.-Sep. 2010). |
Hillmann et al., “Holoscopy—holographic optical coherence tomography,” Opt. Lett., vol. 36, No. 13, pp. 2390-2392 (Jul. 2011). |
Ikeda et al., “Hilbert phase microscopy for investigating fast dynamics in transparent systems,” Opt. Lett., vol. 30, No. 10, pp. 1165-1167 (May 2005). |
Bhaduri et al., “Diffraction phase microscopy with white light,” Opt. Lett., vol. 37, No. 6, pp. 1094-1096 (Mar. 2012). |
Cheng et al., “Cell death detection by quantitative three-dimensional single-cell tomography,” Biomed. Opt. Express, vol. 3, No. 9, pp. 2111-2120 (Aug. 2012). |
McNally et al., “Three-Dimensional Imaging by Deconvolution Microscopy,” Methods, vol. 19, pp. 373-385 (1999). |
Popescu et al., “Diffraction phase microscopy for quantifying cell structure and dynamics,” Opt. Lett., vol. 31, No. 6, pp. 775-777 (Mar. 2006). |
Popescu, “Quantitative Phase Imaging of Nanoscale Cell Structure and Dynamics,” Method Cell Biol., Chapter 5, vol. 90, pp. 87-115 (2008). |
Wang et al., “Spatial light interference tomography (SLIT),” Opt. Express, vol. 19, No. 21, pp. 19907-19918 (Sep. 2011). |
Number | Date | Country | |
---|---|---|---|
20140307261 A1 | Oct 2014 | US |
Number | Date | Country | |
---|---|---|---|
61582599 | Jan 2012 | US | |
61704005 | Sep 2012 | US |
Number | Date | Country | |
---|---|---|---|
Parent | 13727875 | Dec 2012 | US |
Child | 14313118 | US | |
Parent | 13776232 | Feb 2013 | US |
Child | 13727875 | US |