1. Field of the Invention
This invention relates generally to the field of imaging. More specifically, the invention relates to a method and apparatus for high speed, three-dimensional microscopy with enhanced resolution.
2. Background of the Invention
Real time observations of dynamic processes in living cells are of increasing importance in experimental biology and have inspired the development of various noninvasive imaging techniques. One example is fluorescence-based light microscopy (fluorescence microscopy) which Applicant believes is presently the most popular technique for quantitative imaging of live specimen. The spatial resolution of such far-field microscopy is limited by the smallest possible size of a light spot produced by the focusing optics. The size of this light spot is determined by the wavelength of light and the numerical aperture of the employed objective lens. However, many subcellular structures of high research interest, such as mitochondria, endoplasmic reticulum, microtubules, and vesicles, have often sizes that are below this physical limit and thus cannot be resolved with conventional light microscopy.
Although there exist imaging techniques of significantly higher resolution than light microscopy, most of them require conditions that are hostile to living specimen, e.g., the high vacuum environment of electron microscopy. Therefore, considerable efforts have been made to increase the effective resolution of fluorescence microscopy, as it generally supports aqueous and thus physiological environments. Approaches of very different technical complexity have been taken and resulted in different levels of enhanced optical resolution. Applicant believes the most popular of these is confocal microscopy, which provides a resolution improvement of about √2. [See Pawley, J. B., “Handbook of Biological Confocal Microscopy,” 3 Ed., Springer Science, New York (2006)]. A best lateral resolution of approximately 16 nm has been achieved by stimulated emission depletion (STED) microscopy. This improvement to values well below Abbe's classical diffraction limit is achieved by STED of the fluorescent volume excited by a focused laser beam through an additional donut-shaped focus of a second laser beam at the emission wavelength. [Westphal, V., and Hell, S. W., “Nanoscale resolution in the focal plane of an optical microscope,” Phys. Rev. Lett. 94(14), 143903 (2005)]. Both techniques employ high-intensity point of illumination and thus may suffer from photo damage, which is a drawback for both approaches. In addition, STED instrumentation is highly cost intensive.
Consequently, there is a need for an improved three-dimensional imaging methods and systems.
A resolution enhancement of 2 (and greater), which is significant for many biological structures, can be achieved with by wide-field approaches known as structural illumination microscopy (SIM) [Gustafsson, M. G. L., “Surpassing the lateral resolution limit by a factor of two using structured illumination microscopy.” J. Microsc. 198(2), 82-87 (2000) (Gustafsson 2000)] and standing wave fluorescence microscopy (SWFM) [Frohn, J. T., Knapp, H. F., and Stemmer, A., “True optical resolution beyond the Rayleigh limit achieved by standing wave illumination.” Proc. Natl. Acad. Sci U.S.A. 97(13), 7232-7236 (2000) (Frohn, et al. 2000)]. Such resolution improvement is achieved by means of a periodic excitation pattern with high spatial frequencies. These patterns can be created by projecting a diffraction grating on the specimen (SIM) [Gustafson 2000; Heintzmann, R. and Cremer, C., “Laterally Modulated Excitation Microscopy: Improvement of resolution by using a diffraction grating,” Proc. SPIE, 3568, 185-196 (1999) (Heintzmann, et al. 1999)] or through interferometric schemes [Frohn 2000; Chung, E., Kim, D., and So, P. T. C. “Extended resolution wide-field optical imaging: objective-launched standing-wave total internal reflection fluorescence microscopy,” Opt. Lett. 31(7), 945-947 (2006) (Chung, et al. 2006); Gliko, O., Reddy, G. D., Anvari, B., Brownell, W. E., and Saggau, P., “Standing wave total internal reflection fluorescence microscopy to measure the size of nanostructures in living, cells.” J. Biomed. Opt., 11(6), 064013 (2006)] utilizing the objective lens for both illuminating and imaging.
Since the resolution improvement of SWM is one-dimensional, i.e., normal to the interference pattern, a two-dimensional (2D) lateral resolution requires rotating the pattern relative to the specimen. This has been achieved by mechanical rotation of the projected diffraction grating [Gustafsson 2000; Heintzmann, et al. 1999] or the specimen itself [Chung, et al. 2006; Chung, E., Kim. D., Cui, Y., Kim, Y. H., and So, P. T. C., “Two-dimensional standing wave total internal reflection fluorescence microscopy: Superresolution imaging of single molecular and biological specimens.” Biophys. J., 93(5), 1747-1757 (2007) (Chung, et al. 2007)], resulting in low imaging speed insufficient for real-time imaging. In addition, the excitation pattern is shifted by mechanical adjustment of the projected grating or interferometer path length.
Consequently, there is a need for an improved three-dimensional imaging methods and systems with sufficient imaging speed for real-time imaging.
This invention relates to methods and apparatuses for enhanced resolution microscopy.
In general, in one aspect, the invention features an imaging system that includes a light source, two programmable diffractive optical elements, a dichroic mirror, a lens, and an image acquisition device. The light source is directed at a beam splitter that can split the light from the light source to form two light beams. The two programmable diffractive optical elements are configured such that they can generate and control standing waves utilizing the two light beams. The lens defines a focal plane that is a fixed distance from the lens.
Implementations of the invention can include one or more of the following features:
The two programmable diffractive optical element can be acousto-optic deflectors.
The light source can be a laser.
The image acquisition device can be a CCD camera.
The imaging system further can include a third programmable diffractive optical element (for example, a third acousto-optic deflector). The three programmable diffractive optical elements (for example, the three acousto-optic deflectors) can be configured such that they (i) generate and control the standing waves utilizing the two light beams and (ii) provide two-dimensional control of the standing waves. In such configuration, for instance, both of the first two acousto-optic deflectors can be oriented orthogonally to the third acousto-optic deflector. Furthermore, the third acousto-optic deflector can be positioned to be employed before the beam splitter forms the two light beams.
The imaging system further can include a fourth programmable diffractive optical element (for example, a fourth acousto-optic deflector).
The four programmable diffractive optical elements (for example, the four acousto-optic deflectors) can be configured such that they (i) generate and control the standing waves utilizing the two light beams and (ii) provide two-dimensional control of the standing waves. In such configuration, for instance, both of the first two acousto-optic deflectors can be oriented orthogonally to both of the third and fourth acousto-optic deflectors (i.e., the first acousto-optic deflector is oriented orthogonally to the third acousto-optic deflector, etc.).
The four programmable diffractive optical elements (for example, the four acousto-optic deflectors) can also be configured such that they (i) generate and control the standing waves utilizing the two light beams and (ii) provide three-dimensional control of the standing waves. In such configuration, for instance, both of the first two acousto-optic deflectors can be oriented orthogonally to both of the third and fourth acousto-optic deflectors (i.e., the first acousto-optic deflector is oriented orthogonally to the third acousto-optic deflector, etc.). Furthermore, one of the acousto-optic deflectors (such as the third acousto-optic deflectors) can be positioned to be employed before the beam splitter forms the two light beams.
In general, in another aspect, the invention features a method that includes splitting a initial light beam with a beam splitter to form two light beams. The method further includes directing each of the two light beams to two programmable diffractive optical elements (i.e., one light beam to one programmable diffractive optical element and the other light beam to another programmable diffractive optical element). The method further includes using the two light beams (i) to scan the back-focal plane of at least one objective lens and (ii) to control the phase and orientation of standing waves. The method further includes acquiring images collected by the objective lens using an image acquisition device.
Implementations of the invention can include one or more of the features listed above, as well as the following features:
The two programmable diffractive optical element can be acousto-optic deflectors.
The method can include passing the light beam through a third programmable diffractive optical element (i.e., a third acousto-optic deflector) before the beam splitter to control the phase and orientation of the standing waves. Such method can also include (i) utilizing the three acousto-optic deflectors to control the phase and orientation of the standing waves to obtain a two-dimensional image and (ii) reconstructing the two-dimensional image using the images acquired.
The method can also include passing the light beam through third and fourth programmable diffractive optical elements (i.e., a third and fourth acousto-optic deflectors) to control the phase and orientation of the standing waves.
Such method can also include (i) utilizing the four acousto-optic deflectors to control the phase and orientation of the standing waves to obtain a two-dimensional image and (ii) reconstructing the two-dimensional image using the images acquired.
Such method can also include passing the light beam through the third programmable diffractive optical element (i.e., a third acousto-optic deflector) before the beam splitter. Such method can further include (i) utilizing the four acousto-optic deflectors to control the phase and orientation of the standing waves to obtain a three-dimensional image and (ii) reconstructing the three-dimensional image using the images acquired.
The method can include electronically controlling the acousto-optic deflectors.
The method can include using the acousto-optic deflectors to control the penetration depth of the standing waves.
The method can include that the initial light beam has a wavelength ranging from about 300 nm to about 1000 nm.
The method can include using the acousto-optic deflectors to laterally position the two light beams in the back focal plane.
In general, in another aspect, the invention features a microscopy system that includes a light source (for generating a light beam), a back focal plane scanner, a microscope, and an image acquisition device. The back focal plane scanner includes a beam conditioner, a scanner (including three acousto-optic deflectors), and a scanner control. The microscope includes a dichroic mirror and a lens.
Implementations of the invention can include one or more of the features listed above, as well as the following features:
The scanner can be a dual scanner or a triple scanner.
The system can include a beam splitter that is positioned to split the light from the light source to form two (or more) light beams.
One of the acousto-optic deflectors (such as the third acousto-optic deflector) can be positioned to be employed before the beam splitter. The other acousto-optic deflectors (such as the first two acousto-optic deflectors) can be positioned to be employed after the beam splitter. Such acousto-optic deflectors can be configured (i) to generate and control standing waves utilizing the two light beams and (ii) to provide two-dimensional control of the standing waves.
The system can include a fourth acousto-optic deflector with all four of the acousto-optic deflectors positioned to be employed after the beam splitter. Such acousto-optic deflectors can be configured (i) to generate and control standing waves utilizing the two light beams and (ii) to provide two-dimensional control of the standing waves.
The system can include a fourth acousto-optic deflector with one of the acousto-optic deflectors (such as the third acousto-optic deflector) positioned to be employed before the beam splitter. The other acousto-optic deflectors (such as the first two and the fourth acousto-optic deflectors) can be positioned to be employed after the beam splitter. Such acousto-optic deflectors can be configured (i) to generate and control standing waves utilizing the two light beams and (ii) to provide three-dimensional control of the standing waves.
The foregoing has outlined rather broadly the features and technical advantages of the invention in order that the detailed description of the invention that follows may be better understood. Additional features and advantages of the invention will be described hereinafter that form the subject of the claims of the invention. It should be appreciated by those skilled in the art that the conception and the specific embodiments disclosed may be readily utilized as a basis for modifying or designing other structures for carrying out the same purposes of the invention. It should also be realized by those skilled in the art that such equivalent constructions do not depart from the spirit and scope of the invention as set forth in the appended claims.
For a detailed description of the preferred embodiments of the invention, reference will now be made to the accompanying drawings in which:
Certain terms are used throughout the following description and claims to refer to particular system components. This document does not intend to distinguish between components that differ in name but not function.
In the following discussion and in the claims, the terms “including” and “comprising” are used in an open-ended fashion, and thus should be interpreted to mean “including, but not limited to . . . ” Also, the term “couple” or “couples” is intended to mean either an indirect or direct connection. Thus, if a first device couples to a second device, that connection may be through a direct connection, or through an indirect connection via other devices and connections.
The present invention is a method and apparatus that utilizes an inertia-free diffraction mechanism to control both phase and rotation of the standing wave pattern that results in super-resolution at unparalleled imaging speeds. In some embodiments of the present invention, AODs are utilized to control period, phase, and rotation of the SW pattern in contrast to the commonly used mechano-optical principles. This allows 2D (and 3D) super-resolution imaging at high stability and speed not limited by mechanical constraints.
While the making and/or using of various embodiments of the present invention are discussed below, it should be appreciated that the present invention provides many applicable inventive concepts that may be embodied in a variety of specific contexts. The specific embodiments discussed herein are merely illustrative of specific ways to make and/or use the invention and are not intended to delimit the scope of the invention.
The present invention utilizes programmable diffractive optical elements (DOEs) for controlling phase and orientation of standing waves (SWs) in contrast to the commonly used mechano-optical principles. The present invention makes use of the diffraction of light at an optical grating induced by high frequency sound waves propagating through a refractive medium. [See e.g., Milton, G., Ireland, C. L. M., and Ley, J. M., “Electro-optic and Acousto-optic Scanning and Deflection, Optical Engineering, Vol. 3, Marcel Dekker (1983); Xu, J. and Stroud, R., “Acousto-optic Devices: Principles, Design and Applications, John Wiley and Sons, New York (1992)]. A high speed microscope with three-dimensional laser beam scanning that included an acousto-optic deflector for controlling the lateral position and collimation or the light Beam was disclosed and taught in U.S. Pat. No. 7,227,127, issued Jun. 5, 2007 to Peter Saggau, et al., which patent is incorporated herein by reference in its entirety for all purposes.
The delay for shifting the phase of the SW is approximately 1000 times shorter than by moving a mirror with a fast piezo actuator, i.e., microseconds instead of milliseconds. This maximizes the ratio of imaging time to SW adjustment and increases the imaging speed. Also, since the phase is electronically controlled and no optical components are required to be moved, mechanical artifacts are excluded. Additionally, utilizing two AODs provides the ability to laterally position the focused laser beams in the back focal plane 307 of the objective lens 308. This provides the ability to achieve the smallest possible fringe spacing Δs by electronically matching the beam separation to the physical size of the back focal aperture.
In embodiments of the present invention, two 2D scanners 403 (such as shown in
As shown in
The AODs in the present embodiment were driven by RI frequency signals (such as about 80 to about 120 MHz) generated by direct digital synthesis (DDS) boards (AD9958, Analog Devices) and amplified by RF power amplifiers (DA134-2-100, Isomet). Optionally, spatial filters and mirrors (not shown) can be used to block the zero order beams and to align the deflected beams after passing through the AODs (x/yAODs 503 and x/yAODs 504). Both beams were combined by a second beam splitter and focused by a lens 509 at the back focal plane of an oil immersion 100× objective lens with NA 1.45. This resulted in two collimated beams that interfered at the local plane, creating a lateral periodic excitation pattern with adjustable fringe spacing and angular orientation. Both beams were circularly polarized by two polarizers (not shown) placed in front of the AODs. This resulted in slightly lower pattern contrast in comparison with s-polarized interfering beams.
This scheme can be used to avoid the mechanically (and therefore slow) rotation of polarizers to adjust polarization during the pattern rotation. The position of each beam a the back focal plane is determined by the two frequencies fx and fy. The frequency of each beam after being deflected by two AODs is shifted to ν=ν0+fx+fy, where ν0 is the frequency of the laser light. The point of symmetry of beam positions at the back local plane allows, optionally, the application of the same fx and fy, fx≠fy) to corresponding AODs, which results in the same shifted frequency v of both beams, which allows formation of the SW. The beam separation at the back focal plane determined the incident angle Θ and the SW period Δs=λex/(2nglass sin Θ), where λex is the excitation wavelength, and nglass is the refractive index of the glass (which was 1.52). The angular beam orientation γ at the back focal plane determines SW pattern orientation at the object plane. By adjusting the phase delay between the x/yAODs 503 and x/yAODs 504 RF signals, the SW phase can be controlled.
The 2D SWFM of
As illustrated in
This flexible AOD-based approach allows combining SWFM with total internal reflection fluorescence microscopy (TIRFM). Switching from SWFM to SW-TIRFM is achieved by adjusting the incident angle above the critical angle, Θc=61.2° by controlling the beam separation at the back focal plane. SW-TIRM allows axial selectivity (<100 nm) in addition to improved lateral movement. This technique can provide a real-time imaging of the subresolution structures in live biological specimens near the glass/water interface.
The modulated excitation pattern caused fluorescent emission, which was intensity-modulated with the same period as the excitation field. The fluorescence emitted by the specimen was collected by the same objective lens 710 and passed through the dichroic mirror 706 and an additional long-pass filter (not shown) to block residual excitation light. The fluorescence image was then magnified and captured with a cooled CCD camera 704. The phase of the SW pattern (
To achieve enhanced resolution in three dimensions, the SW pattern has to be extended in an axial direction. [Frohn. J. T., Knapp, H. F., and Stemmer, A., “Three-dimensional resolution enhancement in fluorescence microscopy by harmonic excitation.” Opt. Lett. 26(11) 828-830 (2001)]. One mechanism to accomplish this is to introduce a third coherent focused laser beam at the center of the back focal plane. This extra laser focus, together with the two lateral, axially symmetrical foci employed for 2D SWs (such as reflected in
The 2D SWFM illustrated in
In order to allow for SW formation, the frequency of the central beam is matched to the equally frequency shifted lateral beams. The design of
An optical unit such as a beam conditioner 1203 may be employed to improve the quality of the laser beam used for SW generation. This unit can also match the diameter of the laser beam (such as from CW laser 1201 illustrated in
Scanner 1204 may generate two output beams from one input beam and will allow steering of the two beams with the axial symmetry shown in
Scanner 1204 may contain a total of three computer-controlled AODs. These DOEs can be custom-made to specification. Precision mechanical parts may be used to hold and position the AODs and appropriate telecentric coupling optics with adjustable slit apertures to obstruct unwanted secondary diffraction orders.
The AODs are electronically controlled by RF (radio frequency) signals that are transformed into acoustic waves to interact with the laser beam. This can be done utilizing the scanner control 1205. In the interferometric application, the high frequency and phase stability between the AODs is optimal, and can be easily achieved by using Direct Digital Synthesis (DDS). Since this digital technique may be used with one stable central clock for multi-channel systems, it results in frequency and phase synchrony that easily exceeds requirements.
The output of scanner 1204 may be coupled to an inverted epi-fluorescence microscope 1216. Microscope 1216 may be modified to support access to the back focal plane 1210 of the objective lens 1215 without utilizing the existing illumination path that is not suited for interferometric use such as SW generation since it would cause significant wave front distortions. The microscope 1216 further includes a dichroic mirror 1209 and has an object plane 1211 associated with the objective lens 1215). A highly sensitive and fast EMCCD (electron multiplying charge coupled device) camera 1214 (or other fast and sensitive detector unit) may be coupled to the microscope with optics that provide appropriate secondary magnification. Such magnification is favored since the pixel size of 16 μm and the objective lens magnification is 100, resulting in an effective pixels size of 160 nm. This effective pixel size is similar to the fringe spacing of the proposed system (approximately 170 nm), when using a 488 nm laser line. Therefore, a secondary magnification of approximately 10 is optimal in order to achieve sufficient lateral resolution of the phase-shifted raw images taken by the camera 1214. This magnification could also be regarded as a spatial oversampling factor of approximately 5.
To further illustrate various illustrative embodiments of the invention, the following examples are provided.
The resolution of the SW-TIRFM system was tested by imaging fluorescent beads with a known diameter of 100 nm, which is below the resolution limit of standard microscopy.
Beads suspended in water were delivered to a polylysine coated coverslip, which caused a portion of the beads to adhere to the coating. Utilizing TIR illumination improved the axial selection, which resulted in images of high contrast, with very bright beads adjacent to the glass/water interface and a dark background undisturbed by non-adhered beads. Both TIRFM and reconstructed SW-TIRFM images of the same 100 nm bead are shown in
The resolution is enhanced in the direction normal to interference fringes. This intensity profile has two side lobes, which are intrinsic features of SWM. Such side lobes result from the convolution of an object with the effective point spread function (PSF) of SWM, which is a product of the standard PSF and excitation SW intensity. Since the side lobes were below 50% of central maximum, the object size can be unambiguously extracted by linear deconvolution. A method known as inverse-filtering [Krishnamurthi, V., Bailey, B, and Lanni, F., “Image processing in 3D standing wave fluorescence microscopy,” Proc. SPIE. 2655: 18-25 (1996)] was used as follows: the Fourier transform of the reconstructed SWFM image was divided by the effective optical transfer function (OTF) of SWM, then transformed back using only values within the bandwidth of the effective OTF. To obtain the effective PSF, the intensity profile of the TIRFM image of a 100 nm bead and the measured standing wave pattern were multiplied. The result of linear deconvolution applied to the SWM image (
SWM vas applied to image biological nanotubes, i.e., membrane tethers between living cells. The formation of such membrane tethers is a general phenomenon that occurs during cell adhesion, communication and spreading. Previously, direct measurement of tether diameters had not been possible with light microscopy, since they are considerably below the lateral resolution limit of conventional light microscopy. Scanning electron microscopy (SEM) measurements performed on fixed cells have suggested that tethers are 50-200 nm thick [Rustom, A., Saffrich, R., Markovich, I., Walther, P., and Gerdes, H. H., “Nanotubular highways for intercellular organelle transport,” Science 303(5660):1007-1010 (2004)].
An SWM setup was used to determine the diameter of membrane tethers that formed spontaneously between cultured human embryonic kidney (HEK) cells. These tethers stretch between interconnected cells and are up to several cell diameters in length. Since most of the tethers were not close to the culture substrate, TIR was not utilized. Cell membranes were stained with the fluorescent label Alexa Fluor 488 conjugated to wheat germ agglutinin, a lectin that binds to membranes. The label was added directly to the culture medium (50 μg/ml final concentration), cells were incubated for 15 minutes, then the unbound label was washed away. The Petri dish with labeled cells was placed onto the translational stage of the inverted microscope. The diameter of the tethers oriented about parallel to the SW fringes was determined by the FWHM of the intensity profile of the deconvolved SWM image.
Applicant believes that these were the first measurements of membrane tether diameters from living cells.
An image of the specimen illuminated by the SW was formed by utilizing the 2D SWFM illustrated in
Despite the use of large (10 mm) aperture AODs, the total time to change between the nine patterns was less than 100 μs. The acquisition speed was only limited by the time needed to collect a sufficient number of photons. When acquiring the preliminary data, the low sensitivity of the employed cooled CCD camera 304 resulted in a total acquisition time of 9 seconds. However, if a sensitive electron-multiplying camera is utilized, the total acquisition time can be about 100 times smaller, i.e., less than 100 msec. The SWFM reconstructed image was calculated using a linear algorithm similar to disclosed in Gustafsson 2000. From nine raw images, nine information components were calculated and shifted in real space by multiplying the images with the appropriate cosine functions. Then, all components were added together, divided by the effective 2D optical transfer function (OTF) in Fourier space, and transformed back to real space using only values within the bandwidth of the effective OTF. To calculate the effective OTF, the standard fluorescence image of a 100 nm bead as the PSF and the measured SW patterns were multiplied, and the product was then Fourier transformed.
The resolution of the 2D SWFM of
The results of imaging a different sample of 100 nm sub-resolution beads (using an 2D SWM and process similar to that described in Example 3A) is shown in
This procedure gave a measured 2D bead size of 102 nm (FWHM, full width at half maximum), which demonstrates the predicted resolution enhancement of 2D SWM, when compared to 270 nm measured with wide-field microscopy (WFM) (
The results of imaging a different sample of 100 nm sub-resolution beads (using an 2D SWM and process similar to that described in Example 3A) is shown in
While embodiments of the invention have been shown and described, modifications thereof can be made by one skilled in the art without departing from the spirit and teachings of the invention. The embodiments described and the examples provided herein are exemplary only, and are not intended to be limiting. Many variations and modifications of the invention disclosed herein are possible and are within the scope of the invention. Accordingly, the scope of protection is not limited by the description set out above, but is only limited by the claims which follow, that scope including all equivalents of the subject matter of the claims.
The disclosures of all patents, patent applications, and publications cited herein are hereby incorporated herein by reference in their entirety, to the extent that they provide exemplary, procedural, or other details supplementary to those set forth herein.
This application claims priority to: provisional U.S. Patent Application Ser. No. 61/021,755, filed on Jan. 17, 2008, entitled “Method and Apparatus for Enhanced Resolution Microscopy of Living Biological Nanostructures,” which provisional patent application is commonly assigned to the assignee of the present invention and is hereby incorporated herein by reference in its entirety for all purposes.
Number | Date | Country | |
---|---|---|---|
61021755 | Jan 2008 | US |