Patent Application
20200064616
The invention generally relates to super-resolution three-dimensional (3-D) microscopy. In particular, the invention relates devices and methods that improve the resolution of optical wavelength, 3-D microscopes.
In general, super-resolution microscopy can be achieved through a method that uses a special fluorescent dye. The dye is attached to features to be viewed under a super-resolution microscope. The dye is caused to fluoresce under illumination by light of a first wavelength, called the activation wavelength. The dye is also is inhibited from fluorescing with light of a second wavelength. By creating two, orthogonal sets of interference fringes with the inhibition wavelength, it is possible to pattern a uniform distribution of the activation light into a 2-D array of narrow probes that have dimensions well below the diffraction limit.
One problematic issue associated with this method involves the spacing between the probes. The spacing is limited by the Abbe criterion to the wavelength divided by two times the numerical aperture, and the effective size of the probes depends on the relative levels of the inhibition and excitation intensities as well as the characteristics of the two-color fluorescent material. Because the spacing between the probes is generally much larger than the size of the probes, a number of pictures, each containing a sparse array of pixels, and each corresponding to a slightly different position of the probes, must be combined to yield a complete picture. Because each picture is available electronically, it is relatively easy set a threshold which removes most of the noise between valid pixels.
Although the resultant picture has excellent resolution in the lateral dimensions (X & Y), the resolution in the normal direction (Z) is generally a large multiple of the lateral resolution. If the two interfering inhibition beams in both lateral planes have equal intensities, then the extent of the probes in the focus direction extend over the entire overlapped region. This makes it very difficult to get an accurate 3-D representation of a complicated 3-D object.
Many micro-organisms are quite complicated and have fine appendages that extend well beyond the main body. Very often, these appendages are well below the diffraction limit of visible microscopes, which currently represent the only means of viewing living micro-organisms. Super-resolution visible microscopes are available, which have sufficient resolution laterally, but they suffer from a very large depth-of-focus. As a result, the large depth-of-focus can superimpose the contributions from multiple parts of the micro-organism, and thereby produce an inaccurate 3-D model of the micro-organism.
What is needed is a microscope capable of imaging very thin slices of the micro-organism so that an accurate 3-D representation can be easily obtained.
By their very nature super-resolution microscopes obtain a picture in which the depth-of-focus of each pixel is very much longer than the lateral resolution, which makes it very difficult to accurately construct a 3-dimensional model of the object under observation. This invention addresses this issue by limiting the size of each pixel in the focus direction to a size comparable to the resolution in the lateral direction. This is done by generating a very thin, super-resolution slice in the specimen, normal to the microscope axis in which fluorescence is possible and by suppressing fluorescence elsewhere in the object. By taking a series of pictures, each corresponding to a different position of the activated slice in the object, it is possible to obtain an accurate 3-dimensional model of the object.
Thus, the invention provides an apparatus for obtaining a 3-D, super-resolution model of an object in an image space. The apparatus provides a super-resolution microscope having a microscope axis and focused on a 3-D specimen labeled with a fluorescent dye that is fluorescent when illuminated with an activation wavelength and inhibited from fluorescing when illuminated with an inhibition wavelength, the microscope employing a crossed grid of fringes generated from the inhibition wavelength to define a sparse array of super-resolution areas covering the specimen, with an optional activation source optionally used at normal incidence for 2-D imagery. The apparatus may also include a means for generating two or more fringe patterns of inhibition light arranged normal to a microscope axis and having slightly different spatial frequencies so when the patterns of inhibition light are overlapped on the specimen a narrow, uninhibited, super-resolution zone is generated normal to the microscope axis.
The means for generating two or more fringe patterns may do so normal to the microscope axis, by employing a modified Dyson relay, which images ±1 diffraction orders from a grating onto the specimen. This may be achieved by employing a block of glass containing a grating at one end, an opaque element in the block of glass to block the zero order from the laser illuminated grating, and polished sides to redirect the ±1 orders from the grating to an output end of the block, where the diffraction orders overlap, thereby generating inhibition light fringes throughout the volume of the specimen.
Where activation light is supplied by one or more laser beams, the beams are focused into a middle image area with an NA that is large enough to minimize widths of the beams in a Z-direction and small enough to keep beam sizes approximately equal over a lateral dimension of the image field.
Fringe patterns of inhibition light arranged normal to the microscope axis are typically generated by illuminating a grating with monochromatic, collimated light and causing the resultant ±1 diffraction orders to be overlapped in the image field. In such a case, a laser serves as a source of monochromatic, collimated light.
When a modified Dyson system is present, the system includes a plane parallel plate of glass having a higher index of refraction than the glass used for the Dyson lens.
In addition, fringe patterns may be generated by overlapping grating orders that are described by sine squared functions, and a frequency ratio defined between first and second frequencies is about 1:1.2, and a frequency ratio defined between first and third frequencies is about 1:1.0655.
In a particular embodiment of the invention, the specimen is positioned on top of a glass pedestal, which contains grating patterns responsible for generating the interference patterns projected at normal incidence to the microscope axis. In addition, the specimen and the grating patterns may be immersed in a liquid. The specimen may be positioned by moving the liquid in which the specimen is immersed.
The invention also provides for a system for obtaining a 3-D, super-resolution model of an object in an image space. The system comprises: an activation light source that employs conventional means to illuminate a thin slice through the object, that is perpendicular to an axis of a 2-D super-resolution microscope; two or more inhibition light fringe patterns that overlap each other and the thin slice of activation light to create an activation source having super-resolution dimensions and which serve to inhibit any activation light existing outside of the super-resolution dimensioned slice; a means for providing relative incremental movement between the object and the thin slice of constrained uninhibited activation light; and a means for taking a super resolution picture of each incremental slice resulting from the relative incremental movement between the object and the thin slice, thereby creating a set of voxels from which an accurate 3-D super-resolution model of any fluorescent object contained within the image space can be constructed.
The relative incremental movement providing means may move the object relative to the thin slice of constrained uninhibited light, e.g., via movement of the object via a liquid, i.e., a substantially incompressible fluid such as liquid water.
The invention may be used to observe a live or living specimen with an optical resolution of 200 nm or less. The specimen may include one or more fluorescent dyes.
In a further embodiment, a system for positioning and incrementally moving an object of interest for 3-D microscopy is provided, comprising: a 3-D field located in a liquid surrounding the object of interest; and a means for displacing liquid surrounding the object of interest in a manner that allows for positioning and incrementally moving the object of interest.
In still another embodiment, an optical system is provided for generating a very regular, high spatial frequency, fringe pattern over a large volume containing a liquid by imaging a grating, with an optical system, comprising a plate of high index material, a lens of low index material, and a spherical mirror.
Also envisioned is a microscope apparatus for creating a very thin slice of activation space spanning the lateral field of a 2-D super-resolution microscope, but having a focal depth equal to only one super-resolution element. The microscope apparatus includes a means for superimposing 2 or more inhibition interference patterns having slightly different spatial frequencies and fringes oriented normal to the microscope axis with a similarly-oriented, narrow-beam of activation light.
Before describing the present invention in detail, it is to be understood that the terminology used herein is for the purpose of describing particular embodiments only, and is not intended to be limiting.
In addition, as used in this specification and the appended claims, the singular article forms “a,” “an,” and “the” include both singular and plural referents unless the context clearly dictates otherwise. Thus, for example, reference to “a microscope” includes a plurality of microscopes as well as a single microscope, reference to “a pattern” includes a single pattern as well as a collection of patterns, and the like.
In this specification and in the claims that follow, reference will be made to a number of terms that shall be defined to have the following meanings, unless the context in which they are employed clearly indicates otherwise:
The term “NA” is used in its ordinary optics sense and refers to the numerical aperture of an optical system, i.e., a dimensionless number that characterizes the maximum angle over which the system can accept or emit light
“Optical,” “optically” and the like are used in their ordinary sense and refer to matters that relate to ultraviolet, visible and infrared parts of the electromagnetic spectrum. Typically, but not necessarily, the visible part of the electromagnetic spectrum is recognized as corresponding to a wavelength range of about 390 nm to about 700 nm.
“Optional” or “optionally” means that the subsequently described circumstance may or may not occur, so that the description includes instances where the circumstance occurs and instances where it does not.
Super-resolution as applied to the field of microscopy refers to achieving a resolution beyond the diffraction limit as defined by Ernst Abbe, who defined the smallest object that might be resolved by a microscope as being equal to the wavelength of light divided by 2 times the numerical aperture. Only fairly recently has a way been found around this fundamental limit. This involves staining the object of interest with a fluorescent dye that can be caused to fluoresce when exposed to one wavelength and caused to stop fluorescing when exposed to a second wavelength. By patterning the inhibition radiation so that fluorescence is possible only in a very sparse pattern of tiny areas and by taking many pictures each having a slightly different position on the object it is possible to construct a complete picture of the object having a resolution well beyond the Abbe limit.
Thus, “super-resolution” as applied to the field of microscopy refers to the ability of a microscope device, system, and/or apparatus to resolve object features smaller than the diffraction limit described by Ernst Abbe. Typically, the optical resolution of super-resolution systems is about 200 nm or less. In some cases, the resolution can be as small as 10 nm or less.
3-D Optical Microscopy
The invention, then, pertains to super-resolution 3-D optical microscopy. Such microscopy makes use of two-dimensional (2-D) pictures (in the X and Y directions) obtained using a high NA microscope objective, which images multiple pictures, each being a sparse array of regularly spaced pixels that constitute a small portion of the whole picture, Combining each of the sparse pictures into a single picture yields a super-resolution picture. The inhibition fringe pattern that defines the sparse array of super-resolution pixels can be projected through the microscope objective, or it can be incident on the specimen from the opposite direction. Each sparse picture corresponds to a slightly different lateral position of the inhibition array on the object. The inventive technique, to limit the resolution in the focus or Z-direction, is compatible with either configuration.
Whereas 2-D pictures can be obtained by illuminating the object with excitation radiation introduced along the Z-axis, in some situations the best arrangement for a 3-D picture is to illuminate the object normal to the Z-axis with a narrow sheet of excitation radiation, limited in width by the size of the object in the X and Y-directions.
The invention thus also provides a method for improving the resolution in the third dimension so that an accurate 3-D model of a complicated object can be obtained. For example, excitation illumination can be used to generate a relatively narrow band of light across a plane in the volume of interest and normal to the Z-axis, which is close to the diffraction limit in width, and this band can be narrowed further to super-resolution dimensions by also introducing inhibition fringe patterns running normal to the X-axis, which also serves to narrow the width of the excitation band. This allows each super-resolution, 2-D picture to be confined to a narrow zone through the volume of interest. Incrementally moving the object of interest with respect to the narrow band of illumination generates a series of super-resolution pictures, each picture corresponding to an incrementally different section through the object of interest. Thus, the entire volume of interest can be divided into voxels, which are approximately spherical in shape, and which provide a very accurate description of any 3-D object no matter how convoluted its shape happens to be.
The following is very simplified description of the physics underlying super-resolution. Assuming that a Watt of activation light at 405 nm is equally effective at stimulating fluorescence as a Watt of light at 532 nm at inhibiting fluorescence, then when the two intensities are equal, nothing should be stimulated to fluoresce. And the amount of fluorescence should be proportional to the intensity of the stimulated light minus the intensity of the inhibition light. The result is illustrated in
This is only a first order approximation to what really happens—a better approximation would involve molecular cross-sections and time constants, which can vary the inhibition to excitation ratio in either direction. Very little data on molecular cross-sections and time constants is currently available, but current research programs seem likely to change this situation in the near term.
It is possible to increase resolution by decreasing the size of the small holes where fluorescence is possible, by increasing the ratio of the inhibition intensity to the activation intensity. This is illustrated in
To a first approximation the width of a super-resolution probe can be predicted using the following equation:
Probe Width=((0.2 microns)(Intensity of inhibition/Intensity of excitation))0.5 (1)
Other assumptions built into this equation are an excitation wavelength of 405 nm, an inhibition wavelength of 532 nm, and an NA of 0.858. However, the general principle of this equation should also hold for other excitation wavelengths, other inhibition wavelengths, and other NA as well.
Shaping the Activation Illumination
Generally, it is desirable to constrain the excitation illumination to a narrow band passing through the volume of interest, and normal to the Z-axis, so that there is no possibility of fluorescent material located outside of the limited inhibition zone contributing to the picture.
The volume of interest is defined as a rectangular area having a given length and width, in the X and Y axis, and which is normal to a main axis, Z, that contains the optical axis of a microscope that views the orthogonal fringe patterns superimposed on the rectangular area. The third dimension, Z, will be called the depth or focus dimension, and it is normal to this dimension that the narrow slice of uninhibited volume is formed by focusing a narrow beam of activating light originating from one or more objectives, each having an axis at right angles to the main Z-axis. The output from the objective supplying excitation illumination can be focused into a small beam roughly λ/2NA in width and λ/NA2 in length, where λ is the wavelength and NA the numerical aperture of the beam. The NA can be chosen so that the length of the beam remaining in good focus is approximately equal to the length of the volume of interest and the beam can be broadened to span the width of the volume of interest, possibly by introducing a little astigmatism. This arrangement provides relatively good illumination uniformity at a given depth and over the length and width of the volume of interest.
However, the dimension of the beam in the depth direction is roughly λ/2NA. This dimension may be narrowed to the size of the super-resolution elements obtained in the other dimensions.
A possible arrangement showing one of the many possible relative positions of the microscope axis and the axis of the relays that serve to narrow the activation plane in the Z-direction is shown in
Narrowing the Activation Zone
There are at least two rather different methods for generating a periodic fringe pattern. Both involve superimposing collimated beams having equal and opposite incidence angles. Although it is not mandatory, both methods may involve using a laser source, because of the much higher brightness of laser sources. In both cases a single-mode, single-wavelength laser is preferred because this generally yields a greater depth of focus and sharper fringes. A single-mode, single-wavelength laser is preferred because there are inevitably appreciable path differences between the interfering beams in the final image and a single mode laser increases the depth-of-focus. The two collimated beams interfere wherever they overlap thereby creating a very high contrast pattern of horizontal fringes. Because there is no zero-order present in the grating image, the resulting intensity profile follows a sine-squared function having a period that is half that of the object grating.
The interference pattern contrast depends upon the intensity of the interfering beams being equal. Since the beams overlap over a considerable area and are incident with a large angle between them, the only way to achieve high contrast is to make the intensity profile of the beams flat and constant over the desired interference volume. Typically, a single mode laser beam has a Gaussian profile, which is difficult to efficiently shape into a beam with a flat intensity profile over an extended region.
However, there are optical systems designed to broaden the center portion and tuck in the periphery so good uniformity can be achieved over a considerable area, albeit with some loss of light.
In this special case the depth-of-focus doesn't follow the usual λ/NA2 relationship because the zero order is missing.
Narrowing the excitation zone is done by projecting two or more fringe patterns formed using inhibition light through the volume of interest and at right angles to the main axis.
The fringe patterns have different spatial frequencies and combine to generate a thin slice, normal to the Z-axis, where the inhibition intensity projected through the volume of interest reaches zero and an extended zone either side of this slice where the inhibition intensity is sufficient to squelch any fluorescence. The width of the extended zone is ideally made to exceed the dimension of the excitation beam in the depth direction, thus ensuring that extraneous dyed material outside the volume of interest will not contribute to the desired picture.
One possibility for generating a fringe pattern is shown in
An alternate way of generating the horizontal interference patterns in illustrated in
The system shown in
In
The parameters of the design shown in
The radius, thickness, and diameter values in the table are in millimeters. Suprasil is a commercially-available, high-quality, fused silica glass. Note that this entire optical system would fit comfortably in a one-inch cube.
A magnified picture of the region around the object and image planes is shown in
NA=1.335 sin(40°)=0.858
A further enlargement of the water cavity and its surrounds is shown in
A cross-section through the centers of the three relays is shown in
Excitation Illumination
One possibility for a source for the excitation illumination is a laser that may have multiple longitudinal modes resulting in several wavelengths over a narrow spectral band. An example might be a high power GaN laser, which operates typically in a band about 2-3 nanometers wide in the 403 nm region, and which can have an output power of 0.5 to 1.5 Watts. The Nichia NDB711E Blue Laser Diode has an output beam with a 10° angular spread in one plane and a 25° angular spread in the other plane. Assuming the 10° angular spread is primarily due to diffraction, the diode aperture size is given by:
Aperture size=λ/2NA=0.403 μm/2 sin(5°)=2.31 μm
Excitation Illumination Example:
In this example, an activation wavelength of 405 nm is used. The activation beam is associated with a NA2 equal to λ/DoF (0.403 microns/62 microns=0.0065). Accordingly, the NA is equal to 0.0806.
The size in Z direction is equal to λ/2NA, which translates to ±1.25 microns.
A beam this size could be obtained by simply reimaging the Nichia Diode beam with a magnification of 2.5/2.31, or 1.082. This would increase the 2.31 μm size calculated at the diode to 2.5 μm. Spanning the 62 μm width of the sample could be done simply by introducing some astigmatism in the optical system that reimages the diode onto the sample volume. In fact, as many diodes as needed could be imaged onto the sample volume since the required NA of the activation imaging system is quite modest and the sample can be illuminated from numerous points in the X-Y plane spread across the center plane of the sample volume.
Inhibition Illumination
The volume of interest is really determined by the super-resolution target and the number of pixels desired in the final composite picture. In this example, the super-resolution goal is 1/10 the size of the inhibition fringe spacing (0.310 microns/10=0.031 microns) and the resultant picture size is 2000 by 2000 pixels, which is about all the pixels that can be shown on a high-resolution monitor.
Inhibition Fringe Example
In this example, an inhibition wavelength of 532 nm is used. The index of water at 532 nm is 1.335. The NA is equal to 1.335 sin(40°, which equals 0.858. The fringe spacing is equal to λ/2NA (0.532/(2×0.858) or 0.310 microns). The volume of interest is 62 by 62 by 62 microns, which is equal to 200 by 200 by 200 fringes, which is equal to 500 by 500 camera pixels by 200 pictures (considerable oversampling in the X-Y plane).
Increasing the resolution by a factor of 10 requires that the inhibition intensity is substantially higher than the activation intensity. The ratio, R, of the two intensities is very approximately given by equation (1).
Oversampling each individual sparse picture on the microscope detector by using a high microscope magnification ratio and a camera detector array that has more pixels than is warranted by the information content in the picture, eliminates any need to reposition the image on the camera as the fringe pattern is moved. The raw data from the camera can be used to locate the position of each picture element defined by the black holes in the inhibition interference pattern, and it is possible to predict the peak intensity of the fluorescence from each black hole incident on the camera detector array, which is typically spread over several detectors and the gaps between them.
Turning Off Every Z-Slice but One
In the general case where a substantial gain in resolution is required, the relative intensities of the inhibition wavelength to the activation wavelength is weighted strongly in favor of the inhibition wavelength so that only the activation energy in the very bottom of the black hole between the crossed inhibition fringes is effective in image formation. For example, assuming that the activation and inhibition wavelengths are equally effective in activating or deactivating the fluorescent molecules, then a gain of 10 in resolution would require that the peak inhibition intensity is 41.6 times higher than the excitation intensity.
If it is imagined that the volume of interest is divided into a number of equally thick slices normal to the Z-axis, then the X-Y picture details from a single slice can be obtained by arranging it so all the other slices have an inhibition intensity above some safe level such as 5% of the peak inhibition intensity. This can be done by isolating the activation energy to as few slices as diffraction considerations will allow, and then trimming it with multiple inhibition fringe patterns of different spatial frequencies, which inhibit every part of the activation beam with the exception of the one slice from which information is desired.
An example of the intensity profile possible by combining 2 sets of fringes is shown in
An intensity profile obtained by adding the intensities of 3 different sets of fringes, each with a different spacing, is shown in
I=((sin(10.1334x))2+(sin(12.16x))2+(sin(10.797x))2)/2.940527
where x is the lateral position and all angles are in radians.
As before, the first coefficient of x, k1=10.1334, is determined by the available NA and the inhibition wavelength:
k
1=2πNA/λ=2π(0.858)/(0.532)=10.1334
As before, the first coefficient of x is determined by the NA and the inhibition wavelength. There is some overlap in the effects generated by the other coefficients, but the second coefficient is chiefly responsible to the height of the first minimum past zero and this is 0.057 at x=0.283. The third coefficient is chiefly responsible for the location of the first minimum that falls below the 5% level. In this case the minimum is located at 4.65 microns where the fluorescence stimulating intensity is less than 0.001 of the peak intensity. Thus, all the requirements for generating an isolated, narrow uninhibited zone passing through the middle of the specimen can be achieved with three sets of fringes in this case.
If the desired picture volume had been one-quarter of the size, then the NA of the excitation beam would be twice as large, making the beam waist half as big, and the possibility of using only two sets of fringes quite viable.
Scanning the Object Through the Activation Zone
Since the object may be immersed in liquid, one way of moving the object vertically through the narrow zone of uninhibited activation light is to add liquid at one end of the column and remove it from the other. It should also be possible to laterally shift the object laterally in the column by adding and removing liquid from the column at positions level with the object being studied. There is lots of available room near the mid-plane of the object because the NA of the beams in this direction are quite small.
It is to be understood that, while the invention has been described in conjunction with the preferred specific embodiments thereof, the foregoing description merely illustrates and does not limit the scope of the invention.
In addition, numerous alternatives and equivalents exist which do not depart from the invention set forth above. For example, the inventive apparatus may be constructed to contain or exclude specific features and components according to the intended use of the apparatus, and any particular embodiment of the invention, e.g., those depicted in any drawing herein, may be modified to include or exclude element of other embodiments. Alternatively, stated, different features of the invention described above may be combined in different ways. Other aspects, advantages, and modifications within the scope of the invention will be apparent to those skilled in the art to which the invention pertains.
All patent applications, patents, and publications mentioned herein are incorporated by reference to an extent not inconsistent with the above disclosure.
This application claims priority to U.S. Provisional Patent Application No. 62/720,749, entitled “Super-resolution, 3D Microscope,” by David Markle, filed Aug. 21, 2018, and is related to U.S. patent application Ser. No. 13/871,031, entitled “Apparatus and methods for microscopy having resolution beyond the Abbe limit,” filed Apr. 26, 2013, which has matured into U.S. Pat. No. 9,075,013 to Markle et al., the disclosures of which is hereby incorporated by reference in their entireties.
| Number | Date | Country | |
|---|---|---|---|
| 62720749 | Aug 2018 | US |
