Patent Grant
11598630
The present invention relates to a method and an optical measuring device. It finds applications in particular in microscopy, for example in the field of biology and the acquisition of biological information from optical observation.
A microscope is an optical instrument generally used to view, analyze or measure objects too small for the naked eye.
We use the term biological to describe any biological entity in life sciences, regardless of its origin, human, animal or vegetal and of the purpose of the observation, be it for research, diagnostic or therapeutic application. This term includes the medical uses of the method described. Microscopy is used in the field of biology, for example, to observe, study and measure biological entities (objects) and their dynamics.
The usual definitions are used for: optical diffraction limit, Rayleigh criterion, Airy disk and its radius and diameter. We use in the context of the invention, the terms of superresolution, superresolved, superresolution imaging and superresolution microscopy to describe optical data acquisition, optical imaging and microscopy at a resolution higher than the optical diffraction limit. The usual definitions are used for fluorescence and for fluorophores.
Referring now to
Microscopy comprises the illumination, by a light source, not shown, using a microscope, 10, of a biological sample, 11, and the time-dependent measurement, using either visual observation or a detection module 12, of the light emitted by the sample.
The sample in Biology comprises a single—or a plurality—of different biological entities, 13 and 14, positioned at different positions.
Examples of such objects are, among others, a cell, a virus, a protein and a DNA fragment.
Fluorescence microscopy is one of the variants of microscopy, it has replaced in many biological applications, the other microscopy techniques.
A fluorescence microscope is an optical microscope used to study properties of organic or inorganic substances using the phenomena of fluorescence instead of, or in addition to other modalities such as reflection and absorption.
We refer again to
The sample is illuminated by light of wavelength, or specific wavelengths, which is absorbed by the fluorophore, thereby inducing the emission of light at different, higher, wavelengths.
The illumination light is separated from the emitted fluorescence, which is lower, by the use of a spectral emission filter.
Fluorophores have become an important tool for the visualization of biological objects. The activity and the biological information including details above the limit of resolution of 200 nm are systematically viewed and measured using fluorescence microscopy. This resolution limit is derived from the Rayleigh criterion, which in the best case, reaches 200 nm in systems designed specifically. For a long time, until the emergence of superresolution techniques described below, it was assumed that optical techniques, including fluorescence microscopy, are unable to visualize details smaller than the Rayleigh criterion, which is about 200 nm.
However, other fundamental biological activities also occur at scales smaller than 200 nm in biological samples. At this level of spatial resolution, important phenomena can be observed: the biological processes at the scale of intracellular, cell information transfer, the folding and unfolding of the proteins and changes in the DNA and RNA. For example, the measurement of this intracellular information open new avenues for understanding the biological activity, and lead to progress in understanding and monitoring of research and medical diagnostics.
The main implementations of fluorescence microscopy, as described in detail in the literature, are the confocal microscope, often used in a scanning configuration or spinning disc microscope, and the wide-field imaging microscope.
Referring now to
A confocal fluorescence microscope,
a light source, 20,
an optomechanical frame not shown,
a cube filter, 21,
a microscope objective 22,
a detector assembly, 23, and
a processing unit, not shown.
The light source 20, which may be an arc lamp or a laser, creates light energy necessary for fluorescence.
The Optomechanical frame, not shown, is the support of all the optical components and auxiliary optics and includes alignment capacities.
It also includes optical elements, not shown, capable of shaping the beam to allow its focus point of a minimum size by means of the microscope objective.
It can also comprise, in a confocal scanning fluorescence, a spatial or angular scanning mechanism, not shown, to change the position of the point source with respect to the object to be measured.
The scanning mechanism can alternatively
In a confocal scanning fluorescence, the information is collected point by point, using the scanning mechanism.
It can also comprise, in a rotating disk type confocal fluorescence, a rotating disc having a plurality of pinholes, allowing the simultaneous projection of a plurality of points. In a confocal fluorescence rotating disk, a set of points, corresponding to the pinhole is acquired at any time and the rotation of the disk allows to scan the entire surface of the sample for a given longitudinal position.
The cube of filters, 21, channels the different optical signals and avoids contamination of the fluorescence signal by the emission. The cube is composed of filters: excitation filter, 210 dichroic mirror, 211, and emission filter 212. The filters and the dichroic mirror are selected according to the wavelength of excitation and emission spectral characteristics of the fluorophore.
The microscope objective 22 focuses the light created by the source in the focal plane of the lens 24, a light distribution pattern of small size, the optimum light distribution consisting of the Airy disk. The microscope objective 22, also collects back fluorescent light emitted by the fluorophores.
For a confocal scanning fluorescence the system can be descanned, that is to say, the return light can pass through the scanning mechanism to compensate for the translation due to scanning.
A detector lens, 25, creates, in the image plane of the detector 26, a magnified image of the focal plane of the lens 24.
A confocal hole, 28, is theoretically placed in the image plane of the detector 26. In most practical systems, the confocal hole, 28, is placed in an intermediate imaging plane, not shown, and reimaged onto the image plane of the detector 26.
The assembly of the detector, 23, detects the fluorescent intensity in the overall illuminated volume, and converts it into digital signal. For a confocal scanning microscope, the detector assembly comprises a detector of a single element, such as a PMT or SPAD. For a confocal microscope using a rotary disc, the detector assembly is comprised of a matrix of detector elements, such as a CCD, a EMCCD, a CMOS or a matrix of SPAD.
All components mounted from the light source to the dichroic filter is the illumination path, 201. The detection channel, 202, represents all the components mounted from the dichroic filter to the assembly of the detector.
The elementary optical process of a confocal microscope can be segmented into six steps:
Projecting light on the volume analyzed
Display of the measured intensity as a pixel value in an image
Fluorescence microscopes are available from several manufacturers, such as Nikon, Zeiss, Leica and Olympus. Fluorescence microscopes can be either standard microscopes suitable for fluorescence or microscopes optimized specifically for fluorescence. Modern microscopes are versatile instruments capable of operating in many different modalities, including, but not limited to, fluorescence modalities, using the same platform and most optomechanical components. Most fluorescence microscopes are developed as an open platform, capable of performing several additional features with minimal modifications. Other fluorescence microscopes are instruments dedicated, adapted for a specific task, such as medical diagnosis or pharmaceuticals.
New optical methods, methods for superresolution are capable of discriminating fluorophores, below the Rayleigh criterion. These methods are being developed by several companies, laboratories and researchers and some of the instruments using these methods, the superresolution microscopes, are commercially available. Several comparative analysis of superresolution methods have recently been published in the literature, as the article written by Ricardo Henriques and Mr Musa Mhlanga, entitled “PALM and STORM: What hides beyond the Rayleigh limit?”, or Article written by Kelly Rae Chi called “Super resolution microscopy: breaking the limits.”
New optical methods, methods for superresolution are capable of discriminating fluorophores, below the Rayleigh criterion. These methods are being developed by several companies, laboratories and researchers and some of the instruments using these methods, the superresolution microscopes, are commercially available. Several comparative analysis of superresolution methods have recently been published in the literature, such as the article written by Ricardo Henriques and Mr. Musa Mhlanga (“PALM and STORM: What hides beyond the Rayleigh limit?”, Biotechnology Journal, 4, 846-857 (2009)), or the article written by Kelly Rae Chi (“Super resolution microscopy: breaking the limits”, Nature Methods, 6, 15-18 (2008)).
An updated bibliography on the superresolution is on the website of the company Zeiss Co. (“Zeiss Microscopy and image analysis”, (2011), retrieved at http://www.zeiss.eom/4125681C00466C26/7Qpen) and on the website of the company Nikon Co. (“MicroscopyU: the source for Microscopy Education” (2011) retrieved at http://www.microscopvu.com/) (hereinafter, “Nikon (2011)”.
New superresolution techniques allow to obtain information beyond the resolution limit. The main problem of all existing superresolution techniques is that the envelope of performance, expressed in terms of lateral resolution of longitudinal resolution, speed, light intensity necessary for phototoxicity in the biological object, of ability to measure different objects, is very limited.
In addition, most of the methods and instruments can provide superresolution either a good lateral resolution or a good longitudinal resolution, but rarely both.
In addition, all these instruments are complex and require a highly skilled operator.
In addition, these instruments can generally observe a small part of biological specimens due to strong operational limitations, such as, for some of them, a shallow depth of field or a requirement of very high light intensities, harmful to cells.
Another problem with the methods and instruments of super resolution, is that most of them are able to recover in the illuminated volume, the attributes of a single fluorophore, but fail to recognize the presence of simultaneously several fluorophores and measuring their attributes.
An additional problem with the methods and instruments of superresolution is that these methods and instruments are presented to users and perceived by them as a general tool, able to replace the standard or confocal microscopes. However, the methods and instruments superresolution lack the simplicity, robustness, ease of use and competitive prices of standard microscopes which hinders their use as research tools or as general diagnostic tools.
Another problem with existing superresolution methods and tools is that most of these methods and tools are designed as stand-alone instruments designed to replace standard microscopes. Such an approach requires the replacement of existing instruments and the renewal of all systems and devices all the knowledge and know-how related to microscopy platforms and developed over many years.
Another problem with most methods and instruments fluorescence microscopy and superresolution is that these methods and tools are designed on a paradigm of image acquisition, the entity for which basic information is—or more images, or—or more—ROI regions—Region Of Interest bi- or three-dimensional. Algorithmic, systemic and superresolution methods described later in the context of the invention will, by their inherent flexibility, the development of new strategies of acquisition. These acquisition procedures, dynamic and selective, will be defined by an optimized sequence acquisition and interactive and deferred processing. They allow a more sophisticated optimization of the useful information, as defined by criteria based on the shape, geometry and dynamics of one or more fluorescent objects, separately or relative one to the other.
So there is still an urgent need to provide superresolution methods and tools and algorithms methods capable of measuring with high accuracy the attributes of a fluorophore. It is also necessary to provide methods and tools to detect and quantify the presence of multiple fluorophores placed in the same volume illuminated.
A goal of at least one embodiment of the present invention is to provide a technique for superresolution fluorescence microscopy in Biology and more generally to life sciences, and additionally to pharmacology, medicine and diagnostics, that will overcome the shortcomings of the prior art devices.
One of the goals of at least one embodiment of the present invention is to provide a technique for superresolution fluorescence microscopy in biology to achieve an optical system that is capable of measuring with high accuracy the attributes of a fluorophore and recognizing and measuring the attributes of multiple fluorophores located in the same illuminated volume.
Another goal of at least one embodiment of the invention is to provide a technique for superresolution fluorescence microscopy in biology to measure with high precision the attributes of a fluorophore.
Another goal of at least one embodiment of the invention is to provide a technique for superresolution fluorescence microscopy in biology that acquires and measure with great precision, the attributes of multiple fluorophores present in the same illuminated volume.
To this end, a first aspect of the invention provides a method of optical measurement to determine the spatial position of at least one light nanoemitter on a sample, the method comprising:
The detection may comprise the detection of the reflected light at a mean wavelength λ. The reflected light can be collected by a high numerical aperture objective. Both distributions can be compact.
The plurality of distributions may be generated sequentially in time, or created simultaneously.
According to an embodiment the two compact light distributions are collocated on the sample.
According to one embodiment said at least two compact light distributions of different topological families are created by interference between a regular wave and a singular wave, or between two singular waves, and spatial differentiation between said at least two distributions is created by varying at least one of the following parameters:
The various parameters may include amplitude, phase, polarization, coherence, for example.
According to one embodiment of the process included the creation from an incident light wave, two light collocated waves one regular and one singular.
According to one embodiment the method further includes the separation of an incident regular wave, into two regular waves following separate geometric paths; transforming, at least one of the regular optical waves in a singular optical wave and fusion of the two created emerging optical waves.
According to one embodiment the method further includes at least one of the following steps:
According to one embodiment, the process further included formatting, statically or dynamically, the emerging polarization of the said superimposed light distributions, able to mitigate vector effects, on the shape and size of said compact light distributions, effects created by a high numerical aperture lens used for generating optical images, by shaping the emerging polarization by providing a static, rotationally symmetrical, polarization state such that a circular polarization, radial or azimuthal and/or a dynamic polarization state.
According to one embodiment said at least two light distributions are created by controlling the intensity of the different modes of a multimode laser.
According to one embodiment, a region of a size substantially less than a mean wavelength λ of the reflected light exists in the sample, wherein the value of a specific mathematical combination of the intensities of said at least compact two light distributions of different topological families is positive for lateral positions of said light distribution portion included in said region, and wherein said specific combination mathematical approaches zero in all other parts of the light distribution beyond said region.
According to one embodiment at least one nanoemitter is a fluorophore with a sequence of fluorescence light intensities which depends of the incident intensity of the sequence of compact light distributions of different topological families on said fluorophore thus characterizing the spatial position of said fluorophore.
According to another embodiment at least two of the nanoemitters are fluorophores, located at different spatial positions, each fluorophore emitting light with an intensity depending on the incident intensity of the sequence of compact light distributions of different topological families on the said at least two fluorophores thereby characterizing the spatial position of at least two fluorophores.
According to one embodiment a region whose size is substantially smaller than the wavelength λ of the reflected light, exists on the sample, wherein a comparison of a plurality of mathematical combinations of the said sequence of compact light distributions of different topological families, can differentiate between at least one of the following:
According to one embodiment the method further comprises varying the sequence of said at least two light distributions and/or the position of the sequence of said at least two light distributions as a function of measured data or external information.
According to one embodiment, the projection of light distributions of different topologies is created by the conical diffraction and modified by a variation of the polarization states of input and output of at least one crystal creating the conical diffraction effect.
According to one embodiment, the spatial position of at least one measured light nanoemitter is the lateral position of said at least one light nanoemitter.
A second aspect of the invention provides a method of optical measurement to determine the spatial position of a plurality of light point-sources the method comprising detecting light emitted by the plurality of light point-sources; and the separation of the light emitted on a plurality of sensors for simultaneous or sequential detection; and the proportion of the light emitted by a light point-source, channeled to a specific detector, being dependent on the spatial position of said light point-source; and the generation of the optical images from the detected light; and the algorithmic analysis of optical images to obtain a location information of the plurality of light point-sources.
According to one embodiment, each light point-source corresponds to one or more of the several nanoemitters and the method includes a preliminary step comprising the illumination by incident light of the nanoemitters of the sample for detecting the light remitted by nanoemitters for locating the nanoemitters.
According to an embodiment the spatial position measured is the longitudinal position of each light point-source or light nanoemitter.
According to one embodiment, the emitted light is separated according to the longitudinal position of each light point-source or nanoemitter or according to the wavelength emitted by each light point-source or nanoemitter.
According to an embodiment the separation of the emitted light is performed so as to separate, to a plurality of detection channels, the collimated emitted light, emerging from the light sources positioned at the focal plane of the lens, from the non-collimated emitted light emerging from point-sources lying above or beyond the focal plane.
According to one embodiment the method further comprises the parameter variation of the detection channels based on measured data or external information.
According to one embodiment the method further comprises at least one of the following steps:
According to one embodiment the method further comprises the separation of the light intensities of the plurality of re-emitted light sources into a plurality of independent channels with orthogonal polarizations, by using a polarization beam splitter, and merging the light intensities emerging from each channel, so that the length of the longitudinal position of the polarization or the pipe geometry of the light intensity in the light is maintained by the merger.
According to one embodiment the method further comprises focusing the light point sources positioned at a given longitudinal position, and the defocusing of the light sources positioned before and after said longitudinal position, the light distribution different from the focus in dependence on the longitudinal position of each point source.
According to one embodiment the method further comprises generating, from the incident light wave, two light waves of different polarizations, the ratio of energy between the waves of different polarizations being a function of the spatial distribution at the point of focusing and through it of the longitudinal position of the light point-source, and the separation of the regular and singular waves on the basis of its state of polarization or of its geometric shape.
According to one embodiment the detection of the light emitted or re-emitted is limited in the focal plane of the microscope objective. according to one embodiment, said at least optical image is generated by using the optical microscope, as for example a confocal microscope
A third aspect of the invention provides an optical measuring device for determining the spatial position of at least one light nanoemitter positioned on a sample, the device comprising:
According to an embodiment of the means of projections are configured to colocate both compact light distributions on the sample.
According to one embodiment, the projecting means is adapted to create said at least two light distributions by interference between a regular wave and a singular wave or between two singular waves, and to create a spatial differentiation between said at least two distributions by varying at least one of the following parameters:
According to one embodiment, the device comprises a crystalline sub-module to create from an incident light wave, two collocated light waves one regular and one singular said sub-module lens comprising a thin biaxial crystal and/or a uniaxial crystal.
According to one embodiment the device comprises an optical sub-module adapted to separate a regular incident wave, in two regular waves along separate geometric paths, the sub module being configured to transform at least one of the regular optical waves in an singular optical wave, using processing means comprising at least: a subwavelength grating, the grating step being smaller than the mean wavelength of the reflected light, and a thin biaxial or uniaxial crystal, and wherein the optical module is designed as to combine the two created emerging optical waves.
According to one embodiment the device comprises a polarizer part adapted to control the relative amplitude of regular and singular waves and optionally translating the central position of the light distribution of a wave in relation to the other.
According to one embodiment the device comprises an optical control sub-module of amplitude polarization or phase, comprising at least one controllable or adjustable optical element capable of controlling, in a predetermined sequence, the state of polarization and/or phase of the light input or output wave of the sub-module lens.
According to one embodiment the device comprises a control sub-module, consisting of at least one adjustable optical element capable of controlling the shape of regular or singular waves.
According to one embodiment, the device comprises a polarization analyzer and/or a, static or dynamic, sub-module shaping the emerging polarization of said light distributions, able to mitigate the vector effects on the shape and size of each compact light distribution, effects created by a high numerical aperture of the lens used for generating optical images, shaping the emerging polarization by providing a static polarization state, rotationally symmetrical, such that the circular, radial or azimuthal polarizations and/or a dynamic state of polarization.
In another embodiment, the device comprises a multimode laser, in which the intensity of the laser modes are controllable and at least two compact light distributions of different topological families are created by controlling the intensity of the different modes of laser.
According to one embodiment, a region of a size substantially smaller than the mean wavelength λ of the reflected light exists in the sample, wherein the value of a specific mathematical combination of the intensities of at least two compact light distributions of different topological families created by the illumination means is positive for lateral positions included in said region, and wherein said specific combination mathematical approaches zero in all other parts of the light distribution beyond said region.
According to one embodiment at least one is a nanoemitter is a fluorophore with a sequence of fluorescence light intensities which depends on the sequence of incident compact light distributions of different topological families on said fluorophore thus characterizing the spatial position of said fluorophore.
According to one embodiment at least two of the nanoemitters are fluorophores, located at different spatial positions, each one of the fluorophore with a sequence of light fluorescence intensities which depends on the sequence of incident compact light distributions of different topological families on the at least two fluorophores thereby characterizing the spatial position of the at least two fluorophores.
According to one embodiment a region whose size is substantially smaller than the wavelength λ of the emitted light, exists in the sample, and further comprising a comparator for comparing a plurality of mathematical combinations of said sequence of compact light distributions of different topological families, for differentiating between at least one of the following:
According to one embodiment, the projecting means is configured to vary the sequence of said at least two light distribution and/or the position of the sequence of said at least two light distributions as a function of measured data or external information.
According to one embodiment, the projecting means comprises at least one conical crystal to perform conical diffraction, and means for varying the polarization states of input and output of said at least one conical crystal.
According to an embodiment configured to measure the lateral position of said at least one light nanoemitter.
According to a fourth aspect, the invention provides an optical measuring device for determining the spatial position of a plurality of light point-sources, the device comprising:
According to one embodiment the device is configured to measure the longitudinal position of each light point-source.
According to one embodiment a point-source comprises one or more nanoemitters of the sample and the device comprises illumination means adapted to illuminate the nanoemitters of the sample by incident light, and detecting means adapted to detect the light remitted by nanoemitters for locating the nanoemitters.
For example, a fluorophore can be a nanoemitter.
In one embodiment the separating means are configured to separate the reflected light as a function of the longitudinal position of each light point-source or as a function of the wavelength emitted by each light point-source.
In one embodiment the separating means are configured for separating into a plurality of detection channels, the reflected light collimated emerging from the light point-sources, positioned at the focal plane of the lens, from the remitted non-collimated light emerging from the light point-sources located before or beyond the focal plane.
According to one embodiment the device further comprises variation means to vary the parameters of the detection channels based on measured data or on external information.
According to one embodiment the device further comprises channeling means able to channel the light intensities from the plurality of point sources of light emitted, placed in a small light volume on separate detectors, and/or at separate geometric positions on the same detector, depending on the longitudinal position of each point source.
According to one embodiment the device further comprises a sub-module that interfaces to a microscope objective and to a sub-detection unit and one to the other by auxiliary optics.
According to one embodiment the device comprises optical means adapted to create, from each point source of the light emitted, a beam of quasi parallel light, the light beam differing from the parallel light by an angle of convergence or divergence, and the value of the angle being a function of the longitudinal position of each light point-source.
According to one embodiment the device comprises a crystalline sub-module, or a cascade of crystalline sub-modules, each sub-crystalline module consisting of a biaxial crystal and/or a uniaxial crystal, and auxiliary optics, the aforesaid sub-crystalline module being adapted to create from the incident collocated light wave two light waves of orthogonal polarizations and different geometries, one regular and one singular, the ratio of energy between the regular and singular waves being function of the angle of convergence or divergence of the beam and through him the longitudinal position of the point source emitter.
According to one embodiment the device comprises means for changing the polarization or geometry suitable for separating the regular and singular waves based on a polarization state thereof, or a geometric shape thereof.
According to one embodiment, the device includes
According to one embodiment the device comprises optical means able to focus light point sources of remitted light, positioned at a given longitudinal position, and, to slightly defocus the light sources of the reemitted light, positioned before and after said longitudinal position, the light distribution differing from the focusing in dependence of the longitudinal position of each point source.
According to one embodiment the device comprises means for spatially varying polarization and having at least a uniaxial crystal of variable thickness, a subwavelength grating with step below the mean wavelength of the reflected light and/or a phase waveplate, creating from the incident light wave two light waves of different polarizations, the ratio between the energy of the waves of different polarizations being a function of the spatial distribution at the focusing point and through it of the longitudinal position of the point source emitter.
According to one embodiment, the device comprises polarization or geometry means capable of separating the regular and singular waves on the basis of their polarization states or on their geometric shapes.
In a fifth aspect, the invention provides an optical measurement to determine the spatial position of at least one light nanoemitter on a sample comprising a microscope, for example, a confocal microscope, a measuring device according to one embodiment of the third aspect of the invention, and/or a measuring device according to one embodiment of the fourth aspect of the invention.
According to a sixth aspect, the invention provides a method of optical measurement to determine the spatial position of at least one light nanoemitter on a sample comprising a measuring method according to an embodiment of the first aspect of the invention to determine the lateral position of at least one nanoemitter in a sample, and a measuring method according to an embodiment of the second aspect of the invention for determining the position of at least one longitudinal nanoemitter on a sample.
In another aspect, the invention provides a method of optical measurement consisting of a sequential projection of at least two compact light distributions of topological families on a sample.
In another aspect, the invention provides an apparatus for performing optical measurements with a projector for sequential projecting at least two compact light distributions of different topological families on a sample.
In another aspect, the invention provides an optical system configured to create in sequence, on a sample, at least two light distributions spatially separated from each other, each of said distributions having a diameter less than 1, 5 time the mean wavelength of the reemitted light from the sample, such as combinations of intensities of said distributions creating localized characteristics of a size less than 0.5 said mean wavelength.
In another aspect, the invention provides an optical system including an optical apparatus and a illuminated measurement region, the optical apparatus being configured to sequentially create at least two light distribution spatially separated from each other, and defining a set of points in the measurement region comprising all the points of zero intensity one one or another of the distributions, a point located in the region close to a measurement point of said assembly being less than or equal to 0, 5 of the radius of the measuring region.
In another aspect, the invention provides an optical system comprising an optical device and a measurement region to be illuminated, the optical device being configured to sequentially create at least two light distributions spatially differentiated from each other and defining a set of points in the illuminated measurement region, comprising all points of zero intensity and local intensity maximum in one or other of the distributions, a point located near a point of said assembly being less than or equal to ⅙ of the diameter of the measuring region.
In another aspect, the invention provides an optical apparatus, and a compact measuring region having a line passing through the center of the measurement region, the optical apparatus being configured to create a light distribution spatially differentiated, the ratio energy between the intensity along the line passing through a maximum, a minimum and another maximum.
In another aspect, the invention provides a lateral superresolution module comprising an optical device adapted to sequentially create at least two light spots spatially differentiated with a diameter less than 15 times the emitted average wavelength λ, such as combinations of the intensities of said spots create localized details of a size less than 40% of the wavelength λ
An object of at least one embodiment is to provide a new device, a module for lateral superresolution, comprising an optical device adapted to create at least two light spots spatially differentiated from a diameter less than 1, 5 times the wavelength, λ, such as localized features of combinations of the intensity of said spots are smaller than 40% of the wavelength λ
Another goal is to provide a vision system, referenced as a module superresolution longitudinal configured to change at least one of geometry, geometry and polarization or the polarization of a light beam emerging from a point source depending on the longitudinal position of a point source.
Another object of at least one embodiment is to provide a superresolution system for fluorescence microscopy comprising the lateral superresolution module incorporated in the illumination path of the microscope, projecting on a sample consisting of a plurality of fluorophores, positioned at different lateral positions in the compact light distribution, a sequence of light spots, of spatially differentiated size of the order of a half wavelength, each of the fluorophores fluorescing with a sequence of fluorescent light intensities depending linearly or non-linearly of the light incident on the fluorophore and characterizing a lateral position of the fluorophore.
Another object of at least one embodiment is to provide a superresolution system for fluorescence microscopy comprising the longitudinal superresolution module incorporated in the detection path of the microscope, wherein the light intensity emerging from a plurality of point sources, placed in a small illuminated volume, are separated either on separate detectors or either on distinct geometric positions on the same detector or on a combination of both.
Another object of at least one embodiment is to provide a superresolution system for fluorescence microscopy, the system comprising either a lateral superresolution module incorporated in the illumination path of the microscope, and/or a longitudinal superresolution module incorporated in the detection path of the microscope in a detection circuit for said superresolution fluorescence microscope.
Some embodiments of the present invention provide a new technique based on two optical superresolution modules, new and complementary, referenced herein as the lateral module of superresolution and. The lateral module of superresolution provides mainly additional lateral resolution and the longitudinal module of superresolution module provides mainly additional longitudinal resolution. Embodiments using a single module can also be implemented. Both optical modules are complemented by suitable algorithms and an improved detection module. The modules can be integrated into existing fluorescent microscopes. Alternatively, fluorescence microscopes dedicated and optimized can be developed around the three dimensional superresolution system.
The methods and devices for superresolution, according to embodiments of the present invention differs significantly from conventional techniques and designs of the prior art, and in so doing, provide a device developed in order to achieve the techniques and devices of a superresolution system capable of measuring with high accuracy the descriptors of a fluorophore and recognize and measure the descriptors of several fluorophores placed in the same illuminated volume.
The present invention will be better understood from the following detailed description of preferred embodiments thereof, taken together with the drawings.
Other objects and advantages of the invention will become apparent to the reader, and it is considered that these objects and advantages are part of the present invention.
The invention will now be described in connection with certain preferred embodiments with reference to the following illustrative figures so that it can be better understood.
With specific reference now to the figures in detail, it is emphasized that the indications represented are presented as an example and for purposes of illustrative discussion of the preferred embodiments of the invention and are presented only in order to provide what is considered to be the description of the most useful and easy to understand principles and conceptual aspects of the invention.
In this regard, no attempt is made to show structural details of the invention in more detail than is necessary for a fundamental understanding of the invention, the description taken with the drawings making apparent to those skilled in the art how the several forms of the invention may be embodied in practice.
In the drawings:
In all the figures, like reference numerals identify like parts.
The usual definitions are used for the description: phase and polarization, polarimetry, Stokes parameters and measurement techniques Stokes parameters.
The center or centroid of a light distribution is the center of gravity of the intensity. The diameter of a light distribution is the diameter of the first zero intensity, both for regular and singular waves, without taking into account the central zero of a singular wave. Two light distributions are collocated if their centers coincide or are separated by a fixed, predetermined spatial value.
In this paper we use the emission wavelength, as the basic metric system.
In this paper, the usual definitions are used for the following optical components: lens whose definition has been broadened to include all optical means which transmit, refract or reflect the light, auxiliary optics—optical sub-module to interface and adjust either the geometric parameters or the parameters of phase and/or polarization between two other optical sub-modules or modules-, polarizer, analyzer, retardation plate, beamsplitter, polarizing and non-polarizing, beam combiner, polarizing and non-polarizing.
We refer to a partial polarizer to describe a component or a module whose absorption is different for the two linear polarizations—linear dichroism—or for the two circular polarizations—circular dichroism.
We refer to dynamic sub-modules of polarization or phase, to describe the optical means, which polarization or phase vary over time in a controlled manner, discrete or continuous.
These dynamic polarization or phase sub-modules include, but are not limited to: rotating on their axes wave plate, light valves based on liquid crystal technology, electro-optical devices, also known as Pockels cells, Kerr cells, electro-optical resonant devices, magneto-optic devices, also known as cells Faraday, acousto-optic or elasto- or any combination of these means.
We refer to “centroid algorithm” to describe the standard procedure for measuring the centroid and possibly the width (FWHM—Full width Half Maximum) of a light distribution.
Many articles have been published on this algorithm such as the article Lindegren in 1978 (“Photoelectric astrometry—A comparison of methods for precise picture location,” in Modern Astrometry, Proceedings of the Colloquium, Vienna, Austria, Sep. 11-14, 1978, 197-217 (1978)).
In this paper, the usual definitions are used for following optoelectronic components: photoelectric detector, CCD, EMCCD, CMOS SPAD—Single Photon Avalanche Diode and SPAD matrix.
We use the following terms:
To simplify the reading and understanding of the text we will use the term image to the output of a single pixel detector such as PMT or SPAD, considering it as an image consisting of a single pixel.
Where no ambiguity exists, or where the distinction between the three types of images is not necessary, we will use the simplified generic term of image.
The images described in this document may be characterized as microimages, images of size substantially equal to a small number of the Airy disc diameters, typically less than 5 diameters, and/or low number of pixels, typically 4*4 to 32*32.
In a digital image Aj, the indices m and n represent the indices of the pixels, and the origin of the pixels will be selected as the projection of the center of the analysis volume defined in a later paragraph.
We presented the images using the terminology used for matrix detectors, such as CCD, EMCCD and CMOS. For SPAD and SPAD arrays the measurement result is an ordered list in time of photons impact detailing, for each photon, the time of impact and the position of the impact. To simplify the presentation of this document, we will include this case in our definition of images.
Polarimetry and Stokes Vector
Polarimetry refers to the measurement of the polarization state of incident light.
The polarization state of the incident light can be described by the Stokes parameters, a set of values introduced by George Gabriel Stokes in 1852 and used in optics.
Additional technical information known to the skilled in the art
In this chapter we take a set of technical elements necessary for the description of the invention and known to those skilled in the art.
Cartesian and Polar Coordinates
The polar coordinates of a point, ρ, θ are deduced from the Cartesian coordinates x, y using the equation:
Electric Field in Polar Coordinates and Angular Modes
Given a complex electric field vector, E(ρ,θ), described in polar coordinates (ρ, θ), the electric field can be represented by an real amplitude, A(ρ,θ), an real phase φ(ρ,θ) and a unit vector of polarization, u(r,θ):
E(ρ,θ)=A(ρ,θ)exp[iφ(ρ,θ)]u(ρ,θ) (EQ. 2)
It is customary in Optics to decompose the field components, i.e. its amplitude, phase and polarization in orthogonal modes, Cartesian or polar.
Many decompositions in orthogonal polar modes, such as Gaussian, Hermite-Gaussian and Laguerre-Gaussian modes are known to those skilled in the art.
We mainly use in this paper, the decomposition of the amplitude of the electric field in Hypergeometric-Gaussian modes, HyGG, with the following form:
A(ρ,θ)∝ρp+|m|exp(—ρ2+ilθ) (EQ. 3)
In this decomposition, ρ is the radial mode and l is the azimuthal order.
Singular Waves
A singular wave includes a null intensity at the center and an azimuthal phase variation of a multiple of 2π. This research topic in optics, initiated by the seminal article by J F Nye and M. Berry in 1974, is now known as “singular optics.” Examples of regular and singular waves are presented in the following.
Topology and Compact Light Distributions
A singular wave includes a null intensity at the center and an azimuthal phase variation of a multiple of 2te. This research topic in optics, initiated by the seminal article by J F Nye, et al. (“Dislocations in Wave Trains,” Proceedings of the Royal Society of London, Series A, Mathematical and Physical Sciences (1934-1990) 336, 165-190(1974)).
We distinguish different families of point light distributions, of different topologies:
Two compact light distributions will be deemed being of different topological families if they meet at least one, and any of the following conditions:
One is regular and the other is singular,
Alternatively, two light distributions projected onto a given volume will be considered of different topologies if a significant portion of the surface illuminated together, the gradients are of reversed direction.
The fluorophores are the best known example of the family of point-source, light nanoemitters of size substantially smaller than the diffraction limit. A nanoemitter is a small secondary light emitter attached to an object, and it is significantly smaller than a fraction of a wavelength, typically but not limited to a size smaller than one fifth of the wavelength; a light nanoemitter absorbs the incident energy and re-emits light at the same wavelength as the incident light or different wavelengths; the light emitted by the nanoemitter may be coherent, partially coherent or incoherent with the absorbed light. The main examples of nanoemitters are fluorophores and nanoparticles, but also include many other elements.
The definition in the context of the invention of nanoemitters light is determined by the following two conditions:
Creating a secondary point-source light emitter, and
The physical mechanisms that can create a nanoemitter are numerous, and include but are not limited to absorption, scattering or reflection, fluorescence, emission-depletion, photo activation phenomena, fluorescence of two or more photons, or non-elastic scattering, Raman scattering, or any other physical mechanisms known to those skilled in the art. We use the term light emission to describe the emission of electromagnetic waves by a light nanoemitter, the light being coherent, incoherent or partially coherent.
The physical mechanisms that can create a nanoemitter are numerous, and include but are not limited to absorption, scattering or reflection, fluorescence, emission-depletion (S W Hell, et al., “Breaking the diffraction resolution limit by stimulated emission: stimulated-emission-depletion fluorescence microscopy,” Optics Letters 19, 780-782 (1994)), photo activation phenomena (M J Rust, et al., “Sub-diffraction-limit imaging by stochastic optical reconstruction microscopy (STORM),” NatMeth, 3, 793-796 (2006)), and E. Betzig, et al. (“Imaging intracellular fluorescent proteins at nanometer resolution,” Science, 313, 1642 (2006)), fluorescence of two or more photons, W. Denk, et al. (“Two-photon laser microscopy,” (Google Patents, 1991)), or non-elastic scattering, Raman scattering, or any other physical mechanisms known to those skilled in the art. We use the term light emission to describe the emission of electromagnetic waves by a light nanoemitter, the light being coherent, incoherent or partially coherent.
We refer to in this patent to descriptors of a single fluorophore to denote the set of information describing a fluorophore as a point source at a given moment. Since the nanoemitter is considered as a point source, all the information representing it contains a limited number of parameters, namely: its position in space, its intensity, its spectral characteristics of the intensity, coherence, phase and polarization of the light emitted by the fluorophore as a function of the incident light.
However, in most cases, and in the description of the invention, we refer, under the designation of descriptors, a subset of descriptors of a fluorophore including its geometric position, its intensity, and the type of fluorophore, whether several populations of light nanoemitters, differentiated for example by their emission spectrum, are present in the same sample. This simplification used in the description does not alter the scope of the invention which will include in its scope all the descriptors of light nanoemitters.
To simplify the understanding of the context of the invention, the following description refers only the simplest case, one in which the nanoemitter is a fluorophore and physical interaction is the one photon fluorescence. However, this description should be understood as a simplified illustration of a general description of the methods and concepts applicable to all light nanoemitters mentioned previously or known to those skilled in the art, regardless of the underlying physical phenomenon.
It is striking that the nanoemitter samples the incident light intensity field at a three-dimensional position accurately without influence of the complete spatial distribution of the incident intensity.
We will reference this remarkable property in this document as the sampling ability of light nanoemitter.
We refer again to the
We refer again to the
The luminous biological object contains information that is relevant to the biological object, mainly spatiotemporal information, the object position and orientation with respect to time, and morphological information, for example in the case of division of a cell in two.
The primordial Information, the map in the terminology of general semantics, is the set of descriptors fluorophores and their evolution over time. Biological and geometric information will only be extrapolations of this primordial information.
The measurement system will calculate an evaluation of the descriptors of the fluorophores, the measured map. This measured map differs from the original map, due to noise, measurement conditions, the system limits or measurement uncertainty. This information map can be developed later into different levels of abstraction.
The map, the basic level, therefore, comprises an evaluation of a set of descriptors of fluorophores, and this information may, for example, be structured as a list of fluorophores and their descriptors. This level of abstraction, which presents the results of direct measurement, contains a priori no biological information but is the results of a physical measurement described by points of light, which could also represent any marked entity.
The second level, the geometric level of abstraction, structures nanoemitters in the form of geometric objects. It comprises a description of luminous objects and their dynamic characteristics, such as their position or orientation, or their morphology. At this level, the information is still physical and geometric information describing a set of objects. The geometrical information uses the measured card and auxiliary information, potentially external to the system, the relation between light spots and objects.
The biological level of abstraction, allows some understanding of the biological reality through a constitutive relationship between objects measured and corresponding biological entities. It contains a set of information on the biological object, mainly the position and its dynamics, its shape and morphology. The biological information uses the measured card and the geometrical information and auxiliary information, potentially external to the system, the relation of the light spots and objects with biological entities. A number of conclusions on the biological functionality of the sample can be obtained at this level.
Conical refraction is an optical phenomenon predicted by Hamilton in 1832, and two months later confirmed experimentally by Lloyd. Conical refraction describes the propagation of a light beam in the direction of the optical axis of a biaxial crystal. Hamilton predicted that the light emerges in the form of a hollow cone of rays. Conical refraction is an important phase in the history of science and has played a role in the demonstration of the theory of electromagnetic waves.
Conical refraction is an optical phenomenon predicted by W. R. Hamilton in 1832 (“Third Supplement to an Essay on the Theory of Systems of Rays,” Trans. Royal Irish., Acad., pp 1-144 (1833)), and two months later confirmed experimentally by Lloyd (“On the Phenomena presented by Light in its Passage along the Axes of Biaxial Crystals”, The London and Edinburgh Philosophical Magazine and Journal of Science ii, 112-120 (1833), and “Further Experiments on the Phenomena presented by Light in its Passage along the axes of Biaxal Crystals”, The London and Edinburgh Philosophical Magazine and Journal of Science H, 207-210 (1833)). Conical refraction describes the propagation of a light beam in the direction of the optical axis of a biaxial crystal. Hamilton predicted that the light emerges in the form of a hollow cone of rays. Conical refraction is an important phase in the history of science and has played a role in the demonstration of the theory of electromagnetic waves.
A renewed interest in the conical refraction occurred in the last years of the twentieth century has led to a complete theory by M. V. Berry, et al. (“Conical diffraction asymptotics: fine structure of Poggendorff rings and axial spike,” Journal Of Optics A-Pure And Applied Optics, 6, 289-300 (2004)), Berry, et al. “Conical diffraction complexified: dichroism and the transition to double refraction,” Journal Of Optics A-Pure And Applied Optics, 8, 1043 (2006), and Berry, et al, “Chiral conical diffraction,” Journal Of Optics A-Pure And Applied Optics 8, 363 (2006)), validated experimentally in 2009 (C. Phelan, et al,
Conical diffraction has attracted considerable theoretical and experimental, but “no practical application seems to have been found,” (MV. Berry, et al., “Conical diffraction: Hamilton's diabolical points at the heart of crystal optics,” Progress in Optics 50, 13 (2007)).
Other effects exist, creating inherently weaker conical diffraction effects or creating conical diffraction along a short optical path. These effects include polymers, liquid crystals and induced externally birefringence effects. The polymers include but are not limited to: stretched polymer sheets and cascade polymerization, liquid crystals include but are not limited to: thermotropic biaxial nematic phase, the external effects induced birefringence include, but are not limited to: applying an electric field creating an electro-optical effect on a non-centrosymmetric cubic crystal, and the photo-elastic modulator.
Other effects exist, creating inherently weaker conical diffraction effects or creating conical diffraction along a short optical path. These effects include polymers, liquid crystals and induced externally birefringence effects. The polymers include but are not limited to: stretched polymer sheets and cascade polymerization (A. Geivandov, et al. “Printable Thin Film birefringent Retarders for LCD”). Liquid crystals include but are not limited to thermotropic biaxial nematic phase (B. Acharya, et al. “Biaxial Nematic Thermotropic The Elusive Phase in Rigid Bent-Core Molecules,” Pramana 61, 231-237 (2003)); the external effects induced birefringence include, but are not limited to applying an electric field creating an electro-optical effect on a non-centrosymmetric cubic crystal (T. Aldonado, “Electro-optic modulators,” in Handbook of Optics, M. Bass, ed. (McGraw Hill, Orlando, 1995)); and the photo-elastic modulator (J. Kemp, “Piezo-Optical Birefringence Modulators: New Use for a Ion-Known Effect,” Journal of the Optical Society of America 59, 950-953 (1969)).
Referring now to
The spatial variable, R, the conical imaging plane, and the wave vector, U, are represented by cylindrical coordinates R, θR et U, θU. λ is the wavelength of light.
The behavior of the electric field emerging from the conical crystal 32 is fully characterized by a single parameter, the radius conical R0; the conical radius depends on the material and geometrical characteristics of the crystal, as defined in [Berry, 2004].
We introduce standardized parameters for the description below of the light distribution, to be valid in both conical imaging plane and at the focus of the microscope objective, in the limits of the scalar theory of diffraction.
The normalized radial position, ρ, the wave vector normalized, u, represented by cylindrical coordinates par ρ, θR et u, θU, and the normalized radius conical ρ0 are given by:
U0 being the numerical aperture of the system. For ρ0<2, we refer here to a thin conical crystal, for ρ0<<1, we refer here to the form of a linear thin conical crystal and for ρ0<0.5 to a thin sinusoidal conical crystal.
The wave emerging crystal thin conical, E(ρ, θR), expressed in normalized coordinates, is constituted by the superposition of two waves, referred to herein as the fundamental wave, EF (p), a regular wave, and vortex wave, EV (ρ,θR), a singular wave; these two waves are coherent one with another, collocated, and circularly polarized with an inverse direction of chirality:
In this equation, EF (ρ) is the scalar fundamental amplitude, Fv (ρ) is the reduced scalar magnitude of vortex and they are given by:
EF(ρ)=2π∫du u cos(ρ0u)J0(ρu);Fv(ρ)=2π∫du u sin(ρ0u)J1(ρu). (EQ. 6)
For a thin linear conical crystal, the fundamental wave can be approximated by an Airy disk and the vortex wave can be approximated to a linear vortex, represented by:
Fv(ρ)=2πρ0∫du u2 J1(ρu). (EQ. 7)
Assuming that the action of partial polarizer, 29, is the scaling of the vortex wave by a parameter a, the Stokes parameters can be deduced from the above equations:
S0=(EF(ρ))2+(α2Fv(ρ))2
S1=2αEF(ρ)Fv(ρ)sin θR;S2=2αEF(ρ)Fv(ρ)cos θR;
S3=(EF(ρ))2−(α2Fv(ρ))2
β=θR; (EQ. 8)
We use the terms of “sparse object” to describe a set of light emitting point like emitters, of a number less than twelve, positioned in a volume whose size in each dimension is less than 3 wavelengths, at the wavelength of transmission or at the wavelength of the reflection of the emitters. The volume of a size less than 3 wavelengths that contains the sparse object is referred to as a analysis volume of reduced size.
We refer now to
The functionality of the volumic containment is limited in all three spatial dimensions, the observed region of the sample volume to a size as small as possible, analysis volume. The functionality of the volumic containment limits the analysis volume by the combination of two effects: the confinement of the light projected onto a small area, ideally the size of the Airy spot, 50, and the elimination of defocused light by the confocal hole, 28, of
Consider a sparse object, 51, consisting of a plurality of fluorophores, 53 to 59. The fluorophores from 53 to 55 positioned in the test volume 60, and only they are both excited by the light source and the photons emitted by them arrive at the detector module. The fluorophores not located in the cone of illumination, 56 and 57 are not illuminated by the incident light. The light emitted by the fluorophores 58 and 59, located at the conjugate plane of the confocal hole, 28 of
Two different Cartesian coordinates are defined in the system,
The reference “i”: The axes referenced “i” represent a Cartesian reference system centered on the center of the analysis volume, 61.
The reference “a”: the axes referenced “a” represents a Cartesian reference centered for each light nanoemitter on the nanoemitter considered as a discrete point, 62.
When using the PSIT method, described later, if a vortex is projected on the sample, the center of the vortex will be generally defined as the center of the analysis volume.
The confocal microscope limit the analysis volume using the volumic confinement described above. The volumic confinement volume is obtained by the combination of two effects: confinement of the light projected on a small surface, ideally of the size of the Airy disk, 50, and removal of defocused light by the confocal hole, 41. The superposition of these two effects creates a small volume, the analysis volume 60. This volume determines the size of the elementary cell detected by the system.
At least one embodiment of the invention uses conical diffraction to realize the fundamental optical modules of the technique. However, alternative implementations, replacing the modules based on conical diffraction by modules based on other optical concepts, are able to provide the same functionality. They are part of the scope of this invention. Alternative optical concepts include but are not limited to uniaxial crystals, subwavelength gratings, structured laser modes, holographic components and other techniques known to the skilled in the art.
These concepts, techniques and optical and optoelectronic devices are known to those skilled in the art and all such optical means are described in numerous publications such as the book written by D. Goldstein, “Polarized Light”, Pawley, “Handbook of Biological Confocal Microscopy,” Bass, “Handbook of Optics”, and many other publications known to those skilled in the art.
We use in this paper the acronym, SRCD, “Super Resolution using Conical diffraction” to name the platform, modules and systems specific to the preferred implementation of this invention.
We use in this paper the acronym PSIT “Projected Sequence of Intensities with various topologies”
We use in this paper the acronym, PDOS, “Position Dependent Optical Semaphore”.
The SRCDP platform, “Conical diffraction using Super Resolution Platform” is a platform for microscopy, implementing the measurement methodology and using optical modules based on conical diffraction.
SRCDP platform is the preferred implementation of the measurement methodology. We use in this paper the acronym LatSRCS to name the optical module implementing the PSIT method for the preferred implementation of this invention.
We use in this paper the acronym LongSRCS to name the optical module implementing the preferred implementation of the method PDOS of this invention.
Some embodiments of the present invention comprise a new measuring methodology; the measurement methodology, and a coherent set of systemic and algorithmic method, hardware tools, software tools and algorithms for its implementation
The measurement methodology according to embodiments allows acquisition of nanosized optical data and image superresolution.
The measurement methodology is primarily, but not exclusively, used for the measurement of super-resolved biological samples data marked with fluorophores.
The measurement methodology can be implemented using the different methods of measurement and processing algorithms, described below.
Among other things, the measurement methodology can be implemented together or separately using two new measurement methods, referred to as:
Some embodiments of the invention also relate to a system—a platform for microscopy—implementing the methodology of measurement using the measurement methods PSIT and PDOS. This system, the SRCDP platform, “Conical diffraction based Platform Super Resolution” is the preferred implementation of the measurement methodology.
These concepts, techniques and optical and optoelectronic devices are known to those skilled in the art and all such optical means are described in numerous publications such as the book written by D. Goldstein, et al. (“Polarized Light” (CRC, 2003), Vol. 83). The “Handbook of Microscopy Confocal”, J B Pawley, (Springer Verlag, 2006). “Handbook of Optics”, M. Bass, (McGraw-Hill, 2001)) and many other publications known to those skilled in the art.
Additionally, the SRCDP platform includes an improved detection module, a control module of the system, and software support.
The measurement methodology comprises using both measurement methods, the methods PSIT and PDOS. However, in some applications, the use of both methods may not be necessary, we will refer in this case to the simplified measurement methodology, which is part of the scope of this invention.
Some embodiments of the invention also relate to methods of using the measurement methodology for measuring distribution of fluorophores and fluorophores, and monitoring in two or three dimensions of fluorophores.
In addition, certain embodiments of the invention relate to a large number of variants of implementations of the methodology and methods PSIT and PDOS, platform SRCD, optical modules and LatSRCS LongSRCS and algorithmic SRCDA.
The functionality of the confocal microscope described by Minsky, and explained previously, is limiting in three spatial dimensions, the observed region of the sample volume to a size as small as possible, the volume analysis.
The functionality of the confocal microscope described by M. Minsky (“MicroscopyApparatus,” (Google Patents, 1961)) and explained previously, is limiting in three spatial dimensions, the observed region of the sample volume to a size as small as possible, the volume analysis.
Referring now to
In
A system implementing the method according to at least one embodiment of the invention is capable of recovering independently and accurately the attributes of several fluorophores in a luminous volume of dimensions similar to those of confocal microscopy. To achieve this goal, the methodology according to some embodiments of the invention is designed to create optically for each illuminated volume, a large amount of information in both time and spatial domains.
The most developed process of the measurement methodology, according to an embodiment of the invention, can be segmented into seven steps, five optical steps, an optoelectronic detection step and an algorithmic step.
Optical Steps:
Detecting the light intensity by one or more point like or matrix photodetectors.
Algorithmic Step:
According to another embodiment of the present invention, the measurement methodology consists in the realization of optical steps, previously described and omitting either the first is or the fourth optical step.
The compound optical process that implements the methodology comprises: performing a series of optical measuring processes, controlled by the control module of the system, by varying the sequence of illumination and/or the functionality of the channels and/or the position of the sequence illumination as function of measured data or of external information. An example of compound optical process implementing the methodology according to an embodiment of the invention will be detailed below.
The intermediate result, the raw information is obtained at the end of the detection step. Raw information comprises a set of images Aop(m, n) representing for the o light distribution, the image from the detection channel p.
As in a confocal microscope, the measurement process analyzes a small volume in a much larger object. It will therefore require the addition of additional modules, similar to those of a confocal microscope including a scanning process, a software module integration, analysis and visualization of data points in surfaces and/or three-dimensional objects.
A method of measurement PSIT according to one embodiment of the invention, projects a sequence of light distributions of different topologies, on the analysis volume.
The measurement method PSIT, performs the following functions:
In more detail, it is noted that:
The transmission sequence comprises at least two point like light distributions, of different topological families
The transmission sequence is projected onto a biological sample labeled with fluorophores which are referenced as light nanoemitter.
The light emitted, emerging from each light nanoemitter, is dependent for each nanoemitter of the light intensity, in the incoherent case or on the electromagnetic field, in the coherent case, incident on the three-dimensional spatial position of the light nanoemitter, the aforesaid light sampling property of the nanoemitter discussed previously.
For each light distribution pattern of the transmission sequence projected on the sample, an optical image is created.
The set of images corresponding to all the light distributions of the transmission sequence is referred to as the sequence of images.
The PSIT method according to this embodiment can acquire mainly lateral information, that is to say, the lateral position of each of the fluorophores.
In a preferred embodiment, the PSIT method is implemented by the projection of light distributions of different topologies created by conical diffraction and modified by a variation of the polarization states of input and output.
A PDOS method according to an embodiment of the invention includes the distribution of an “optical semaphore” of the light reemitted by the fluorophores between at least two detectors.
Ideally, the function of the optical semaphore is to separate different areas of the test volume on different detectors. Practically, the optical semaphore creates, for each detector, a transfer function of the light emitted by a light nanoemitter, depending on the position in space of the light nanoemitter and different for the different detectors.
In a preferred embodiment, the PDOS method is implemented to separate on different detectors the collimated light, emerging from fluorophores positioned at the focal plane of the lens, from non-collimated light emerging from fluorophores lying within or beyond the focal plane.
The PDOS method, in its preferred embodiment, allows acquiring essentially longitudinal information, that is to say, the longitudinal position of each of the fluorophores.
Mathematically, the method according to some embodiments of the invention provides a transfer function converting the spatial distribution of the fluorophores in space in unprocessed information consisting of a set of images. The algorithmic performs the inverse operation: it reconstructs the spatial distribution of the fluorophores in space from the set of images in the unprocessed information.
In mathematical terms the algorithm solves an inverse problem or parameter estimation. The model equations are known and the number of fluorophores in a sparse object is a-priori limited. 11 the mathematical procedures known to those skilled in the art can be used for solving inverse problems and parameter estimation. We describe later an example of algorithm adapted specifically to the measurement methodology according to an embodiment of the invention.
In addition, we present, for its symbolic value, a new solution to the problem of discrimination of two points located at a small distance from each other. This problem studied by Lord Rayleigh, is the base of the resolution criterion in many areas of Optics.
It has thus been described, rather broadly, the characteristics of the invention in order that the detailed description thereof may be better understood, and in order that the present contribution to the art may be better appreciated. Many additional features of the invention will be described below.
The preferred implementation of the method according to one embodiment of the invention is a hardware platform and algorithms, referred to as the SRCDP platform, 500, shown in
The SRCDP platform, 500, implements the method according to an embodiment of the invention, by combining the two methods PSIT and PDOS above.
The platform SRCDP observed,
The platform SRCDP, 500, as shown in
In its hardware part:
We describe, with reference to
The optical module LatSRCS, 700 according to this embodiment is an optical module, projecting on a plurality of fluorophores in a sample, a sequence of compact light distributions of different topology. Each fluorophore fluoresces with a sequence of fluorescent light intensities dependent on the incident intensity on the fluorophore and characterizing the lateral position of the fluorophore. In most embodiments, the light compact distributions of different topologies are created by interference with variable amplitudes and phases between an ordinary wave and singular wave. In the preferred embodiment, the regular and singular waves are created by a thin conical crystal.
The optical module LatSRCS, 700, is positioned in the illumination path of the confocal microscope 200; it projects a sequence of compact light distributions of different topologies on the sample 11 using the confocal microscope objective 200. In the preferred embodiment using the conical diffraction, the incident intensity at a specific position on the sample 11 will be proportional for each light distribution pattern, to a specific combination of the Stokes parameters.
The optical module LatSRCS, 700, uses an inherent feature described above, specific to the fluorophore, which samples the intensity of light incident on its precise position (the fluorophore), and reemits fluorescent light dependent on the incident light. It is remarkable that the measured information is directly related to the position of the fluorophore in the compact light distribution, relayed by the Stokes parameters. This information is frozen by the functionality of the fluorophore, its ability to absorb and re-emit light, breaking the optical chain. This information is carried by the fluorescent light as an emerging light distribution recoverable by a detector assembly 65.
If the incident light varies temporally according to a sequence of compact light distributions of different topologies, the intensity of the fluorescent light reemitted varies in the same proportions. The sequence of the re-emitted fluorescent light is proportional to the sequence of compact light distributions of different topologies. From this information, it is possible to retrieve the position of the fluorophore, as explained below.
The PSIT method, according to embodiments of the invention, refers to the projection of a sequence of compact light distributions of different topologies in a microscope, the interaction with fluorophores, collecting the reflected light by the objective of microscope, 22, detecting the fluorescent light by the improved detector assembly 65, and the analysis of the information by a suitable algorithmin some embodiments, the improved detection assembly, 65, comprises a single detector, and recovers only the overall intensity as a function of time, while in other embodiments the improved detection assembly comprises a small area of pixels and recovers also the spatial distribution of the fluorescent light. All retrieved information consisting of a plurality of images, the named as lateral superresolution images.
In a preferred embodiment, the contribution of a fluorophore in the illuminated volume positioned in a specific lateral superresolution image is proportional to a specific combination of the Stokes parameters of the incident light at the fluorophore position.
Lateral superresolution images, the information created by compact light distributions of different topologies, is new and was not present in the prior art. This new information helps to refine the position of the fluorophores, to quantify the number of fluorophores present in the illuminated volume and to differentiate multiple fluorophores present in the same volume.
We refer now to
Referring to
Referring to
We denote mainly the following light distributions:
The elementary light distributions described in
This redundancy allows some averaging of random phase errors inevitably present in many measurement process of biological objects. This reinforces the robustness of the measurement methodology of the embodiments of the invention and its applicability.
New light distributions can also be obtained as mathematical combinations of elementary light distributions. The “pseudo-vortex”, light distribution, calculated from arithmetic combinations of the four distributions Stokes has the feature of having a strong gradient at the origin.
Method PS1T was originally designed to allow lateral superresolution, however PS1T method can also be used to obtain the longitudinal position of a fluorophore. Indeed, some elementary light distributions are relatively insensitive—within reasonable limits—to a variation of the longitudinal position of the fluorophore, others are rather sensitive. A sequence of compact light distributions, some of them independent and some of them depend on the longitudinal position would reveal the longitudinal position of fluorophores.
In addition for the light distributions which are highly dependent on the longitudinal position of the fluorophore, a series of elementary light distributions slightly shifted longitudinally, one relative to the other can be projected on the sample, allowing a set of images containing longitudinal information.
In addition, some more complex elementary light distribution, consisting of more complex overlapping of waves with a strong longitudinal dependence exist, eg the “three-dimensional dark spot” described by Zhang, which create a black spot surrounded in three dimensions by a luminous sphere. As described by Zhang, “Investigation and applications of the three dimensional (3D) dark spot surrounded by a light shell in all directions have attracted a great deal of attention. In superresolution fluorescence microscopy, a 3D dark spot is used as the erase beam”. These “three dimensional dark spots” consist of a superposition of Laguerre-Gauss functions, which can be achieved within a laser cavity or using a hologram or a phase plate, as suggested by Zhang, or using uniaxial or conical crystals as suggested by the inventor.
In addition, some more complex elementary light distribution, consisting of more complex overlapping of waves with a strong longitudinal dependence exist, eg the “three-dimensional dark spot” described by Zhang (“Generation of three-dimensional dark spots with a perfect light shell with a radially polarized Laguerre—Gaussian beam,” Applied optics, 49, 6217-6223 (2010)), which create a black spot surrounded in three dimensions by a luminous sphere. These “three dimensional dark spots” consist of a superposition of Laguerre-Gauss functions, which can be achieved within a laser cavity or using a hologram or a phase plate, as suggested by Zhang, or using uniaxial or conical crystals as suggested by the inventor.
Vector Effects
The theory developed so far describes the light distribution in the imaging plane of the microscope 35. The distribution of the light projected onto the sample is, according to the theory of the geometrical imaging, a reduced image of the light distribution in the image plane.
However, as described extensively in the literature, for a high numerical aperture objective, the imaging geometric theory is not accurate and vector effects must be taken into account. These effects consist essentially in the presence of a component, longitudinally polarized.
Referring again to
Alternatively, the output polarization adaptation submodule, 74, may be variable and adapted to the topology and the symmetry of each of the compact light distribution.
LongSRCS Optical Module Implementing the PDOS Method
We describe below an optical module LongSRCS with more details. The system of longitudinal superresolution, according to an embodiment of the invention, channels the incident light intensities of a plurality of point sources located in a small illuminated volume, either on separate detectors or on distinct geometric positions on the same detector or on a combination of both, as function of the spatial position of each point source.
In simpler words, the intensity emitted by a fluorophore positioned longitudinally at the point A will be physically separated from the intensity emitted by a fluorophore positioned longitudinally to point B.
The optical module LongSRCS, according to an embodiment of the invention, allows the separation in volume slices, different slices of the illuminated volume being physically separated on different sets of detectors.
In the preferred embodiment, which will be explained below, the optical module LongSRCS separates an illuminated volume in at least three adjacent slices, separating the middle slice from of the other two slices on sets of independent improved detectors, and creating a spatial differentiation between the two remaining slices on the same set of improved detectors.
We refer now to
The optical module LongSRCS channels the incident light intensity of a plurality of point sources located in a small volume of light, either on separate detectors or on distinct geometric positions on the same detector either a combination of both, depending on the longitudinal position of each point source.
In a preferred embodiment, it operates on the fluorophores, represented by 80, 80′ or 80″, according to their longitudinal position. It comprises a first collimating lens 81, which may consists, in some embodiments of the microscope objective 4.
The fluorophore 80 is positioned in the focal plane of the collimating lens 82, the light from the fluorophore 80, emerging from the collimating lens 81 is collimated.
The fluorophores 80′ and 80″ are placed before and after the focal plane of the collimating lens, 82, at a distance of ±Δz, the light from the fluorophores 80′ or 80″ emerging from the collimating lens 81 being convergent or divergent.
The LongSRCS optical module includes a polarization beam separator, shown in
Two quarter waveplates, 86 and 87, transform, for each channel, the linearly polarized circular polarizations.
A conical crystal is placed in each of the channels 88 and 89. In each channel, a conical diffraction setup, as described in the
For the fluorophore, 80, positioned in the focal plane of the collimating lens 82, the light emerging from the collimator lens 81, is, as discussed above, collimated; referring to the setup of conical diffraction, the Numerical Aperture of the collimating lens 81, in the image space, and the normalized radius cone are zero, so that the effect of conical diffraction on the beam from the fluorophore 80 is zero. Therefore, the conical crystal does not change the geometry of the fluorescent light emitted by the fluorophore, or its polarization, which remains circular with the same chirality.
For fluorophores, 80′ or 80″, which are not positioned in the focal plane of the collimating lens 82, the light diverges or converges; Referring again to setup conical diffraction described above, the Numerical Aperture in the image plane of the collimating lens 81, which is equivalent to the first lens of the conical diffraction setup, 31, is non-zero. For a given value Δz defocus, positive or negative, most of the light emerging from the crystal is contained in the conical wave vortex, which has a form of a vortex, and is inverted chirality.
The functionality of the conical diffraction setups positioned in each of the channels is to distinguish the collimated light from the light converging or diverging by reversing the chirality of the circular polarization of the light for converging or diverging light.
Two other blades quarterwave plates, 90 and 91 transform the circular polarizations, emerging each channel, linear polarizations. We refer, for each channel, to the linear polarization, which would have emerged from the retardation plate, if the crystal had been removed, as the polarization of collimation
The optical module comprises a LongSRCS combiner/separator of four ports, shown in
For each channel it separates the two polarizations, and merges the two polarizations of collimation in the same path, the path of collimation, 93, and the polarized light orthogonal to the collimation polarization in another path, the path of non-collimation, 94. The directions of the axes of the quarterwave plates, 86, 87, 90 and 91 must be chosen appropriately. The combined beams do not interfere, because they come from originally unpolarized beam.
The incident light into the path of collimation is focused onto the detector of collimation, 96, using the focusing lens of the collimating path, 95, which behave functionally as the second lens, 32, of the conical diffraction setup.
In the path of non-collimation, an additional lens 97 is inserted, and the additional lens 97, together with the collimating lens, 81, creates a new lens system, 98, whose focal plane, 99 is positioned at a different position of the focal plane of the collimating lens 82, the position of the fluorophore 80′. An additional quarter waveplate, 100, cancels the action of the quarter waveplates, 90 or 91, turning back the incoming beams of each of the channels of polarization, to the circular polarization, which they were at the output polarized crystals conical, 88 or 89.
An additional conical crystal, 101, is added in the way of non-collimation as a third conical diffraction setup—the auxiliary conical diffraction setup—with the system of lenses 98, acting as the first lens of the conical diffraction setup, 31.
The fluorophore 80′ have been positioned before the focal plane of the collimating lens, 82, at a distance of Δz, but, relative to the lens system 98, it is positioned at the focal plane 99. The light from the fluorophore 80′ had already been converted into a vortex by one of the conical diffraction setups consisting of the collimating lens 81, and one of the conical crystals 88 or 89, depending on the channel of polarization traveled by the light. The light from the fluorophore 80 is collimated at the output of the lens system, 98, after the additional lens, 97.
Referring to the new conical diffraction setup, the numerical aperture of the lens system in the image space, and the normalized conical radius are zero for fluorophore 80; the effect of conical diffraction, of the auxiliary diffraction setup, on the beam emerging from fluorophore 80 is zero. Therefore, the conical crystal does not change the geometry of the fluorescent light emitted by the fluorophore. Light incoming from a fluorophore 80′, is a vortex before and after the conical crystal 98.
The fluorophore 80″ had been placed after the focal plane of the collimating lens, 82, at a distance of Δz on the lens system 98; it is placed at a distance of −2 Δz of the focal position, 99, and the light from the fluorophore 80″ converge also to the output of the lens system, 98, after the front lens, 97. The light from the fluorophore 80″, has already been converted into a vortex by one of conical diffraction setup consisting of collimating lens 81, and either one of the conical crystals 88 or 89, depending on the channel of polarization followed by the light. The conical crystal 101 changes the light from fluorophore 80″, and, for relevant parameters of the material, i.e. the size and orientation of the conical crystal, it reverts to a regular wave, slightly different from the Airy disk.
The objective lens of the non-collimation path, 102, is adapted to focus the plane containing fluorophore 80″, which is a regular wave, 104, and not the fluorophore 80′, which is singular on the pixelated detector assembly, 103. The incident light emerging from a fluorophore positioned in plane 104, such as the fluorophore 80″, is perfectly focused and is positioned at the center of the pixelated detector, 103. Incident light emerging from a fluorophore positioned at the plane 99 is a vortex and therefore focuses on an outer ring with a central zero. By separately recording the intensity at the center and the intensity at the outer part of the detector, it is possible to separate, with a slight overlap between the incident light planes 104 and 99. In addition, a fluorophore positioned at the plane 104 as the fluorophore 80, is slightly delocalized because the objective is calculated so as to focus on the detector plane 104. This improves the action of the optical module LongSRCS, pushing further the intensity of the vortex center and reducing duplication.
This simplified description of a preferred embodiment of the optical module LongSRCS, 800, allows many possibilities of variations and adaptations by changes in the optical design through changes known to one skilled in the art. These changes include, but are not limited to: the crystal material and orientations, the choice of polarization components, the choice of the polarization axes of the cascade elements, the number of sensors, or, reversing the roles of fluorophores 80′ and 80″. In addition, the module is ideally conditioned to be constructed as a set of monolithic subsets or even as a single monolithic unit.
Method PDOS and Lateral Measurements
Method PDOS was originally designed to allow longitudinal superresolution, however PDOS method can also be used for measuring the lateral position of a fluorophore. Indeed, the elementary light distributions are also sensitive to variation of the lateral position of the fluorophore. For a plane sample, in the case where the light projection is not possible, the method PDOS may replace the method PSIT for performing superresolution measurements
All these variants of the measurement methodology are considered part of the invention. The inventor has yet chosen in the preferred implementation to separate into two disjoint, separated, but complementary optical modules the lateral measures from the longitudinal measures to reduce the complexity of each one of the add-ons.
Detection Module
The corollary of the potency of the measurement methodology is the requirement of a more complex detection module, able to detect and retrieve information created. In scanning confocal microscopy, the detector is a detector consisting of a single element as a PMT or SPAD. The acquisition time of the detector is determined by the scanning mechanism.
The measurement methodology requires, in some embodiments, two detector modules, instead of one, the fundamental and vortex detector modules. In addition, the measurement methodology requires, in some embodiments, for each illuminated volume, the acquisition of the optical information on a small spatial grid, typically 16*16, at a rate higher than the pixel time, due to the requirement to identify and quantify the sequential signals.
An improved detection module, 65, may be implemented using small detectors with low number of pixels. Such a module would not have been possible ten or twenty years ago, due to the lack of appropriate technologies. Today, small detectors with small number of pixels, at high speed, with low noise characteristics are available on the basis of several technologies: SPAD arrays with a small number of pixels, such as 32*32 have been shown recently with acquisition rates up to 1 MHz. The improved detector module 65, may also be implemented using CCD, EMCCD or CMOS sensors. CCD sensors, CMOS and EMCCD with a small number of pixels exist or can be specifically designed. In addition, CCD sensors, CMOS EMCCD can be used using features as region of interest, sub-windowing or “binning”, available in many detectors.
The spatio-temporal information referenced herein is the position and the time of the impact of each fluorescent photon. In real systems, the spatio-temporal information is corrupted by the noise of the detector, which creates incorrect photons, and by inefficient detection, creating photons which are not detected, thereby reducing performance. In SPAD arrays, for each photon, the pixel that has detected it and the time of impact are received, i.e. the full spatiotemporal information is available. For CCD sensors, CMOS or EMCCD, the acquisition of multiple frames is necessary to approximate the spatio-temporal information.
In several implementations we will refer to separate detectors; in many cases the sensor can be either physically separated or consisting of different areas on a single detector, or a combination of the two previous cases.
Algorithms SRCDA
As stated previously, the algorithmic SRCDA can be implemented using the inverse problem methods of estimating parameters methods known to those skilled in the art.
We also present an algorithm according to one embodiment, specific to the measurement methodology, based on a set of descriptors.
Referring now to
An algorithmic procedure, presented in the
The preprocessing procedure, 111, reorganized the spatiotemporal information, 110, in sets of superresolution images, 112. This can be done using a filter bank procedure. The data set is then a small series of small images, typically 16*16 pixels. The pretreatment procedure is applied to a small number, of the order of several thousand, of spatiotemporal elements; it can be performed in real time using existing hardware.
The procedure descriptor, 113, the main step of the calculation, created from each image, a set of descriptors, 114, and their statistical significance. Descriptors include, but are not limited to: the intensity of each image, the presence in the image of a light distribution and its characterization as a regular distribution or as a vortex, the center of gravity, and moments of first and higher orders.
The third step is a filtering operation, 115, wherein only the descriptors that are statistically relevant, are retained.
The classification operation, 116, is the last step of the algorithm. The algorithm is capable of recognizing, on the basis of the set of descriptors, 114, and a knowledge base, 117, where the measurement different cases as a single fluorophore, two fluorophores separated longitudinally or laterally and three or more fluorophores.
Note that, due to the amount of information created, numerous cases that were ambiguous in fluorescence microscopy will be clearly identified. For example, as described in more detail later, a single fluorophore must meet a long list of conditions and cannot be confused with a case of multi-fluorophore. Two longitudinally separated fluorophores will create independent sets of descriptors and two laterally separated fluorophores differ clearly on at least one descriptor from a single fluorophore.
Algorithm Process Consists Implementing Optical Measurement Methodology
The compound optical process according to at least one embodiment of the invention is the logical complement of the descriptors algorithm. Indeed, the result of the descriptors calculation procedure can lead to the conclusion that an additional image would improve performance of the measurement. The SRCDP microscopy platform allows the acquisition of one—or more additional images from a set of light distribution of the PSIT or PDOS methods.
An example is explained below.
Position Measuring Point by the Method PSIT
PSIT method can be used as a technique for measuring the position of a fluorophore with high precision. This measure can use the descriptors algorithm presented previously.
Consider a fluorophore positioned at the position x, y in Cartesian coordinates and (ρ, θ) in polar coordinates.
A sequence of illumination consisting of a fundamental wave, and a couple of the so-called “half-moon” distributions aligned along orthogonal axes is projected onto the fluorophore.
The preprocessing procedure created two images:
Referring to
The azimuth position can be measured by measuring the intensity ratio between the total intensity emitted by the fluorophore illuminated by the first half-moon distribution, IH, and the total intensity emitted by the fluorophore illuminated by the second half moon distribution, Ive. The ratio between these two intensities is a geometric tangent square law:
Both measures are redundant. This redundancy is a measure to qualify the observed object as a single point and separate it from other objects potentially present in the sample.
Representation in a Higher Dimensional Space: CartesianoPolar Representation
This result can be generalized. We introduce in this paper an entire new representation of a plane, combining the Cartesian representation and the polar representation. We named this representation the CartesianoPolar representation. A point in the plane is represented by a quadruplet: x, y, p, θ. This representation is non-Euclidean and redundant. A similar representation of space can be defined mutatis mutandis.
At first sight this representation seems unnecessary: it is a highly complex representation for a much simpler reality. It is well known that the position of a point in a plane can be represented, alternatively, either by using the Cartesian coordinates, x and y, or either by using polar coordinates p and θ.
Representation in a Higher Dimensional Space: Pythagoras Space
In this paper only the simplified version of the CartesianoPolar representation is detailed, wherein a point with coordinates x, y and p is represented. We named this space the space of Pythagoras.
Defining the geometric area to be a two-dimensional surface in three-dimensional space, which fills the constitutive geometric equation ρ2=x2+y2; assumes a measurement system that simultaneously measures x, y and ρ, as the measurement system such as described in the previous paragraph together with a centroid algorithm on the same data. A point will be physically positioned in the space of Pythagoras, on the geometrical surface. Consider the case of two or more physical points: The center of gravity of the two points of measurement is outside the geometric surface and creates a point outside this area. This representation is a mathematical formalization and generalization of the deterministic algorithm for separating the case of an isolated point from that of an aggregate of points previously described.
Recognition and Measurement of Two Points: A New Resolution Criterion
Consider now two fluorophores positioned symmetrically about the center at positions, ρ, θ and ρ, −θ in polar coordinates. We will use the system described in the previous paragraphs. Three descriptors give the following results:
The centroid measure the centroid of the light distribution, which will be the origin,
As mentioned above, if the value of the descriptor p is not zero, we know that the case study is not a point but two or more. In addition, descriptors p and 0 allow us to measure the characteristics of the two points at a much higher resolution than that defined by the Rayleigh criterion. Moreover, using a compound process it is possible to separate this case from the vast majority of cases of three or more points. An additional light distribution can be projected onto the sample, a half-moon inclined at an angle θ; the assumption of the presence of two points will be confirmed or refuted based on the results of this image. Indeed, the measured energy will be zero for two points, for a line or for a series of dots aligned in the direction of the angle θ.
Control Module
With reference to
The control module, 1100 (shown in
In one embodiment of the PSIT method, regular and singular waves are created by the propagation of a incident regular wave through a uniaxial crystal, replacing the conical crystal 32.
In another embodiment of the method PSIT, regular and singular waves are created by the positioning at the Fourier plane of an optical system of a phase plate—such as a spiral phase plate—or a subwavelength grating, or by positioning a suitable holographic optical element.
In another embodiment of the PSIT method—thick point, not shown, the illumination of the sample comprises a sequence of at least two compact compound light distributions, every compact compound light distribution being composed consisting itself of at least two simple compact light distributions projected simultaneously. Said at least two simple compact light distribution being optically coherent, partially coherent or incoherent relative to each other, said at least two simple compact light distributions being positioned at different spatial positions and said at least two simple compact light distributions differing in at least one of characteristics, such as their central lateral position, their central longitudinal position, their polarization, amplitude or phase. The ensemble of simple compact light distributions contains compact light distributions from different topological families.
In another embodiment of the PSIT method, not shown, compact light distributions are created by different modes of a multimode laser, and the sequence of compact light distributions is created by successively creating modes or, alternatively, by controlling the balance of energy between the modes.
In another embodiment of the PSIT method, not shown, the relationship between regular and singular wave is dynamically changed.
In another embodiment of the PSIT method, not shown, the regular wave and the singular wave are created by a physical separation of the incident beam—in at least—two paths, the transformation in one path, of the regular beam to singular being realized by known means such as phase plates or spiral phase plates, holographic optical element, subwavelength gratings, uniaxial or biaxial crystals or combination thereof, and the recombination of the two beams using a beam combiner into a single beam. In this embodiment, the differentiation of the compact light distributions can be performed either on the combined beam or on each beam, independently after separation and before recombination.
In another embodiment of the method PSIT, dynamic following, not shown, the system comprises means, including but not limited to controllable mirrors, electro-optical or acousto-optical devices or piezoelectric actuators capable to move the compact light distribution or the sequence of compact light distributions in space with high precision. In the system of dynamic monitoring, the position of the compact light distribution and of the sequence is dynamically controlled so as to follow at least one specific target.
In another embodiment of the method PSIT, black fluorophore, not shown, the compact light distribution or a mathematical combination of compact light distributions is configured so that there is zero intensity at the center of the compact light distribution. The system comprises means adapted to move through space the compact light distribution and these means are used to follow the fluorophore and for positioning the fluorophore at its center, a function of time. When the fluorophore is positioned at the center of the compact light distribution, without movement, its position can be measured with high accuracy without fluorescent light emerging from a fluorophore, thereby substantially reducing the effects of photo-bleaching. A movement of the fluorophore can be compensated by appropriate movement of the position of the compact light distribution to follow the fluorophore using a small amount of emitted fluorescent light.
In another embodiment of the method PSIT, dynamic sequences choice, not shown, the system dynamically determines, on the basis of a positioning hypothesis or of a first set of measures, the optimal sequence of compact light distributions.
In another embodiment of the method PSIT, sequences choice and dynamic positioning of the compact light distribution, not shown, the system comprises means, including but not limited to controllable mirrors, electro-optic and acousto-optic devices or piezoelectric actuators, capable of moving in space the compact light distribution, or a combination of compact light distributions with great precision. The system dynamically determines, on the basis of a positioning hypothesis or of a first set of measures, the optimal sequence and the position of the compact light distributions.
In another embodiment of the PSIT method, PSIT method of triangulation, two or more measurement process of the method PSIT, previously described, are carried out on the same sample with different projection axes. The variation in lateral position between the two measurements permits the measurement of the longitudinal position of nanoemitter light.
In another embodiment of the PSIT method, the parallel PSIT method, light is incident on a micro lens array—or other optical means, known to those skilled in the art, allowing the realization of a set of light distributions in parallel, these light distributions being modified by an optical module to perform simultaneously the PSIT method on a large number of discrete points.
In another embodiment of the PSIT method the multispectral PSIT method (not shown), the sample is illuminated sequentially or simultaneously by at least two illumination sequences, each sequence projecting light onto the sample at different wavelengths
In another embodiment of the method PDOS, not shown, the channeling of the incoming light from different point sources according to their longitudinal position is realized in the focal plane. It is carried out using an element having polarization properties dependent on the lateral position. Light entering from a point disposed longitudinally relative to a determined plane, will be incident on a given position and will have specific polarization properties, and the incident light from points located at different longitudinal—and lateral—positions, will be incident on other positions in the focal plane, which have different polarization characteristics.
As to a further discussion of the manner of usage and operation of the invention, it should be apparent from the above description. Therefore, any discussion on the form of the use and operation will not be described.
In this respect, before explaining at least one embodiment of the invention in detail, it is understood that the invention is not limited in its application to the details of construction and arrangements of the components set forth in the following description or illustrated in the drawing. The invention is capable of other embodiments and can be practiced and carried out in various ways. In addition, it is understood that the phraseology and terminology employed herein are for the purpose of description and should not be regarded as limiting.
References cited herein teach many principles that are applicable to the present invention. Therefore, the entire contents of these publications are incorporated herein by reference, as appropriate to the teachings of additional or alternative details, features and/or technical information.
The embodiments of the invention described can be integrated on a fluorescence confocal microscope. Superresolution system according to embodiments of the invention is a new method of measurement, in addition to or in replacement of existing methods of microscopy. However, the superresolution system according to embodiments of the invention may equally be integrated on other microscopy platforms. These microscopy platforms, as described as examples, include but are not limited to: wide field microscopes, Bright field microscope, dark field microscopes, polarization microscopes, phase difference microscopes, differential interference contrast microscopes, stereo microscopes, Raman microscopes, microscopes dedicated to a specific task, such as live cell imaging, cell sorting, cell motility or any other instrument optical microscopy as described for example in Nikon (2011).
It is understood that the invention is not limited in its application to the details set forth in the description contained herein or illustrated in the drawings.
The invention is capable of other embodiments and of being practiced and carried out in various ways. Those skilled in the art will readily understand that various modifications and changes can be applied to the embodiments of the invention as described above without departing from its scope as defined in and by the appended claims.
| Number | Date | Country | Kind |
|---|---|---|---|
| 1004067 | Oct 2010 | FR | national |
This application is a continuation application of pending U.S. patent application Ser. No. 17/087,238, scheduled to issue as U.S. Pat. No. 11,236,992, which is a continuation of U.S. patent application Ser. No. 16/557,030, now issued as U.S. Pat. No. 10,823,551, which is a continuation of U.S. patent application Ser. No. 15/812,016, now issued as U.S. Pat. No. 10,401,153, which is a continuation of U.S. patent application Ser. No. 14/998,051, now issued as U.S. Pat. No. 9,846,030, itself a continuation of U.S. Ser. No. 13/822,355 (issued Feb. 2, 2016 as U.S. Pat. No. 9,250,185), which was a US national phase entry of PCT/FR2011/000555, filed Oct. 14, 2011, through which it claimed the benefit of French application no. FR1004067, filed Oct. 15, 2010. The entire contents of each of the foregoing applications are incorporated by reference herein.
| Number | Name | Date | Kind |
|---|---|---|---|
| 7324273 | Gweon et al. | Jan 2008 | B2 |
| 7541600 | Neuhauser et al. | Jun 2009 | B2 |
| 8084754 | Hell et al. | Dec 2011 | B2 |
| 8514685 | Sirat | Aug 2013 | B2 |
| 8542712 | Rafailov et al. | Sep 2013 | B2 |
| 9250185 | Sirat | Feb 2016 | B2 |
| 9846030 | Sirat | Dec 2017 | B2 |
| 10247931 | Sirat | Apr 2019 | B2 |
| 10401153 | Sirat | Sep 2019 | B2 |
| 10823551 | Sirat | Nov 2020 | B2 |
| 11236992 | Sirat | Feb 2022 | B2 |
| 20020141052 | Iketaki | Oct 2002 | A1 |
| 20030137645 | Fluckiger | Jul 2003 | A1 |
| 20030210405 | Feldman | Nov 2003 | A1 |
| 20040212799 | Hell | Oct 2004 | A1 |
| 20070070496 | Gweon et al. | Mar 2007 | A1 |
| 20080068588 | Hess et al. | Mar 2008 | A1 |
| 20090168613 | Sirat | Jul 2009 | A1 |
| 20090242798 | Bewersdorf et al. | Oct 2009 | A1 |
| 20120104279 | Reuss et al. | May 2012 | A1 |
| 20140145093 | Hendriks et al. | May 2014 | A1 |
| Number | Date | Country |
|---|---|---|
| WO 2009066253 | May 2009 | WO |
| Entry |
|---|
| Abdolvand et al., “Conical refraction Nd:KGd(WO4)2laser,” Opt. Exp., vol. 18, Issue No. 3, pp. 2753-2759 (Feb. 2010). |
| Berry et al., “Conical diffraction: observations and theory,” Proc. R. Soc. A, vol. 462, pp. 1629-1642 (Feb. 2006) |
| Berry, “Conical diffraction from an N-crystal cascade,” J. Opt, vol. 12, Issue No. 8, pp. 1-8 (2010) |
| Boruah, “Lateral resolution enhancement in confocal microscopy by vectorial aperture engineering,” Appl. Optics, vol. 49, Issue No. 4, pp. 701-707 (Feb. 2010) |
| Haeberlé et al., “Saturated structured confocal microscopy with theoretically unlimited resolution,” Opt. Comm., vol. 282, pp. 3657-3664, (2009). |
| Hell, “Far-Field Optical Nanoscopy,” Science, vol. 16, pp. 1153-1158 (May 2007). |
| Keller et al., “Efficient fluorescence inhibition patterns for RESOLFT microscopy,” Opt. Exp, vol. 15, Issue No. 6, pp. 3361-3371 (Mar. 2007) |
| Lunney et al., “The ins and outs of conical refraction,” Europhysics News, vol. 37, Issue No. 3, pp. 26-29 (2006). |
| Oron et al., “The formation of laser beams with pure azimuthal or radial polarization,” Applied Physics Letters, vol. 77, Issue No. 21, pp. 3322-3324 (Nov. 2000). |
| Peet., “Biaxial crystal as a versatile mode converter,” J. Opt., vol. 12, pp. 1-4 (2010). |
| Vlokh et al., “Appearance of Optical Vortex at Conical Refraction. Examples of NaNO2 and YFeO3 Crystals,” Ukr. J. Phys. Opt., vol. 4, Issue No. 2, pp. 90-93 (Jan. 2003) |
| Züchner et al., “Light Microscopy with Doughnut Modes: A Concept to Detect, Characterize, and Manipulate Individual Nanoobjects,” Angewandte Chemie, International Edition, vol. 50, Issue No. 23, pp. 5274-5293 (May 2011) |
| Zhang., “Generation of three-dimensional dark spots with a perfect light shell with a radially polarized Laguerre-Gaussian beam,” Appl. Opt., vol. 49, pp. 6217- 6223 (2010). |
| U.S. Appl. No. 17/087,238, filed Nov. 2, 2020, now U.S. Pat. No. 11,236,992. |
| U.S. Appl. No. 16/557,030, filed Aug. 30, 2019, now U.S. Pat. No. 10,823,551. |
| U.S. Appl. No. 15/012,016, filed Nov. 14, 2017, now U.S. Pat. No. 10,401,153. |
| U.S. Appl. No. 14/998,051, filed Dec. 24, 2015, now U.S. Pat. No. 9,846,030. |
| U.S. Appl. No. 13/822,355, filed Oct. 14, 2011, now U.S. Pat. No. 9,250,185. |
| Number | Date | Country | |
|---|---|---|---|
| 20220155055 A1 | May 2022 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | 17087238 | Nov 2020 | US |
| Child | 17589265 | US | |
| Parent | 16557030 | Aug 2019 | US |
| Child | 17087238 | US | |
| Parent | 15812016 | Nov 2017 | US |
| Child | 16557030 | US | |
| Parent | 14998051 | Dec 2015 | US |
| Child | 15812016 | US | |
| Parent | 13822355 | US | |
| Child | 14998051 | US |
