Patent Application
20220026697
This invention relates to an apparatus, system, and related methods for the conversion of a regular bright-field microscope into a PS-QPI microscope system (Polarization-Sensitivity and Quantitative Phase Imaging).
Most unstained biological specimens have poor intrinsic contrast hampering their imaging using bright-field microscopy. However, these transparent samples induce phase shifts to the transmitted light wave. Microscopy techniques, including phase contrast, differential interference, contrast (DIC), and Hoffman modulation contrast (HMC) have been developed to translate these phase variations into intensity changes. However, these techniques do not enable accurate imaging of the 3D shape of live cells in complex 3D spatial environments. This limitation results in a significant gap in our understanding of dynamic changes occurring in the 3D cell shape and behavior of unstained specimens. Over the last decades, several quantitative phase imaging (QPI) methods have been investigated. Among all quantitative phase imaging (QPI) techniques, digital holographic microscopy (DHM) is one of the most promising to perform live-cell imaging while providing topographical information on the specimen. The five most key advantages of DHM over other QPI methods are high speed, stability, accuracy, spatial resolution, and polarization sensitivity. Temporal-phase stability is identified as one of the most important figures of merit in QPI. As any interferometric technique, all DHM systems are subject to temporal instabilities caused by vibrations and other disturbances on the systems. Among all DHM configurations, common-path DHM systems provide the highest temporal stability, allowing sub-nanometer path-length temporal sensitivity to study dynamic events in live biological specimens. High-speed image acquisition is also critical to capture those rapid dynamic events and investigate how they are affected by external perturbations. Off-axis DHM systems are currently the fastest DHM systems since only a single recorded image is required to provide a quantitative phase image. Although all off-axis DHM systems provide rapid imaging, not all of them provide high-resolution and accurate phase images since the majority of off-axis DHM systems operate in non-telecentric mode. The reconstruction of diffraction-limited (i.e., high-resolution) DHM images can be ensured if off-axis DHM systems work at the diffraction limit. On the other hand, because several quantitative biological parameters such as the integral intracellular refractive index can be estimated from a phase measurement, a vital property of any QPI system is that its response should be accurate and shift-invariant. For this reason, telecentric-based DHM systems are the most suitable QPI systems.
Ultimately, polarization-sensitive (PS) imaging systems are advantageous in biological imaging since they enable determining the isotropic properties of different cellular and extracellular molecular components, such as microtubules and amyloid. PS systems have increased the understanding of cell biological processes, and they have been used for clinical diagnosis.
Accordingly, what is needed is a common-path polarization-sensitive (PS) DHM system and approach that addresses the above deficiencies in the prior art.
In various exemplary embodiments, the present invention comprises a polarization-sensitive quantitative phase imaging (PS-QPI) module that can be implemented in any commercial bright-field microscope. The present invention provides quantitative phase images and birefringence measurements with high temporal sensitivity, speed, accuracy, and spatial resolution. The QPI method of this invention employs a Fresnel biprism.
In several embodiments, the PS-QPI module provides the self-interference of the two replicas of the sample's image generated by the Fresnel biprism. The sample can be illuminated by a spatially- and temporally-coherent source. The light source employed may comprise a laser or a quasi-monochromatic light-emitting diode (LED), wherein the spatial incoherence of the light emission is broken using, for example, a spatial filter. Alternatively, the light source may be a white lamp with a narrow bandwidth filter. Additional light sources may be appropriate for use with the present illumination system in various embodiments. The PS capability is achieved by introducing a polarizing element after the illumination source so the polarization state of the illuminating beam can be changed arbitrarily. The sample is illuminated from a beam whose polarization state can be rotated. The polarization state of the illuminating beam can be as simple as a linear polarization or an elliptical polarization state.
In various exemplary embodiments, the present invention comprises an apparatus, system, and related methods for converting a regular bright-field microscope into a Polarization Sensitive Quantitate Phase Imaging (PS-QPI) system. A bright-field microscope, also known as a compound light microscope, is an optical microscope that uses light rays to produce a dark image against a bright background, and is the standard microscope used in biological and biomedical laboratory studies. A typical bright-field microscope comprises an illumination or light source 10, a condenser that focuses a beam of light onto a specimen or sample 30, a moveable x-y stage 20 (movable by stage controls) for holding the specimen, an aperture diaphragm or contrast control that controls the diameter of the beam of light that passes through the condenser to create high or low contrast, a plurality of microscope objective lenses (MO) 50 which receive and collect the light passing through the specimen or sample 30 and form a magnified image of the specimen or sample, one or more focusing knobs or controls (e.g. Z-axis screw knob) 40 to move the stage and/or objective lenses with respect to each other, and an eyepiece 90 with two eyepiece or ocular lenses for observing the image. The image may also be directed to and captured by a sensor (such as a camera sensor) or imaging apparatus 100 (a folded mirror 82 or similar apparatus may be used to direct the image as desired). A long tube, sometimes known as the body tube, generally connects the objective lens 50 and the ocular lenses and/or sensor or imaging apparatus 100. A tube lens or lenses 60 are contained in the tube. Depending on the configuration of the system, a rotatable filter wheel 70 may be used to apply a desired filter to the light beam and image, and one or more mirrors 80 may be used to re-direct the light beam and image. Additional lenses, such as converging lenses, may be used as well.
Quantitative Phase Imaging (QPI) and Polarization-Sensitivity (PS) are two methods widely used in biological and biomedical applications. The present invention converts a regular bright-field microscope by insertion of a polarizing element 110 in the illumination path and a Fresnel biprism 120 in the image space of the system. In comparison with the prior art, the present invention achieves polarization sensitivity (PS) capability and is unique in being easily be integrated into a commercial microscope. Other advantages of the invention include, but are not limited to, high speed, accuracy, spatial resolution, and temporal stability. The above features make the present invention suitable for real-time quantitative PS phase imaging in a broad number of biological applications aimed at understanding cell growth and dynamic changes occurring during physiological processes, as well as identification of cell/tissue screening and diagnosis.
As seen in
To provide PS measurements, the imaged sample should be illuminated with polarized light whose plane-of-vibration is varied in a controlled way. A polarizing element mounted in a rotational mount is used to arbitrarily select the plane of polarization of the light incident in the sample. Alternate polarizing elements that provide polarized light whose plane of polarization can be rotated may be used.
In the image path of the microscope after the tube lens, the Fresnel Biprism is inserted to generate two polarization preserving images of the magnified image. In this embodiment, the digital hologram is directly generated by the superposition of the two images formed by the Fresnel biprism without any filtering. This embodiment has fewer elements, leading to a compact and low-cost PS-DHM module.
In the illumination path of
The embodiment shown in
A further exemplary embodiment, as seen in
The QPI capability is introduced by the use of a common-path Fresnel biprism 120 interferometer inserted in the light path after the tube lens 60. Because the image plane generated by the tube lens is inaccessible as it is located within the tube or body of the microscope, a 4f relay system is added to locate this image plane onto the camera sensor 110 and/or through the eyepiece. A 4f relay system generally comprises two lenses, a first lens and a second lens. The input plane is one focal length (f) in front of the first lens, while the output lane is located one focal length after the second lens. In between the two lenses is the Fourier plane, with the Fourier transformation of the object placed at the output plane. In several embodiments, the Fresnel biprism 120 is in inserted in the image space of the second lens of the 4f relay system (i.e., at or beyond the output plane of the second lens, between the second lens and the camera sensor 100). Since the position of the Fresnel biprism to achieve the highest FOV is independent of the objective lens used, the Fresnel biprism also may be mounted in a slider 122, which the user can manipulate to set the PS-QPI imaging capabilities “on” and “off”.
In another embodiment for microscopes that have an available side port behind the tube lens, the present system may be implemented by inserting a 4f relay system attached to the side port. In this this implementation, one must ensure that the tube lens and the first lens of the 4f relay system operate in a telecentric regime. The achievement of this condition results in placing the image plane of the microscope at the back focal plane of the second lens of the 4f relay system, where an auxiliary digital imaging sensor may be inserted. Once this relay system has been established, the biprism is inserted between the second lens of the 4f and the plane of the auxiliary sensor.
The invention thus provides accurate quantitative PS phase images (PS-QPI) in a simple, compact, and cost-effective format. The low cost of this implementation (e.g., a few hundred dollars) of this architecture makes the PS-QPI technique accessible to most laboratories with standard bright-field microscopes without the cost of a standard PS-QPI system.
In further embodiments, the present invention uses a computer-based computational approach to reconstruct PS-DINT (polarization-sensitive digital holographic microscopy) images, as shown in
In several embodiments, the present invention thus comprises a polarization-sensitive quantitative phase imaging system comprising of a spatially and temporally coherent illumination source, a polarizing element that can arbitrarily orient the polarization state of the source, a microscope system comprising of a microscope objective lens and tube lens that magnifies the image of the sample, a Fresnel biprism that generates two copies of the magnified image, an optional afocal imaging system comprised of two converging lenses and a pinhole located at the common Fourier plane, and a 2-dimensional detector that can record an image to a computer. The spatial and temporally coherent illumination source in the claim can be any coherent source such as diode lasers, gas lasers, discharge tubes, light-emitting diodes (LED) with limited spectral bandwidth, or wide-band light source including an incandescent, fluorescent light source with appropriate bandpass filters to limit the spectral bandwidth. The polarizing element can be any optical element or elements including one or more linear polarizers (such as, but not limited to, polaroid films, grid polarizers, and the like), waveplates (e.g., quarter-wave, half-wave, and so on), optical modulators (e.g., spatial light modulator, acoustic or electric optical modulators), or depolarizers and scramblers that may be mounted in a rotational mount to modify the polarization state of the illumination source arbitrarily. The microscope system comprises a microscope objective and a tube lens that magnifies the image of the sample that may or may not be part of an existing microscope system. The Fresnel biprism is an optical element consisting of two thin prisms joined at the base to form an isosceles prism that generates two copies of the magnified image that propagate with some angle to each other. The optical system comprises two converging lenses that can magnify the image. The pinhole is a spatial filter placed at the common Fourier plane of the optional optical system that filters the medium and high frequency content of one of the beams generated by the Fresnel biprism. The system forms a digital hologram by the coherent superposition or interference by the two copies of the images created by the Fresnel biprism. The detector is any 2-dimensional detector such as a CCD, CMOS, or other two-dimensional sensor that can record the digital hologram to a computer or a computer-readable storage device. The sample in the claim is illuminated by light from the source, filter (if necessary), and polarizing element either by light transmitting through the sample and/or from light from the source, filter (if necessary), and polarizing element reflected off the sample with the help of a beam splitter placed in the optical path between the TL lens and the Fresnel biprism. Elements of the system can be provided as external PS-QPI module that can be added to and adapted to any commercial microscope.
The apparatus of the present invention measures the polarimetric properties of microscopic birefringent samples by estimating their retardance map. A polarization-sensitive hologram is recorded by the system in a single-shot way, which is suitable for applications including live cell imaging. The computational method, as described above, comprises: the acquisition and processing of polarization-sensitive holograms by means of sequentially rotating the polarization state of the illumination in a controlled way; acquisition of polarization-sensitive holograms by rotating the polarization state of the illuminating light; computation of the phase image for each recorded hologram; and estimation of the retardance map by calculating the phase difference between the extraordinary and ordinary phases. The computation of the phase image for each hologram can be performed by digitally processing the recorded hologram using computational methods, including, but not limited to, applying fast Fourier transforms, Fourier filtering, numerical compensation approaches to eliminate the interference angle, numerical focusing algorithm(s), unwrapping phase algorithm(s), machine learning methods, and deep learning methods. The retardance map of the claim is determined and provided by subtracting the maximum and minimum values of the whole set of reconstructed phase images at each pixel. The recording process of the present invention may be optimized using machine learning and/or deep learning approaches.
The above features make the present invention suitable for real-time quantitative PS phase imaging in a broad number of biological applications. The performance of the system has been verified experimentally on both calibrated phase objects and cultured cells.
These embodiments, as well as other exemplary embodiments, as well as the tools and programs referenced above, are described in detail in the attached materials which are attached hereto and incorporated herein in their entirety (including all text and figures therein) by specific reference:
In order to provide a context for the various computer-implemented aspects of the invention, the following discussion provides a brief, general description of a suitable computing environment in which the various aspects of the present invention may be implemented. A computing system environment is one example of a suitable computing environment, but is not intended to suggest any limitation as to the scope of use or functionality of the invention. A computing environment may contain any one or combination of components discussed below, and may contain additional components, or some of the illustrated components may be absent. Various embodiments of the invention are operational with numerous general purpose or special purpose computing systems, environments or configurations. Examples of computing systems, environments, or configurations that may be suitable for use with various embodiments of the invention include, but are not limited to, personal computers, laptop computers, computer servers, computer notebooks, hand-held devices, microprocessor-based systems, multiprocessor systems, TV set-top boxes and devices, programmable consumer electronics, cell phones, personal digital assistants (PDAs), tablets, smart phones, touch screen devices, smart TV, internet enabled appliances, internet enabled security systems, internet enabled gaming systems, internet enabled watches; internet enabled cars (or transportation), network PCs, minicomputers, mainframe computers, embedded systems, virtual systems, distributed computing environments, streaming environments, volatile environments, and the like.
Embodiments of the invention may be implemented in the form of computer-executable instructions, such as program code or program modules, being executed by a computer, virtual computer, or computing device. Program code or modules may include programs, objects, components, data elements and structures, routines, subroutines, functions and the like. These are used to perform or implement particular tasks or functions. Embodiments of the invention also may be implemented in distributed computing environments. In such environments, tasks are performed by remote processing devices linked via a communications network or other data transmission medium, and data and program code or modules may be located in both local and remote computer storage media including memory storage devices such as, but not limited to, hard drives, solid state drives (SSD), flash drives, USB drives, optical drives, and internet-based storage (e.g., “cloud” storage).
In one embodiment, a computer system comprises multiple client devices in communication with one or more server devices through or over a network, although in some cases no server device is used. In various embodiments, the network may comprise the Internet, an intranet, Wide Area Network (WAN), or Local Area Network (LAN). It should be noted that many of the methods of the present invention are operable within a single computing device.
A client device may be any type of processor-based platform that is connected to a network and that interacts with one or more application programs. The client devices each comprise a computer-readable medium in the form of volatile and/or nonvolatile memory such as read only memory (ROM) and random access memory (RAM) in communication with a processor. The processor executes computer-executable program instructions stored in memory. Examples of such processors include, but are not limited to, microprocessors, ASICs, and the like.
Client devices may further comprise computer-readable media in communication with the processor, said media storing program code, modules and instructions that, when executed by the processor, cause the processor to execute the program and perform the steps described herein. Computer readable media can be any available media that can be accessed by computer or computing device and includes both volatile and nonvolatile media, and removable and non-removable media. Computer-readable media may further comprise computer storage media and communication media. Computer storage media comprises media for storage of information, such as computer readable instructions, data, data structures, or program code or modules. Examples of computer-readable media include, but are not limited to, any electronic, optical, magnetic, or other storage or transmission device, a floppy disk, hard disk drive, CD-ROM, DVD, magnetic disk, memory chip, ROM, RAM, EEPROM, flash memory or other memory technology, an ASIC, a configured processor, CDROM, DVD or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium from which a computer processor can read instructions or that can store desired information. Communication media comprises media that may transmit or carry instructions to a computer, including, but not limited to, a router, private or public network, wired network, direct wired connection, wireless network, other wireless media (such as acoustic, RF, infrared, or the like) or other transmission device or channel. This may include computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism. Said transmission may be wired, wireless, or both. Combinations of any of the above should also be included within the scope of computer readable media. The instructions may comprise code from any computer-programming language, including, for example, C, C++, C#, Visual Basic, Java, and the like.
Components of a general purpose client or computing device may further include a system bus that connects various system components, including the memory and processor. A system bus may be any of several types of bus structures, including, but not limited to, a memory bus or memory controller, a peripheral bus, and a local bus using any of a variety of bus architectures. Such architectures include, but are not limited to, Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MCA) bus, Enhanced ISA (EISA) bus, Video Electronics Standards Association (VESA) local bus, and Peripheral Component Interconnect (PCI) bus.
Computing and client devices also may include a basic input/output system (BIOS), which contains the basic routines that help to transfer information between elements within a computer, such as during start-up. BIOS typically is stored in ROM. In contrast, RAM typically contains data or program code or modules that are accessible to or presently being operated on by processor, such as, but not limited to, the operating system, application program, and data.
Client devices also may comprise a variety of other internal or external components, such as a monitor or display, a keyboard, a mouse, a trackball, a pointing device, touch pad, microphone, joystick, satellite dish, scanner, a disk drive, a CD-ROM or DVD drive, or other input or output devices. These and other devices are typically connected to the processor through a user input interface coupled to the system bus, but may be connected by other interface and bus structures, such as a parallel port, serial port, game port or a universal serial bus (USB). A monitor or other type of display device is typically connected to the system bus via a video interface. In addition to the monitor, client devices may also include other peripheral output devices such as speakers and printer, which may be connected through an output peripheral interface.
Client devices may operate on any operating system capable of supporting an application of the type disclosed herein. Client devices also may support a browser or browser-enabled application. Examples of client devices include, but are not limited to, personal computers, laptop computers, personal digital assistants, computer notebooks, hand-held devices, cellular phones, mobile phones, smart phones, pagers, digital tablets, Internet appliances, and other processor-based devices. Users may communicate with each other, and with other systems, networks, and devices, over the network through the respective client devices.
Thus, it should be understood that the embodiments and examples described herein have been chosen and described in order to best illustrate the principles of the invention and its practical applications to thereby enable one of ordinary skill in the art to best utilize the invention in various embodiments and with various modifications as are suited for particular uses contemplated. Even though specific embodiments of this invention have been described, they are not to be taken as exhaustive. There are several variations that will be apparent to those skilled in the art.
This application claims benefit of and priority to U.S. Provisional App. No. 63/054,354, filed Jul. 21, 2020, which is incorporated herein by specific reference in its entirety for all purposes.
| Number | Date | Country | |
|---|---|---|---|
| 63054354 | Jul 2020 | US |
