Patent Application
20220364992
The present invention relates generally to the field of digital imaging and spectroscopy. More particularly, the invention relates to a hardware and software system that measures electromagnetic energy as quantities of photons in distinct wavelength regions across the ultraviolet, visible, and infrared spectrum.
Optical imaging technology is undergoing increasingly rapid advances in hardware and software. The evolution of sensor technology is expanding the range of new applications and improving the measurement capabilities within existing fields of use. The sensitivity of detectors, the spatial resolution, the spectral sensitivity, and the image acquisition speed are a few of the primary characteristics that are being advanced. In the race for newer technologies to be introduced, previous technologies are sometimes overlooked as users seek to stay on the forefront to address their imaging requirements. In some instances, however, previous technologies can be integrated with newer technologies in novel configurations to produce better solutions.
Earlier and evolving technologies must both address the same obstacles and challenges for optical imaging of detecting as much of the available light, or signal, from the target as possible, particularly the low signal intensity. Detecting the low signal intensity involves distinguishing the signal emitted from the target against two primary sources of interference, a) the background emissions from sources surrounding the target and b) the electronic Noise of the sensor system itself. A detector system's light collection efficiency is the first condition that determines if there is enough light reaching the sensor to produce a sufficient dynamic range in the recorded signal for effective image processing and data extraction to isolate the signal of interest. The electronic Noise characteristics of the system are the second condition.
At present there are perceived obstacles and challenges in measuring low light intensities that result in a low Signal to Noise Ratio (SNR) between the photon emissions from a source and the electronic Noise that is inherent in modern detector systems. An adequate SNR is essential for effective signal processing and data interpretation in imaging applications, particularly in low signal settings and single photon counting regimes.
Some of the current obstacles and challenges to measuring low light intensities at adequate SNR are caused by a reduced detection of photons from a target due to the following conditions:
A weak source of illumination on the target which results in the low intensity of photon emissions from the target.
Interference and attenuation of the photons between the target and the sensor caused by scattering, absorption, reflectance and/or other causes of signal reduction that may be found a) immediately surrounding the target, b) in the space between the target and the detector, and/or c) within the detector system due to the materials used in the optical components or the configuration of the optical components that channel the light to the sensor(s).
Short exposure periods that limit the number of photons that reach the sensor during the time allowed.
A reduction of the photons that reach a sensor results in more of the signal falling below the minimum detection threshold of the detector system. The minimum detection threshold is defined by the level of electronic Noise produced by the sensor in the form of dark current, read Noise and other sources of Noise that appear falsely as signal photons. The Noise from the sensor obscures low photon levels from a target amongst the apparent photon counts generated by the Noise.
Existing and previous multispectral and hyperspectral imaging systems cannot provide a suitable level of low light detection and dynamic range across a broad spectrum of ultraviolet, visible, and infrared wavelengths. Those systems without signal amplification depend on higher levels of light intensity to function. Those systems with signal amplification do not provide the same level of light collection efficiency and Noise reduction solutions compared to the current invention.
Other multispectral systems use beam splitters to channel the incoming light into separate paths regardless of wavelength, which results in a loss of signal intensity across the full spectrum by more than 50% at each beam splitter. Those systems then position a band pass filter in front of the sensor, which blocks unwanted wavelengths and allows certain wavelengths to pass through the filter. It is axiomatic that whenever light is split or blocked the signal is reduced, thereby limiting the SNR and the level of signal that can be detected.
Previous multi-sensor systems using image intensifiers or other forms of signal amplification use the same photocathode for each wavelength channel, thereby reducing the light collection efficiency of those systems compared to the current invention.
Previous multi-sensor systems using image intensifiers or other forms of signal amplification use a lens coupling system or tapered fiber optic connection between the phosphor screen of the image intensifier and the image sensor. These also limit the light collection efficiency compared to the current invention that uses a non-tapered fiber optic plate and other forms of collimated optics with waveguide technology to connect the phosphor to the sensor and maximize signal throughput.
The present invention overcomes the aforementioned obstacles. Embodiments of the invention include a hardware and software system that measures electromagnetic energy as quantities of photons in distinct wavelength regions across the ultraviolet, visible, and infrared spectrum. Embodiments of the invention obtain, record, and analyze measurements of electromagnetic energy as quantities of photons in distinct wavelength regions across the ultraviolet, visible, and infrared spectrum.
Embodiments include a multispectral sensor system comprising: an external housing for system components; a plurality of intensified image sensors and non-intensified image sensors; an optical assembly arranged within the interior of the external housing; an input aperture for collecting and channeling photons into the optical assemblies arranged within the interior of the external housing; input aperture optical elements for focusing an input beam of photons into the interior of the external housing through the aperture; and an optical assembly comprising dichroic mirrors and lenses for separating the input beam into a plurality of unique wavelength channels and directing each of the channels to a different sensor of the plurality of sensors. The sensors and optical assembly are designed to maximize the light collection efficiency of a unique range of wavelengths for each of the channels. Each intensified sensor comprises an intensifier and a sensor connected to each of the intensifiers with a photocathode that is optimized for its particular wavelength channel Each non-intensified sensor comprises an image sensor with a photocathode that is optimized for its particular wavelength channel.
In embodiments of the invention, each intensifier preferably includes a photocathode that converts the incoming photons to photoelectrons, one or more microchannel plates (MCP) that multiply the photoelectrons, and a phosphor screen that converts the multiplied photoelectrons back into photons. The choice of the photocathode for each intensifier is determined according to the wavelengths being measured. The choice of the number of MCPs and type of phosphor screen are determined according to a) the Noise in the selected sensor, b) the required frame rate of the photon measurements, and c) the technical application for the device.
Various sensors may be used depending on the technical application. For example, each sensor may comprise a single element sensor, a one-dimensional linear array sensor, or a two-dimensional image sensor.
The sensor measures the number of photons emitted from the phosphor screen and outputs a digital count. The digital count is preferably output to a digital processor for signal processing and analysis and then recorded on digital media. Software algorithms may be employed to enhance the spatial and spectral fidelity of the recorded signals.
Various designs may be used to ensure that photons are focused appropriately and consistently on the sensors. In one embodiment, all of the sensors are located such that the distance along the path from the focusing elements to the image sensor is equidistant for the plurality of sensors. In an alternative embodiment, the sensors are located such that the distance along the path from the focusing elements to the image sensor is not equidistant and optical elements, such as lenses, are used to focus photons on the sensors.
The input aperture optical elements may be in the form of a lens system or in the form of a fiber optic assembly in the form of either a single fiber or a fiber optic bundle, or in the form of a waveguide optical element that collimates the incoming stream of photons.
To minimize any reduction of signal intensity, dichroic mirrors are used to divide the input beam into separate channels, each channel comprising photons of a different and predetermined wavelength range. A dichroic mirror is a mirror with different reflection and transmission properties for different wavelengths. A dichroic mirror may be used as a high efficiency wavelength-dependent beam splitter.
The multispectral sensor system described herein may be used in various applications. In general, the method includes the steps of collecting photons from a subject area through the aperture of an external housing; channeling the photons within an optical assembly and using lenses and dichroic mirrors to separate the photons into a plurality of channels each having a unique range of wavelengths; directing each of the unique wavelength channels to a different sensor that is designed to maximize the light collection efficiency for the unique range of wavelengths for the respective channel; and for each channel using a sensor to convert an analog signal of photons to digital counts. The digital counts may be used to produce an image or to produce a spectral profile. As noted, the digital count is preferably output to a digital processor for signal processing and analysis and then recorded on digital media. A computer system records the measurements as digital data and, depending on the application, uses software algorithms to enhance the spatial and spectral fidelity of the recorded signals. Data analytics may be used to compare the recorded image data to previously stored data that is associated with known conditions to determine correlations between the newly measured data and previous data sets of known conditions.
In embodiments of the invention, the electro-optical sensor hardware is engineered to maximize the light collection efficiency, especially for low light intensities, by using multiple detectors, each of which is optimized individually to maximize its sensitivity to specific wavelength regions of interest. The detector system also employs a variable amplification process that is dependent on the signal intensity so that low signals can be increased for better detection while high signals are amplified less to stay within the dynamic range of the optical sensor that is used to convert the analog signal to a digital value. The invention solves many of the existing problems of low light detection in varying wavelengths and provides new capabilities for data collection and analysis in previously undetectable low signal regimes. The invention applies to a broad array of imaging applications in diverse fields from biomedical imaging to astronomy and remote sensing.
The present invention overcomes the obstacles and challenges of low SNR caused by the Noise of a detector system by several means, including maximizing the light collection efficiency and amplifying the signal with image intensifier technology and photocathode coating technology that is optimized separately for multiple sensors and different wavelength regions.
The optimum selection of detector components, including photocathode, microchannel plates, phosphor screen and image sensor, for different wavelengths, number of channels, incoming signal intensity and spatial resolution requirements, is essential to increasing the light collection efficiency and the SNR of the data.
The selection of photocathode for maximum light collection efficiency must balance a) the Quantum Efficiency of the photocathode substrate that converts incident photons to electrons, and b) the Noise generated from the photocathode in the form of thermionic emissions of electrons. In the current invention photocathodes with the lowest dark current and thermionic emission characteristics are chosen with consideration of the Quantum Efficiency of the photocathode substrate in order to produce the highest SNR from the photocathode.
The optical components are also selected and configured for maximum light collection efficiency. This includes the selection of materials and coatings used in lenses, mirrors, and in certain configurations, dispersion elements, which are optimized individually for each wavelength region that is channeled to a particular sensor.
Dichroic mirrors are used to separate the incoming light waves into different channels while maintaining the highest transmission and reflection efficiency possible.
The invention represents a novel approach to multispectral and hyperspectral imaging through its configuration of multiple image intensifier tubes and non-intensified sensors that are customized separately for enhanced sensitivity to different wavelength regions and controlled with a variable gain for different signal intensities. The system increases the range of photon measurements into the low signal levels that are unattainable by single sensor systems with or without signal amplification as well as multisensor systems that use the same photocathode on all sensors, or those using beam splitters and bandpass spectral filters instead of dichroic mirrors to separate channels.
The current invention's integration of components is critical to the adaptability of the multisensor system for different applications. Different applications require different numbers of sensors and involve different wavelength regions of interest. The current invention can be configured with any number of sensors tailored for the requirements of the application.
The present invention provides a multispectral sensor system that enables new methods of using multispectral imaging. One example is a medical imaging solution for the early detection of cancer. This safe and non-invasive screening protocol will detect and diagnose different forms of cancer and pre-cancerous conditions earlier and more accurately than existing methods. The screening procedure will serve as a digital optical biopsy, eliminating the need for surgical removal of cells and the time and expense of pathology lab examination. The digital measurements of suspected cancer can be collected in seconds using a multispectral sensor system. Data may be uploaded to a database for analysis that can diagnose and report the condition of the suspected tissue within minutes.
New capabilities include, but are not limited to, the following: biomedical imaging applications in, both, clinical and preclinical settings, biomedical sensing of single point photon emissions from fluorescent, bioluminescent, or chemiluminescent probes or labels, such as those used in surgical visualization, genetic sequencing, flow cytometry or other laboratory applications, astronomy applications involving simultaneous multispectral measurements of celestial objects, remote sensing from space-based and/or aerial platforms, and multispectral imaging in low-light settings for navigation, vision, virtual reality, and augmented reality applications.
The invention includes embodiments of a multi-sensor electro-optical detector system designed to be the most sensitive electro-optical detector for measuring low levels of photon emissions across a broad spectrum of wavelengths simultaneously.
In certain configurations the multispectral channels are further divided into hyperspectral channels by the use of dispersion elements positioned before the sensors.
A high potential is applied across the MCP, enabling the photoelectron to accelerate down one of the channels in the disc. When the photoelectron has sufficient energy, it dislodges secondary electrons from the channel walls. These electrons in turn undergo acceleration which results in a cloud of electrons exiting the MCP. Gains in excess of 10,000 can readily be achieved with a single MCP. The degree of electron multiplication is controlled by the gain voltage applied across the MCP.
The sensitivity of photocathodes vary with wavelength. The efficiency with which a photocathode can convert an incident photon to a photoelectron is known as the photocathode's Quantum Efficiency (QE). The QE of each photocathode is one important property to consider when optimizing the different intensified sensors for a specific application.
The output of the intensifier is coupled to the sensor which may be a single element sensor, a one-dimensional linear array sensor, or a two-dimensional image sensor. The intensifier may be coupled to the sensor by a fiber optic plate or waveguide optical element. Using a straight fiber optic plate as opposed to a tapered fiber optic plate or lens system will increase the light collection efficiency of the intensified sensor.
The novel configuration of multiple unique sensors, and the selection and configuration of optical components for channeling the incoming light to each intensified sensor, is designed to maximize the light collection efficiency at all wavelengths. As shown in
The optical components include, but are not limited to, lenses, fiber optics, dichroic mirrors, and in certain configurations dispersion elements such as gratings or prisms. The materials used for each component are selected for maximum light collection efficiency in the specific wavelengths of the spectral channel involving the specific optical component.
The light is input to the housing enclosing the components of the system through an aperture comprising a lens or fiber optic assembly. The incoming light is separated into distinct wavelength channels by one or more dichroic mirrors. The number of mirrors is determined by the number of spectral channels that are required by the application.
The different channels of light are focused onto separate sensors. In the multispectral configurations the light is channeled onto the surface of a photocathode that is the first element of an image intensifier tube. In the hyperspectral configurations additional wavelength dispersion elements are positioned in front of the intensifier's photocathode.
The detector system incorporates image intensifier technology (
To further increase the light collection efficiency of the detector system that employs two-dimensional (2-D) image sensors, the image sensors, which may be CMOS, sCMOS, CCD, multianode pint's, or other type of 2-D array optical sensor, may be fused to the output of the image intensifier's phosphor screen.
The sensors are connected to a common circuit board where their data is transferred and stored for pre-processing (
The multispectral sensor system measures and records the light intensity (brightness) in separate wavelength regions. The light intensity data for each wavelength range is obtained simultaneously so that the data for all of the wavelength ranges may be combined into an enhanced composite image in real-time. The composite image is enhanced by the hardware configuration because the components are each optimized to maximize light collection for their respective wavelength range. The composite image is further enhanced through data processing. The composite image is useful for analog visualization (human interpretation) while the digital data for each wavelength range allows improved, more accurate, low light measurements useful to improve applications such as cancer detection, medical imaging, remote sensing, and imaging of stars and other objects in astronomical observations.
As shown in
As described, the current invention overcomes the obstacles and challenges of low signal to Noise ratio (SNR) caused by the Noise of a detector system by several means, including maximizing the light collection efficiency and amplifying the signal with image intensifier technology that is optimized separately for multiple sensors and different wavelength regions.
Light collection efficiency and the SNR of the data are optimized by selecting detector components (photocathode, microchannel plates, phosphor screen and sensors) that perform best for the respective wavelengths, number of channels, incoming signal intensity, spatial resolution requirements and other criteria that may be application specific.
In particular, the selection of photocathode for maximum light collection efficiency must balance a) the Quantum Efficiency of the photocathode substrate that converts incident photons to photoelectrons, and b) the Noise generated from the photocathode in the form of thermionic emissions of electrons. In the current invention photocathodes with the lowest dark current and thermionic emission characteristics are chosen with consideration of the Quantum Efficiency of the photocathode substrate in order to produce the highest SNR from the photocathode.
The phosphor screen of the intensified sensors are available with different characteristics. The phosphor screen is selected to maximize the performance of each sensor for different applications according to the required rate of photon measurements and the intensity of photon levels.
Three aspects of the phosphor screen are considered, including 1) the Quantum Efficiency at which the phosphor converts electrons to photons, 2) the peak wavelength of the photons emitted by the phosphor, and 3) the decay time of the phosphor emissions. Examples of these characteristics for several phosphor screens are illustrated in
The present invention factors the characteristics of different phosphor screens to maximize the performance and the light collection efficiency of each intensified sensor for the specific application, operating environment, and wavelengths of interest.
The optical components are also selected and configured for maximum light collection efficiency. This includes the selection of materials and coatings used in lenses, mirrors, and in certain configurations, dispersion elements, which are optimized individually for each wavelength region that is channeled to a particular sensor.
Dichroic mirrors are used to separate the incoming light waves into different channels while maintaining the highest transmission and reflection efficiency possible. The photocathode element of each intensified sensor is selected to maximize the performance of each sensor in specific wavelength ranges, or channels. This is achieved primarily by maximizing the ratio between a) the Quantum Efficiency (QE) of the photocathode with b) the thermionic emissions and dark counts, or Noise, that are generated by the photocathode under varying operating conditions.
Additionally, in the selection of different photocathodes for each spectral channel consideration is given to the specific application for the device and the anticipated ambient operating environment. Ambient temperature impacts the Noise that is generated by the photocathode, as does the intensity of the incident photons from the subject being measured. Minimizing the Noise improves the performance of the sensor, and all contributing influences on the Noise must be calculated and factored into the photocathode selection.
The QE of photocathode substrates varies for each wavelength being measured. Therefore, if specific wavelengths of interest are predetermined for a specific application, then the ratio of a) the QE for the specific wavelengths to b) the expected Noise [QE for specific wavelengths: Noise] must be factored into the choice of each photocathode.
The current invention factors the above variables in the selection of each photocathode to maximize the performance and the light collection efficiency of each intensified sensor for the specific application, operating environment, and wavelengths of interest.
Information such as that shown in
Thus, for example, a multispectral intensified sensor system according to one embodiment of the invention would use distinct photocathodes for different wavelength ranges in separate channels.
Embodiments of the invention provide novel systems and methods for multispectral and hyperspectral imaging through its configuration of multiple image sensors that are customized separately for enhanced sensitivity to different wavelength regions and controlled with a variable gain for different signal intensities. Certain applications are made practical only because the system increases the range of photon measurements into the low signal levels that are unattainable by single sensor systems with or without signal amplification as well as multisensor systems that use the same photocathode on all sensors, or those using beam splitters and bandpass spectral filters instead of dichroic mirrors to separate channels.
As such, use of the system enables novel methods in various applications. Different applications require different numbers of sensors and involve different wavelength regions of interest. As described above, embodiments of the invention can be configured with any number of sensors tailored for the requirements of the application.
Applications for the multispectral sensor system described herein include (without limitation): biomedical imaging applications in, both, clinical and preclinical settings; biomedical sensing of single point photon emissions from fluorescent, bioluminescent, or chemiluminescent probes or labels, such as those used in genetic sequencing; flow cytometry or other laboratory applications; astronomy applications involving simultaneous multispectral measurements of celestial objects; remote sensing from space-based and/or aerial platforms; and multispectral imaging in low-light settings for navigation, vision, virtual reality, autonomous vehicle operation and augmented reality applications.
When used for these applications, the method includes the steps of collecting photons from a subject area through the aperture; channeling the photons within the housing and using lenses and dichroic mirrors to separate the photons into a plurality of channels each having a unique range of wavelengths; directing each of the unique wavelength channels to a different image sensor that is designed to maximize the light collection efficiency and Quantum Efficiency for the unique range of wavelengths for the respective channels; and for each channel using a sensor to convert an analog signal of photons to digital counts. The digital counts may be used to produce an image or to produce a spectral profile. The digital count is preferably output to a digital processor for signal processing and analysis and then recorded on digital media. An embedded computer processing unit records the measurements as digital data and, depending on the application, uses software algorithms to enhance the spatial and spectral fidelity of the recorded signals. Data analytics may be used to compare the recorded image data to previously stored image data that is associated with known conditions to determine correlations between the image data and known conditions.
The method of cancer detection using a multispectral sensor system that includes a plurality of sensors and optical assemblies arranged within the interior of a housing as described herein may further comprise the step of using the output digital counts to produce a spectral profile. In addition, the method may further comprise the step of outputting the digital counts to a processor for signal analysis to produce image data recorded on digital media and using data analytics to compare the recorded image data to previously stored image data that is associated with known conditions to determine correlations between the image data and known conditions.
Embodiments of the invention also include a method of genetic sequencing using a multispectral sensor system that includes a plurality of intensified sensors and optical assemblies arranged within the interior of a housing; an input aperture for collecting and channeling photons into the optical assemblies arranged within the interior of the housing; focusing elements for focusing an input beam of photons into the interior of the housing through the aperture; the method comprising the steps of: collecting photons from a subject through the aperture; channeling the photons within the housing and using lenses and dichroic mirrors to separate the photons into a plurality of channels each having a unique range of wavelengths; directing each of the unique wavelength channels to a different intensified sensor that is designed to maximize the light collection efficiency for the unique range of wavelengths for the respective channels; and for each channel using an intensified sensor to convert an analog signal of photons to digital counts.
Embodiments of the invention also include a method of flow cytometry using a multispectral sensor system that includes a plurality of intensified sensors and optical assemblies arranged within the interior of a housing; an input aperture for collecting and channeling photons into the optical assemblies arranged within the interior of the housing; focusing elements for focusing an input beam of photons into the interior of the housing through the aperture; the method comprising the steps of: collecting photons from a subject area through the aperture; channeling the photons within the housing and using dichroic mirrors and lenses to separate the photons into a plurality of channels each having a unique range of wavelengths; directing each of the unique wavelength channels to a different intensified sensor that is designed to maximize the light collection efficiency for the unique range of wavelengths for the respective channels; and for each channel using an intensified sensor to convert an analog signal of photons to digital counts.
Embodiments of the invention also include a method of using a multispectral sensor system in astronomy where the system includes a plurality of sensors and optical assemblies arranged within the interior of a housing; an input aperture for collecting and channeling photons into the optical assemblies arranged within the interior of the housing; focusing elements for focusing an input beam of photons into the interior of the housing through the aperture; the method comprising the steps of: collecting photons from a subject through the aperture; channeling the photons within the housing and using lenses and dichroic mirrors to separate the photons into a plurality of channels each having a unique range of wavelengths; directing each of the unique wavelength channels to a different intensified or non-intensified sensor that is designed to maximize the light collection efficiency for the unique range of wavelengths for the respective channels; and for each channel using a sensor to convert an analog signal of photons to digital counts.
The foregoing disclosure of the preferred embodiments of the present invention has been presented for purposes of illustration and description. It is not intended to be exhaustive or to limit the invention to the precise forms disclosed. Many variations and modifications of the embodiments described herein will be apparent to one of ordinary skill in the art in light of the above disclosure. The scope of the invention is to be defined only by the claims appended hereto, and by their equivalents.
| Number | Date | Country | |
|---|---|---|---|
| 63188169 | May 2021 | US |
