Patent Application
20230210377
The current invention relates to the use of a neural network to improve the quality of images obtained from light scattered by an intermediate object that scatters light, such as tissue or a frosted screen. This invention also relates to detection means and apparatus used in said methods, as well as to quantum dots useful in said use.
The listing or discussion of a prior-published document in this specification should not necessarily be taken as an acknowledgement that the document is part of the state of the art or is common general knowledge.
The development of targeted therapy or immunotherapy drugs for cancer have typically relied on the identification of antibodies that could inhibit cancer cell growth, deliver payload of drugs to targeted cells, or to activate a targeted immune response on the cancer cells, thereby killing cancer cells without causing damage to healthy cells. Visible fluorescent probes have served as the workhorse in the in vitro and in vivo testing of drug efficacy, since the detection of fluorescent-tagged antibodies around cancer tissues, typically in animal tests, would indicate the successful development of an antibody that is specific in targeting the cancer cells.
However, existing fluorescence imaging methods have significant shortcomings, where both the excitation radiation and the fluorescence signal have difficulties penetrating deep into and out of biological tissues due to the intense absorption and scattering of visible light by the skin, tissues and biological fluids (Weissleder, R., Nat. Biotechnol. 2001, 19, 316-317; and Frangioni, J. V., Curr. Opin. Chem. Biol. 2003, 7, 626-634). Hence, such fluorescence testing methods have been limited to the use of nude mice, which are typically smaller than the size of a child's palm. However, murine (mice) models for cancer immunotherapy studies are vastly inadequate due to the significant differences between human and mice immunology, hence leading to high failure rates in the translation of preclinical studies in mice to clinical trials in humans. The ability to probe cancer drug efficacy in larger animals, such as rabbits or pigs, that possess greater homology with humans could therefore be valuable in cancer drug developments. Nonetheless, due to the depth limitations of existing fluorescence imaging methods, the detection of deeper cancers in larger animals have required magnetic resonance imaging (MRI) or positron emission tomography (PET) techniques, which are prohibitively expensive for cancer drug screening.
Therefore, there is an urgent need for an alternative deep imaging technique that could be deployed, at low cost, for immunotherapy studies and targeted drug development.
There is also a need for improved non-invasive imaging devices and techniques.
Aspects and embodiments of the current invention are provided in the following numbered clauses.
1. A computerized method for processing scattered images obtained by imaging through scattering media, comprising:
In(Zn)As—In(Zn)P—GaP—ZnS
wherein:
and
(d) the atomic percentages in the quantum dot are as follows: In from 35 to 45%; As from 1 to 5%; P from 25 to 35%; Zn from 5 to 10%; Ga from 5 to 9%; and S from 8 to 15%, optionally wherein the atomic percentages in the quantum dot are as follows: In from 39.7 to 39.8%; As from 2.2 to 2.3%; P from 31.3 to 31.4%; Zn from 8.5 to 8.6%; Ga from 7.5 to 7.6%; and S from 10.3 to 10.4%.
15. An imaging device comprising:
In(Zn)As—In(Zn)P—GaP—ZnS
wherein:
In(Zn)As—In(Zn)P—GaP—ZnS
wherein:
In embodiments herein, the word “comprising” may be interpreted as requiring the features mentioned, but not limiting the presence of other features. Alternatively, the word “comprising” may also relate to the situation where only the components/features listed are intended to be present (e.g. the word “comprising” may be replaced by the phrases “consists of” or “consists essentially of”). It is explicitly contemplated that both the broader and narrower interpretations can be applied to all aspects and embodiments of the present invention. In other words, the word “comprising” and synonyms thereof may be replaced by the phrase “consisting of” or the phrase “consists essentially of” or synonyms thereof and vice versa.
The phrase, “consists essentially of” and its pseudonyms may be interpreted herein to refer to a material where minor impurities may be present. For example, the material may be greater than or equal to 90% pure, such as greater than 95% pure, such as greater than 97% pure, such as greater than 99% pure, such as greater than 99.9% pure, such as greater than 99.99% pure, such as greater than 99.999% pure, such as 100% pure.
The methods and apparatus disclosed herein may make use of any suitable nanocrystalline material that capable of fluorescing upon excitation from light from a near-infrared light source. Said materials may be quantum dots or, more particularly, giant shell quantum dots. For example, the methods and apparatus disclosed herein may make use of giant shell quantum dots capable of fluorescing upon excitation from light from a near-infrared light source. Said giant shell quantum dots may have the formula:
In(Zn)As—In(Zn)P—GaP—ZnS
wherein:
Depending on the desired application for the quantum dots, the ZnS outer layer may may further comprise an organic compound that may be hydrophilic or hydrophobic (or may have both). Hydrophobic organic compounds that may be mentioned herein include, but are not limited to oleic acid. Hydrophilic organic compounds include, but are not limited to, mercaptosuccinic acid. When a hydrophilic organic compound (e.g. mercaptosuccinic acid) is present, it may be further functionalised with a biological targeting agent (e.g. the biological targeting agent may be selected from one or more of the group consisting of folic acid and a cancer-specific antibody). As will be appreciated, the presence of targeting agents may make the giant shell quantum dots so functionalised suitable for use in vivo (e.g. for diagnostic purposes on a human or animal or for research purposes, such as determining the location of a tumour in a subject animal).
The giant shell quantum dots may display any suitable emission peak. For example, the giant shell quantum dots may display an emission peak at from 820 to 850 nm, such as from 828 to 837 nm, such as 828 nm or 837 nm. Additionally or alternatively, giant shell quantum dots may display any suitable photoluminescence lifetime, for example, the quantum dots may display a photoluminescence lifetime of from 20 to 100 ns, such as from 30 to 70 ns, such as from 40 to 60 ns, such as 59 ns. Additionally or alternatively, the giant shell quantum dots may absorb light at any suitable wavelength (i.e. in the near-IR range), for example, the giant shell quantum dots may absorb light at a wavelength of from 400 to 800 nm. Additionally or alternatively, the giant shell quantum dots may display any suitable photoluminescence quantum efficiency. For example, the giant shell quantum dots may display a photoluminescence quantum efficiency of from 60 to 75%.
The giant shell quantum dots may have any suitable size. For example, the quantum dots may have:
The atomic percentages in the giant shell quantum dots may be as follows: In from 35 to 45%; As from 1 to 5%; P from 25 to 35%; Zn from 5 to 10%; Ga from 5 to 9%; and S from 8 to 15%. More particularly, the atomic percentages in the giant shell quantum dots may be as follows: In from 39.7 to 39.8%; As from 2.2 to 2.3%; P from 31.3 to 31.4%; Zn from 8.5 to 8.6%; Ga from 7.5 to 7.6%; and S from 10.3 to 10.4%.
The giant shell quantum dots described herein may be used in any of the applications discussed hereinbelow.
In an aspect of the invention (see
While not necessary, the array 130 may be positioned on a moveable platform 150 such that the array is movable relative to the near-infrared light source and/or a light beam from near-infrared light source is moveable relative to the array.
As will be appreciated, any suitable means of moving the light beam may be used and are well-known. For example, the light source apparatus may be on a moveable apparatus, or the light beam itself may be manipulated using standard techniques using optics.
When used herein, the term “array” simply refers to an arrangement of nanocrystals that can provide the desired effect. For example, the array may be presented as a panel of nanocrystals, such as a panel of giant shell quantum dots.
Any suitable nanocrystals capable of generating the effect listed above may be used, such as quantum dots. More particularly, any suitable giant shell quantum dots capable of generating the effect listed above may be used. Examples of such quantum dots are disclosed hereinbefore.
Any suitable near-infrared light source may be used in this device. For example, the near-infrared light source may be a laser capable of emitting light at near-infrared wavelengths (e.g. 721 nm). Any suitable light directing means or apparatus may be used, for example, the light directing means or apparatus comprises a mirror. Other materials that could be used include optical fibers and the like, as well as combinations.
As will be appreciated, the detecting means or apparatus may further comprise an imaging apparatus. More particularly, the detecting means or apparatus may further comprise an imaging processing unit. The imaging apparatus and/or imaging processing unit may be used to provide an image. This image may be enhanced by the use of the neural network disclosed herein.
As depicted in
Thus, in a further aspect of the invention, there is disclosed a method of imaging a part or the whole of a human or animal body, using an imaging device, comprising:
The method disclosed above may further comprise the step of capturing an image of the part or whole of the human or animal body to be imaged based on the detected fluorescent light, the image being a scattered image. The method may further comprise the step of processing the scattered image using the computerized method described herein to enhance the scattered image.
As will be appreciated, the device used in this method may be the imaging method described hereinbefore.
As noted before, the nanocrystals (and more particularly the quantum dots) disclosed herein may also be used in in vivo applications. Thus, there is also disclosed a method of diagnosis comprising the steps of:
(a) supplying a plurality of nanocrystals (e.g. giant shell quantum dots) capable of fluorescing upon excitation from light from the near-infrared light source to a subject;
(b) subjecting a target site on the subject to light irradiation; and
(c) detecting a signal, or lack thereof, generated by the nanocrystals to provide a diagnosis.
Any suitable nanocrystals capable of generating the effect listed above may be used. More particularly, any suitable giant shell quantum dots capable of generating the effect listed above may be used. Examples of such quantum dots are disclosed hereinbefore. More particularly, the quantum dots may be one that are functionalised with a biological targeting agent (e.g. the biological targeting agent may be selected from one or more of the group consisting of folic acid and a cancer-specific antibody).
The method may be one in which there is a further step of capturing an image of the target site based on the detected signal, the image being a scattered image. Additionally or alternatively, the method may further use a step of processing the scattered image using the computerized method described herein to enhance the scattered image for the diagnosis.
Thus, in a further aspect of the invention, there is disclosed a computerized method for processing scattered images obtained by imaging through scattering media. For example, the scattered images are fluorescence images that may be captured as a result of light fluorescing from the quantum dots. The computerized method comprises steps of:
The computerized method further comprises training the neural network model, said training comprising:
It will be appreciated that the scattering media may be any suitable scattering media. For example, it may be the flesh covering the bones of a human or animal, or it may be a frosted pane of glass, obscuring an object behind it.
Further aspects and embodiments of the invention will now be described by reference to the following non-limiting examples. As will be appreciated, the exact methods used for each of the above aspects and embodiments may be derived directly or by analogy from the following examples.
Materials
Indium acetate (99.99%), zinc acetate (99.99%), gallium (III) chloride (99.99%), sulfur (99.5%), trioctylphosphine (TOP, 97%), zinc acetate dihydrate (99%), ammonium hydroxide solution (28-30% v/v in water), mercaptosuccinic acid (MSA, 99%), tri-p-tolyl phosphine (98%), tricyclo[5.2.1.02,6]decanedimethanol diacrylate (TCDDA), paraformaldehyde (PFA, reagent grade) and 2,2-bis(hydroxymethyl)propionic acid (DMPA, 98%) were purchased from Sigma-Aldrich, and used without further purification. 1-octadecene (ODE, 90%) was purchased from Sigma-Aldrich, and dried with activated molecular sieves in a round-bottom flask (RBF) and degassed under vacuum for 30 minutes before use. Both octylamine (99%) and oleic acid (OA, 90%) were purchased from Sigma-Aldrich, and degassed under vacuum before use. 3-(4,5-Thiazolyl Blue tetrazolium bromide (MTT) was purchased from Alfa Aesar. Tris(trimethylsilyl)phosphine (TMS3P, 10% v/v in hexane) was purchased from Alfa Aesar, and concentrated by the removal of hexane under reduced pressure. Tris(trimethylsilyl)arsine (TMS3As) was synthesized according to literature procedures (Wells, R. L. et al., Inorg. Synth. 1997, 31, 150). Tris(trimethylsilyl) arsine, tris(trimethylsilyl)phosphine and gallium (III) chloride are pyrophoric and must be handled carefully in a moisture-free and oxygen-free environment. Isobornyl acrylate (IBOA, technical grade) was purchased from Sigma-Aldrich, and pre-polymerised by mixing IBOA with 0.25 wt % DMPA. The resulting mixture was degassed for 10 minutes and then photo-polymerised with UV lamp (365 nm, 46 W) for 30 seconds. Dimethylsulfoxide (DMSO, 99.9% analytical grade), Hexane (98.5% high performance liquid chromatography, or HPLC grade) and MitoTracker™ Green FM were purchased from Thermo Fisher Scientific and used without further purification. Chloroform (99.8% analytical grade) and ethanol absolute (99.8% analytical grade) were purchased from VWR Chemicals BDH® and used without further purification. Roswell Park Memorial Institute (RPMI) 1640 Medium was purchased from Cytiva and used without further purification. Phosphate buffered saline (PBS, ultra-pure grade) was purchased from Vivantis and used without further purification. Hoechst 33342 trihydrochloride trihydrate (10 mg/mL solution in water) was from Life Technologies Corporation and used without further purification.
Analytical Techniques
UV-Visible Absorbance Measurements
UV-visible absorbance spectra were obtained by measuring the transmitted light intensity of an Ocean Optics HL-2000 broadband light source, using an Ocean Optics Flame-T and Flame-NIR spectrometer.
Photoluminescence Quantum Efficiency (PLQE) Measurements
The PL spectra and PLQE were obtained by photo-exciting the samples in an integrating sphere, using a Spectra-Physics 405 nm (100 mW, CW) diode laser, and measuring the absorption and PL using a calibrated Ocean Optics Flame-T and Flame-NIR spectrometer.
TEM and EDX
TEM images were recorded using JEOL JEM-2100F Field Emission TEM operated at 200 kV. This system was equipped with an Oxford Instruments INCA EDX. TEM samples were prepared by diluting QD-OA solutions in hexane and QD-MSA solutions in water, then drop-casting the solution on a copper grid.
Time-Resolved Photoluminescence (TRPL)
TRPL decays were acquired using a time-correlated single photon counting (TCSPC) setup (Horiba FluoroLog-3 Spectrofluorometer). Samples were excited using a 438 nm nano-LED light source (Horiba NanoLed-440L) with a typical pulse width of 260 ps and a repetition rate of 500 kHz. The PL decay curves were fitted using an exponential equation shown in equation (1) where A are the amplitudes of the exponential terms while r is the PL lifetime. I is the normalized PL intensity and t is the time.
The PLQE is defined as the ratio of the radiative recombination rate constant (Γr) to the sum of the radiative and non-radiative recombination rate constant (Γnr), given by equation (2):
The PL lifetime can be expressed as the reciprocal of the sum of recombination rate constants:
Therefore, using equation (2) and (3), we can calculate the value of Γr and Γnr:
In an ideal scenario, the laser beam penetrates the tissue with minimal scattering and attenuation, and excites the fluorescent QDs. The fluorescence signal would then traverse across the same tissue thickness to reach the detector and form the image. Therefore, we have designed our laser source and our QDs to both function within the near-infrared (NIR) spectral region to take advantage of the fact that NIR light experiences significantly lower attenuation and scattering by biological tissues as compared to visible photons (Weissleder, R., Nat. Biotechnol. 2001, 19, 316-317; and Frangioni, J. V., Curr. Opin. Chem. Biol. 2003, 7, 626-634). Here, we reported a laser-scanning imaging platform that employed a NIR laser and a high-efficiency NIR QD fluorescent dye to perform deep imaging of larger biological systems.
As shown in
We designed two sets of black masks that were placed above the fluorescent glass panel to provide customized fluorescent patterns for imaging. One pattern comprised spatially separated vertical bands, and another comprised the letters “NUS.”
We prepared our NIR QDs using a convenient one-pot continuous injection methodology (Franke, D. et al., Nat. Commun. 2016, 7, 12749; Wijaya, H. et al., Chem. Mater. 2019, 31, 2019-2026; and Wijaya, H. et al., Adv. Funct. Mater. 2020, 30, 1906483).
In(Zn)Oleate Solution
Indium acetate (0.25 mmol, 73 mg), zinc acetate (0.25 mmol, 46 mg) and OA (1.875 mmol, 0.67 mL) were mixed with ODE (8.5 mL) at room temperature (RT). Vacuum was applied to the round-bottom flask (RBF) and the reaction mixture was heated to 80° C. for 30 minutes under vacuum. The reaction mixture was then purged with argon and heated to 160° C., and stirred for 1 hour to form a clear solution. The mixture was subsequently cooled to 80° C. and vacuumed for 30 minutes before back-filling the RBF with argon and cooling the In(Zn)oleate solution to RT.
In(Zn)as Precursor
In a 5 mL RBF, In(Zn)oleate solution (0.45 mL, 0.0125 mmol) was mixed at RT with 0.55 mL of arsine precursor solution (made up of 0.55 mL of ODE, 4.6 μL, 0.0125 mmol of TMS3As and 31 μL of octylamine). The resulting In(Zn)As precursor solution was stirred for 5-10 minutes at RT before being used for injection.
In(Zn)P Precursor
In a 50 mL 2-neck RBF, In(Zn)oleate solution (8.5 mL, 0.25 mmol) was mixed at RT with 1 mL of phosphine precursor (made up of 73 μL, 0.25 mmol of TMS3P, 0.5 mL of octylamine and 0.5 mL of ODE). The resulting In(Zn)P precursor solution was stirred for 10 minutes at RT before being used for injection.
TOP-Sulfur (TOP-S) Precursor
Sulfur (0.50 mmol, 16 mg) was added to ODE (4.5 mL) in a RBF and vacuumed at 80° C. for 30 minutes. The reaction mixture was then filled with argon before addition of TOP (1.125 mmol, 0.52 mL) to form the TOP-S precursors.
Zn(Oleate)2 Solution
A Zn(oleate)2 solution was prepared from zinc acetate (0.50 mmol, 91.7 mg), OA (1.125 mmol, 0.4 mL) and ODE (to make 5 mL) based on the protocol for In(Zn)oleate solution except the Zn(oleate)2 solution needed to be heated to 80° C. to form a clear solution for injection.
In(Zn)as Seed Solution
Indium acetate (0.10 mmol, 30 mg), zinc acetate (0.05 mmol, 10 mg) and OA (0.0375 mmol, 13.2 μL) were mixed at RT with ODE (5 mL) in an argon-filled 50 mL RBF. The reaction mixture was vacuumed and heated to 80° C. for 30 minutes. The reaction mixture was then back-filled with argon, heated to 160° C. and stirred for 1 hour to form a clear and colorless solution. Following that, the reaction mixture was cooled to 80° C. and vacuumed for 30 minutes. The RBF was then filled with argon and heated to 230° C. An arsine precursor solution was prepared by mixing TMS3As (0.066 mmol, 20 μL) and octylamine (0.20 mL) with ODE (to make 1 mL) under an inert argon glovebox environment. The arsine precursor solution was injected into the indium precursor solution at 230° C. over 5 seconds. After stirring at 230° C. for 2.5 hours, a small portion (0.37 mL, 0.005 mmol) of the seed solution was withdrawn and diluted with dry ODE (2.5 mL) in another 3-neck 100 mL RBF. The remaining amount of the In(Zn)As seed solution was stored in the glovebox under argon environment for subsequent use.
InZn(As)—In(Zn)P
The diluted In(Zn)As seed solution was vacuumed at 80° C. for 15 mins before purging the reaction mixture with argon and heating to 230° C. The In(Zn)As precursor solution (1 mL) was injected into the In(Zn)As seed solution at 230° C., using a syringe pump, at a rate of 0.1 mL/min over 10 minutes. The solution was stirred for another 1 hour to give a In(Zn)As core that emits at ˜710 nm. The In(Zn)P precursor solution was injected into the In(Zn)As reaction mixture at 230° C., using a syringe pump, at a rate of 0.1 mL/min. The temperature was raised to 240° C. after 33 minutes, and to 250° C. after 66 minutes. After complete injection at 100 minutes, the temperature was raised to 260° C. and the solution was stirred for another 30 minutes to expend all precursors and complete the In(Zn)P shell synthesis. The slow injection process kept the concentration of the precursors low in the reaction mixture and significantly suppressed undesired side nucleation of In(Zn)P, while promoting the continuous growth of the thick shell. The reaction mixture was cooled to 240° C. for 7 minutes before the end of the In(Zn)P shell synthesis stage. Aliquots of the reaction were withdrawn from the reaction mixture and diluted in anhydrous hexane to track the reaction progress by optical absorbance and PL measurements.
In(Zn)As—In(Zn) P—Gap
Gallium (III) chloride (0.125 mmol, 22 mg) and OA (1.875 mmol, 0.146 mL) were mixed with ODE (to make 3.75 mL) in an argon-filled RBF under an inert argon glovebox environment to give a Ga(oleate)3 precursor solution. The Ga(oleate)3 precursor solution was stirred and degassed at RT for 2 hours under vacuum until a clear, pale yellow solution was observed. The Ga(oleate)3 precursor solution (3.75 mL) was injected into the In(Zn)As—In(Zn)P reaction mixture at 240° C., using a syringe pump, at a rate of 0.15 mL/min. After complete injection at 25 minutes, the reaction mixture was stirred for another 25 minutes at 240° C. to expend all precursors and complete the GaP shell synthesis. Gallium substituted the surface indium on In(Zn)As—In(Zn)P to give a GaP interlayer (Kim, S. et al., J. Am. Chem. Soc. 2012, 134, 3804-3809). This GaP interlayer eases the lattice mismatch between the In(Zn)P and ZnS shells, and offers significant enhancement to the PLQE.
In(Zn)As—In(Zn)P—GaP—ZnS QDs
The TOP-S precursor solution (5 mL) was injected into the In(Zn)As—In(Zn)P—GaP reaction mixture at 250° C., using a syringe pump at a rate of 0.25 mL/min. After complete injection at 25 minutes, the reaction mixture was stirred for another 25 minutes at 250° C. to expend all precursors. This was followed by another injection of the TOP-S precursor solution (5 mL) and Zn(oleate)2 solution (5 mL) at 260° C., using a syringe pump at a rate of 0.25 mL/min. The reaction mixture was then stirred for another 25 minutes at 260° C. to expend all precursors and complete the ZnS shell layer. The reaction mixture was allowed to cool to RT. Ethanol (40 mL) was added to the reaction mixture to precipitate the In(Zn)As—In(Zn)P—GaP—ZnS QDs, followed by centrifugation of the mixture at 6000 rpm for 5 minutes. The clear supernatant was carefully removed using a dropper. The addition of ethanol and centrifugation process was repeated twice to purify the quantum dots. The final precipitate was re-dispersed in anhydrous hexane (20 mL) and stored for further use.
The In(Zn)As—In(Zn)P—GaP—ZnS QDs prepared in Example 2 (QD-OA) were characterized.
Our new NIR-emitting In(Zn)As—In(Zn)P—GaP—ZnS QD (
The transmission electron microscopy (TEM) image of the QDs (
To test the imaging functions of our setup in Example 1, we prepared a fluorescent glass panel by casting a dispersion of NIR-emitting QDs (prepared in Example 2) in a UV-curable resin onto a clear borosilicate glass plate, followed by photocuring the QD-resin under a 365 nm UV illumination.
The QD-resin comprises a homogenous dispersion of QDs in IBOA monomers and TCDDA crosslinkers. To synthesize the QD-resin, 15 mL of the QDs solution was centrifuged in ethanol at 10,000 rpm for 5 minutes. The supernatant was removed and the solid was re-dispersed in chloroform (5 mL). Tri-p-tolyl phosphine (600 μL, 100 mg/mL in chloroform) was added to the solution and stirred for 30 minutes. The solvent was vacuum-evaporated and the resulting solid was re-dispersed in IBOA (500 μL). Pre-polymerised IBOA (5 mL, viscosity ˜1700 cP) and TCDDA (2.25 mL) were added sequentially and the reaction mixture was stirred for 10 minutes between each addition. Finally, a radical photoinitiator, DMPA (37.5 mg, 0.5 wt %), was added and the reaction mixture was stirred for 3 hours to form a clear and homogenous QD-resin mixture (viscosity ˜1100 cP). The resulting resin mixture was then applied onto a transparent glass (16×16 cm2 with 0.6 mm thickness) using a film applicator to achieve a uniform 200 μm thick film (
We used the two designed sets of masks for coupling with the QDs-resin fluorescent panel (prepared in Example 4) to produce fluorescent patterns in the imaging experiments.
To better mimic the compositional complexity of actual animal tissues, we employed real pork loin tissues for our imaging experiments instead of the more commonly-used synthetic tissue phantom. Layers of pork loin tissues with controlled thicknesses were placed successively above the fluorescent patterns to simulate fluorescence in a deep-tissue environment, and images were acquired using our laser-scanning imaging setup. The pork loin tissue was thinly-sliced to give 2 mm layers and stacked incrementally over the mask patterns for up to a total of 8 layers (16 mm) as shown in
For the purpose of comparison, we also performed wide-field of view (FOV) fluorescence imaging using a Canon 200D DSLR camera that was modified with a 720 nm longpass filter, and used 634 nm red LEDs as a blanket-illumination excitation source. As shown in
We sought to further enhance our fluorescence imaging results through the processing of the images using a new machine learning approach. The broad idea involved supplying the machine learning algorithms with scattered images and sharp images, and training the machine to process future scattered images into sharper images.
In some embodiments, the scattered images (such as fluorescence images) are processed using a trained machine learning model such as a neural network model.
There is a computer-implemented or computerized method for processing scattered images using the neural network model that has been trained beforehand. The method comprises a step of providing the trained neural network model trained with a training dataset of scattered images comprising associated pairs of low-resolution images and high-resolution images, each image comprising a series of spatially separated bands. For each associated pair of images, there is a low-resolution image and a high-resolution image of the same sample or region of interest. The low-resolution images are scattered or blurry images, and the high-resolution images have sharp contrast. The high-resolution images are better than the low-resolution images at least in terms of one or more of spatial resolution, contrast, sharpness, and signal-to-noise ratio.
The neural network model comprises a plurality of layers including an input layer, zero or more hidden layers, and an output layer, each layer having a respective set of neurons for processing data. The input layer is configured for receiving input data from an external source, performing calculations via its neurons, and sending the results onto the subsequent layers. The output layer is configured for receiving inputs from the preceding layers (including the processed input data at the input layer), performing calculations via its neurons, and generating the final output data. Optionally, the neural network model comprises at least one hidden layer residing between the input and output layers. As shown in
The method comprises steps of receiving (at the input layer) an input scattered image by the trained neural network model, processing (at the hidden layers) the input scattered image using the trained neural network model, and generating (at the output layer) an output image by the trained neural network model in response to said processing of the input scattered image. The output image has a higher resolution than the input scattered image. Training of the neural network model with the training dataset of low-resolution images and high-resolution images enables the neural network model to process subsequent low-resolution images into high-resolution images. For example, the input image has low resolution (e.g. scattered/blurry) and the trained neural network model enhances the image into a high-resolution one by de-scattering the image and improving the contrast and sharpness. The output image may be referred to as a de-scattered image.
The neural network model is initially trained by the training dataset of scattered images which can be acquired by various methods. For example, the scattered images are fluorescence images which are acquired by the near-infrared laser-scanning excitation and efficient quantum dot photoluminescence methods described herein. The neural network model may be pretrained or the method may comprise training the neural network model.
With reference to
The training further comprises a step (d) of weighting each pixel in a cluster of pixels with a set of weight and bias parameters. Specifically, each layer multiplies the value of each pixel by the respective weight parameter and adds the respective bias parameter, and sends the resultant values to the next layer. In some embodiments, the step (d) comprises successively weighting each pixel at least twice. For example as shown in
The training further comprises a step (e) of generating a processed output pixel from said weighting of the cluster of pixels. Notably, as shown in
The training further comprises a step (f) of determining an error between the processed output pixel and the true output pixel from the corresponding high-resolution image. The error may be calculated as a loss function using a supervised batch learning approach, and the error may comprise a mean squared error between the value of the processed output pixel and the value of the true output pixel.
The training further comprises a step (g) of backpropagating the error to adjust the parameters in step (d) using suitable backpropagation algorithms. With an adjusted set of weight and bias parameters, the pixels from the low-resolution image are processed again to thereby regenerate the processed output pixel. The training further comprises a step (h) of iteratively performing steps (d) to (g) to minimize the error, thus improving the accuracy of the processed output pixel. For example, the training comprises using a gradient descent function as a to adjust the parameters and minimize the error (loss function). The gradient descent function is an optimization algorithm used to determine the parameters that minimize a cost function of the training images. The gradient descent function involves multiple iterations of forward propagating the inputs, backpropagating the error, and adjusting the parameters until an optimized set of parameters is determined. The parameters may first be randomly selected and then iteratively processed until they are optimized.
Therefore, the iterative process minimizes the error, and the minimized error is associated with an optimized set of parameters for the neural network model. Further, the training comprises repeating steps (a) to (h) for each associated pair of images in the training dataset, thus completing the training of the neural network model. The optimized weights and bias parameters can then be used to process and enhance subsequent low-resolution images into high-resolution images.
The trained neural network model was tested using two sets of low-resolution scattered images (fluorescence images) as shown in
Moreover, for the first set of images, in order to better quantify the resolution of the images, the gap distances at which two separate bands could no longer be distinguished from one another were analysed. As shown in
As mentioned above, each image in the training dataset used to train the neural network model comprises a series of spatially separated bands. The bands can be oriented in any direction, such as vertically. The spatially separated bands in the low-resolution and high-resolution images for training help the neural network model to identify and resolve features at varying gap distances. This improves the ability of the trained neural network model to process and enhance low-resolution images, as evidenced in the results above. Even though the training was performed with images having the spatially separated bands, the trained neural network model was equally capable of resolving features on the “NUS” letters. This generality allows the trained neural network model to be employed to enhance a broad range of scattered images including fluorescence images where scattering problems are observed.
By implementing a trained neural network model, the imaging resolution can be enhanced, paving a promising way forward for the use of machine learning to enhance imaging quality in highly scattering media. As an example, the trained neural network model can be used to process a low-quality image of an item hidden behind a scattering medium such as a piece of frosted glass. The trained neural network model is able to enhance the light information coming through the scattering medium and improve the quality and resolution of the image of the item. The image enhancement through the trained neural network model may also have a far-reaching impact on how future images with significant blurring could be effectively repaired. The trained neural network model can be combined with the laser-scanning approach for fluorescence imaging to develop a low-cost yet powerful methodology for performing non-invasive deep-imaging in larger animal models or human tissues.
We reasoned that the ability to resolve millimetre-scale features at tissue depths greater than 10 mm was sufficient for performing imaging functions across sections of an animal body or the human body, for instance, through the palm.
In a new experiment, we separately placed a rack of pork ribs (20 mm thickness) and rested an adult-human palm on a fluorescent QD panel, and performed laser-scanning imaging with our setup. We noted that a mild 200 mW/mm2 CW laser irradiance at a non-ionising NIR wavelength of 721 nm causes no damage to the skin or tissue, especially when rapidly scanned across the objects.
Through the use of laser-scanning excitation and the deployment of artificial neural network for image processing, we have shown that deep imaging performance could be easily achieved across thick sections of animal tissues over a wide FOV. We anticipate that our approach could offer notable improvements to existing in vivo fluorescence imaging and tomography techniques, and could serve as a powerful new research platform for preclinical immunotherapy or targeted therapy studies.
Prior to testing the cytotoxicity of the QDs on HeLa cells, a ligand-exchange step was conducted to replace hydrophobic OA ligands with hydrophilic MSA ligands via a phase transfer method (
2 mL of the In(Zn)As—In(Zn)P—GaP—ZnS QDs dispersed in hexane (prepared in Example 2) was dried using a rotary evaporator and re-dispersed in chloroform (2 mL). In a 20 mL glass vial, MSA (300 mg, 2 mmol) was stirred in chloroform (5 mL) for 15 minutes until a suspension was formed. The 2 mL of QDs in chloroform was then quickly added to the MSA suspension, and the reaction mixture was stirred vigorously in air. After 1 minute of stirring, ammonium hydroxide solution (28-30% v/v in water, 0.5 mL) was added, and the stirring was continued for another 3 hours. The QDs were then precipitated with ethanol (10 mL) in a 15 mL centrifuge tube, and centrifuged at 10,000 rpm for 5 minutes. The supernatant was discarded, and the residue was dispersed in deionized water (1.5 mL) before being precipitated in ethanol (10.5 mL) and centrifuged at 10,000 rpm for 5 minutes. The purification process involving ethanol addition, centrifugation and water dispersion was repeated twice. After the third centrifugation, the residue was dried under vacuum for 1.5 hours and dispersed in deionized water to obtain a mass concentration of 30 mg/mL. The concentrated QD-MSA solution was filtered through a 0.45 μm hydrophilic syringe filter into a clean 2 mL vial and stored in the freezer for subsequent use and characterization.
The QD-MSA prepared in Example 9 were characterized. The stability of QD-MSA was also evaluated to validate the applicability of our NIR QDs for bioimaging applications.
Dynamic Light Scattering (DLS) Measurements
The hydrodynamic size distribution (by volume) and zeta (ζ) potential of the MSA-capped QDs in water was estimated using a dynamic light scattering molecular size analyser (Zetasizer Nano ZS using a He—Ne 633 nm laser). The MSA-capped QD solution was first diluted in deionized water to a concentration of 1.5 mg mL−1 before it was filtered through a 0.45 μm hydrophilic syringe filter membrane to remove any residual dust particles. After that, the size distribution and ζ potential were measured.
Stability Study
QD-OA is soluble in hexane while QD-MSA is soluble in water. QD-OA (1 mL, prepared in Example 2) was mixed with deionized water (1 mL) in a 2 mL vial while QD-MSA (1 mL, prepared in Example 9) was mixed with hexane (1 mL) in another 2 mL vial. As the solution mixture settled, QD-OA remained in the hexane phase while QD-MSA remained in the water phase.
Photo-Stability Study
QD-MSA (1.5 mg/mL in water in 1 cm path length cuvette, absorption 36%) was subjected to continuous laser irradiation (30 mW, 405 nm) for 3 hours. PL spectra was taken at timed intervals, and the peak intensity at 825 nm was plotted against time.
The hydrophilic QD-MSA was highly stable in water as the aqueous phase remained transparent and homogeneous without any precipitation or aggregation at room temperature for at least 1 month (
The spectra in
To further validate the applicability of our NIR QDs for bioimaging applications, we performed an in vitro study on human cervical carcinoma HeLa cells using fluorescence confocal microscopy.
HeLa Cell Culture and Confocal Imaging
HeLa cells were cultured in growth media (RPMI 1640) maintained at 37° C. with a humidified atmosphere of 5%CO
MTT Cell Viability Assay
The MTT cell viability assay was used to determine the cytotoxicity of the QD on HeLa cells. Briefly, an MTT stock solution was prepared by dissolving MTT in PBS at 5 mg/mL, filter-sterilized and stored away from light at 4° C. until further use. Cells were seeded on clear, flat-bottom 96-well culture plates at a density of 6×103 cells in 100 μL of growth media per well. After 24 hours, the culture media were replaced with 100 μL/well of fresh RPMI containing varying concentrations of the QD (15 μg mL−1 to 1 mg mL−1) and incubated for another 24 hours. Thereafter, the QD-containing media were aspirated and replenished with 100 μL/well of MTT solution (diluted to 0.5 mg/mL with RPMI) and incubated for 50 minutes. The MTT solution was then aspirated and DMSO (100 μL/well) was added to dissolve the dark blue crystals formed. Cell viability was determined by measuring the absorbance at 570 nm using a microplate reader and expressed as a percentage of the non-treated control wells.
The MSA-capped hydrophilic NIR QDs in water were incubated with HeLa cells (0.5 mg mL−1 and 1 mg mL−1 QD-MSA in RPMI 1640 medium) for 24 hours before being imaged by a fluorescence confocal microscope with 405 nm laser excitation for Hoechst and the QDs, and with 488 nm laser excitation for MitoTracker Green. The visualized HeLa cells appeared brightly-luminescent under laser excitation, clearly demarcating the Hoechst-stained nucleus (blue), MitoTracker Green-stained mitochondria (green) and the NIR QDs (red) in
Taken together, the high PLQE and strong stability under physiological conditions (described in Example 10), coupled with low cytotoxicity, validated our MSA-capped NIR QD dye as an outstanding candidate for high-contrast bioimaging across different platforms from deep tissues to in vitro and in vivo applications. In combination with the deep imaging performance of our artificial-neural-network-enhanced laser-scanning imaging technique, this could become a powerful research platform for advancing the discovery of drugs for immunotherapy and targeted therapy in cancer.
| Number | Date | Country | Kind |
|---|---|---|---|
| 10202005338X | Jun 2020 | SG | national |
| Filing Document | Filing Date | Country | Kind |
|---|---|---|---|
| PCT/SG2021/050321 | 6/4/2021 | WO |
