Patent Application
20210169336
Evaluation of tissue characteristics can be slow and inefficient due to the biopsy process used to generate the tissue samples. Furthermore, biopsies can be invasive, thus limiting the number and/or size of excised tissue samples taken from a subject. Additionally, biopsies of adjacent regions of tissue are not feasible or desirable. Accordingly, routine control samples are not taken in biopsy procedures.
Recognized herein is a need for improved methods for identifying and detecting tissue characteristics. Provided herein are methods and apparatuses that improve information that may be used to identify characteristics in tissue. Methods and apparatuses described herein may improve machine learning algorithms and applications of such algorithms. Further provided herein are methods and apparatuses that may improve information quality and quantity that can be obtained in a single clinical visit or in real time. Methods and apparatuses described herein may provide information that can be used concurrently with treatment.
In an aspect, the present disclosure provides a method for identifying a tissue characteristic in a subject, comprising: (a) accessing a database comprising a first set of data from a first image obtained from a first tissue region of the subject and a second set of data from a second image obtained from a second tissue region of the subject, wherein the first tissue region is suspected of having the tissue characteristic, and wherein the second tissue region is free or suspected of being free from having the tissue characteristic; (b) computer processing the first set of data and the second set of data to (i) identify a presence or absence of one or more features indicative of the tissue characteristic in the first image, and (ii) classify the subject as being positive or negative for the tissue characteristic based on the presence or absence of the one or more features in the first image; and (c) generating an electronic report which is indicative of the subject being positive or negative for the tissue characteristic.
In some embodiments, the tissue characteristic is a disease or abnormality. In some embodiments, the disease or abnormality is cancer. In some embodiments, the tissue characteristic comprises a beneficial tissue state. In some embodiments, the first image and the second image are obtained in vivo. In some embodiments, the first image and the second image are obtained without removal of the first tissue region or the second tissue region from the subject. In some embodiments, the first tissue region or the second tissue region is not fixed to a slide. In some embodiments, the first image or the second image is generated using at least one non-linear imaging technique. In some embodiments, the first image or the second image is generated using at least one non-linear imaging technique and at least one linear imaging technique. In some embodiments, the first set of data and the second set of data comprise groups of data, and wherein a group of data of the groups of data comprises a plurality of images. In some embodiments, the plurality of images comprises: (i) a positive image, which positive image comprises the one or more features; and (ii) a negative image, which negative image does not comprise the one or more features. In some embodiments, the first set of data and the second set of data comprise groups of data, and wherein a group of data of the groups of data comprises a plurality of images and the plurality of images comprises: (i) a positive image, which positive image comprises the one or more features; and (ii) a negative image, which negative image does not comprise the one or more features. In some embodiments, the electronic report comprises information related to a risk of the tissue characteristic. In some embodiments, the first image or the second image are real-time images. In some embodiments, the first tissue region is adjacent to the second tissue region. In some embodiments, (i) the first image comprises a first sub-image of a third tissue region adjacent to the first tissue region; or (ii) the second image comprises a second sub-image of a fourth tissue region. In some embodiments, the first image or the second image comprises one or more depth profiles. In some embodiments, the one or more depth profiles are one or more layered depth profiles. In some embodiments, the one or more depth profiles comprise one or more depth profiles generated from a scanning pattern that moves in one or more slanted directions. In some embodiments, the first image or the second image comprises one or more depth profiles, and wherein (i) the one or more depth profiles are one or more layered depth profiles or (ii) the one or more depth profiles comprise one or more depth profiles generated from a scanning pattern that moves in one or more slanted directions. In some embodiments, the first image or the second image comprise layered images. In some embodiments, the first image or the second image comprises at least one layer generated using one or more signals selected from the group consisting of second harmonic generation signals, third harmonic generation signals, reflectance confocal microscopy signals, and multi-photon fluorescence signals. In some embodiments, the first image or the second comprise layered images, and wherein the first image or the second image comprises at least one layer generated using one or more signals selected from the group consisting of second harmonic generation signals, third harmonic generation signals, reflectance confocal microscopy signals, and multi-photon fluorescence signals. In some embodiments, the first image or the second image comprises one or more depth profiles generated from a scanning pattern that moves in one or more slanted directions. In some embodiments, the method further comprises outputting the electronic report on a user interface of an electronic device used to collect the first image and the second image. In some embodiments, (b) comprises calculating a first weighted sum of one or more features for the first image and a second weighted sum of one or more features for the second image. In some embodiments, the method further comprises classifying the subject as positive or negative for the tissue characteristic based on a difference between the first weighted sum and the second weighted sum. In some embodiments, (b) comprises calculating a first weighted sum of one or more features for the first image and a second weighted sum of one or more features for the second image and the method further comprises classifying the subject as positive or negative for the tissue characteristic based on a difference between the first weighted sum and the second weighted sum. In some embodiments, the subject is classified as being positive or negative for the tissue characteristic at an accuracy of greater than or equal to about 90%. In some embodiments, the subject is classified as being positive or negative for the tissue characteristic at a sensitivity of greater than or equal to about 90%. In some embodiments, the subject is classified as being positive or negative for the tissue characteristic at a specificity of greater than or equal to about 90%. In some embodiments, the subject is classified as being positive or negative for the tissue characteristic at an accuracy, sensitivity, or specificity of greater than or equal to about 90%. In some embodiments, (b) further comprises applying a trained machine learning algorithm to the first set of data or the second set of data. In some embodiments, (b) further comprises classifying the subject as being positive or negative for the tissue characteristic based on the presence or absence of the one or more features of the first image at an accuracy of at least about 80%. In some embodiments, (b) further comprises applying a trained machine learning algorithm to the first set of data or the second set of data and (b) further comprises classifying the subject as being positive or negative for the tissue characteristic based on the presence or absence of the one or more features of the first image at an accuracy of at least about 80%. In some embodiments, the first image or the second image has a resolution of at least about 5 micrometers. In some embodiments, (i) the first image extends below a first surface of the first tissue region; or (ii) the second image extends below a second surface of the second tissue region. In some embodiments, the first image or the second image has a resolution of at least about 5 micrometers and, (i) the first image extends below a first surface of the first tissue region; or (ii) the second image extends below a second surface of the second tissue region. In some embodiments, (b) further comprises computer processing a third data set from a third image of a third tissue region having the one or more features indicative of the tissue characteristic. In some embodiments, (b) further comprises computer processing a fourth data set from a fourth image of a fourth tissue region lacking the one or more features indicative of the tissue characteristic. In some embodiments, (b) further comprises (i) computer processing a third data set from a third image of a third tissue region having the one or more features indicative of the tissue characteristic; and (ii) computer processing a fourth data set from a fourth image of a fourth tissue region lacking the one or more features indicative of the tissue characteristic. In some embodiments, the third tissue region or the fourth tissue region is of a different subject than the subject. In some embodiments, the third tissue region or the fourth tissue region is of the subject. In some embodiments, the database further comprises one or more images from one or more additional subjects. In some embodiments, at least one of the one or more additional subjects is positive for the tissue characteristic. In some embodiments, at least one of the one or more additional subjects is negative for the tissue characteristic. In some embodiments, the database further comprises one or more images from one or more additional subjects, and wherein (i) at least one of the one or more additional subjects is positive for the tissue characteristic or (ii) at least one of the one or more additional subjects is negative for the tissue characteristic.
In another aspect, the present disclosure provides a method for identifying a tissue characteristic in a subject, comprising: (a) using an imaging probe to obtain a first image from a first tissue region of the subject and a second image from a second tissue region of the subject, wherein the first tissue region is suspected of having the tissue characteristic and wherein the second tissue region is free or suspected of being free from the tissue characteristic; (b) transmitting data derived from the first image and the second image to a computer system, wherein the computer system processes the data to (i) identify a presence or absence of one or more features indicative of the tissue characteristic in the first image, and (ii) classify the subject as being positive or negative for the tissue characteristic based on the presence or absence of the one or more features in the first image; and (c) providing a treatment to the subject upon classifying the subject as being positive for the tissue characteristic.
In some embodiments, the method further comprises treating the subject for the tissue characteristic based on the classifying the subject as being positive for the tissue characteristic. In some embodiments, the tissue characteristic is indicative of a disease or an abnormality. In some embodiments, the disease of abnormality is cancer. In some embodiments, the imaging probe comprises imaging optics. In some embodiments, the imaging probe is configured to measure an electrical signal. In some embodiments, the method further comprises, prior to (c), receiving an electronic report indicative of the tissue characteristic. In some embodiments, the computer system is a cloud-based computer system. In some embodiments, the computer system comprises one or more machine learning algorithms. In some embodiments, the method further comprises using the one or more machine learning algorithms to process the data, wherein the data from the second image are used as a control. In some embodiments, the computer system comprises one or more machine learning algorithms, the method further comprises using the one or more machine learning algorithms to process the data, and the data from the second image are used as a control. In some embodiments, the imaging probe is handheld. In some embodiments, the imaging probe comprises imaging optics. In some embodiments, the imaging probe is translated across a surface of the tissue. In some embodiments, the imaging probe is translated between the first tissue region and the second tissue region. In some embodiments the imaging probe is translated across a surface of the tissue between the first tissue region and the second tissue region. In some embodiments, during (a), a position of the imaging probe is tracked.
In another aspect, the present disclosure provides a method for identifying a tissue characteristic in a subject, comprising: (a) accessing a database comprising data from an image obtained from a tissue region of the subject, wherein the tissue region is suspected of having the tissue characteristic; (b) applying a trained algorithm to the data to (i) identify a presence or absence of one or more features indicative of the tissue characteristic in the image, and (ii) classify the subject as being positive or negative for the tissue characteristic based on the presence or absence of one or more features in the image at an accuracy of at least about 80%; and (c) generating an electronic report which is indicative of the subject being positive or negative for the tissue characteristic.
In some embodiments, the tissue characteristic is indicative of a disease or an abnormality. In some embodiments, the disease of abnormality is cancer.
In another aspect, the present disclosure provides a method for identifying a tissue characteristic in a subject, comprising: (a) accessing a database comprising data from an image obtained from a tissue region of the subject, wherein the tissue region is suspected of having the tissue characteristic, and wherein the image has a resolution of at least about 5 micrometers; (b) applying a trained algorithm to the data to (i) identify a presence or absence of one or more features indicative of the tissue characteristic in the image, and (ii) classify the subject as being positive or negative for the tissue characteristic based on the presence or absence of the one or more features in the image; and (c) generating an electronic report which is indicative of the subject being positive or negative for the tissue characteristic.
In some embodiments, the tissue characteristic is indicative of a disease or an abnormality. In some embodiments, the disease of abnormality is cancer.
In another aspect, the present disclosure provides a method for generating a dataset comprising a plurality of images of a tissue of a subject, comprising: (a) obtaining, via a handheld imaging probe, a first image from a first part of the tissue of the subject and a second set of images from a second part of the tissue of the subject, wherein the first part is suspected of having a tissue characteristic, and wherein the second part is free or suspected of being free from the tissue characteristic; and (b) storing data corresponding to the first image and the second image in a database.
In some embodiments, the handheld imaging probe comprises imaging optics. In some embodiments, the method further comprises, repeating (a) one or more times to generate the dataset comprising a plurality of first sets of images of the first part of the tissue of the subject and a plurality of second sets of images of the second part of the tissue of the subject. In some embodiments, the first set of images and the second set of images are images of the skin of the subject. In some embodiments, the method further comprises (c) training a machine learning algorithm using at least a part of the plurality of signals. In some embodiments, data derived from the second set of signals are used as a control. In some embodiments, the method further comprises (c) training a machine learning algorithm using at least a part of the plurality of signals and the data derived from the second set of signals are used as a control. In some embodiments, the tissue of the subject is not removed from the subject. In some embodiments, the tissue of the subject is not fixed to a slide. In some embodiments, the first part and the second part are adjacent parts of the tissue. In some embodiments, the first image or the second image comprises a depth profile of the tissue. In some embodiments, the first image or the second image is collected from a depth profile of the tissue. In some embodiments, the first image or the second image is collected in substantially real-time. In some embodiments, the first image or the second image (i) comprises a depth profile of the tissue, (ii) is collected from a depth profile of the tissue, (iii) is collected in substantially real-time, or (iv) any combination thereof. In some embodiments, the first image or the second image is collected in real-time. In some embodiments, the first image is obtained within at most 48 hours of obtaining the second image.
In another aspect, the present disclosure provides a method for generating a trained machine learning algorithm to identify a tissue characteristic in a subject, comprising: (a) providing a data set comprising a plurality of tissue depth profiles, wherein the plurality of tissue depth profiles comprises (i) a first depth profile of a first tissue region positive for the tissue characteristic and (ii) a second depth profile of a second tissue region negative for the characteristic; and (b) using the first depth profile and the second depth profile to train a machine learning algorithm, thereby generating the trained machine learning algorithm.
In some embodiments, the first depth profile and the second depth profile are obtained from the same subject. In some embodiments, the first depth profile and the second depth profile are obtained from different subjects. In some embodiments, the first tissue region and the second tissue region are tissue regions of the same tissue. In some embodiments, the first tissue region and the second tissue region are tissue regions of different tissues. In some embodiments, the first depth profile or the second depth profile is an in vivo depth profile. In some embodiments, the first depth profile or the second depth profile is a layered depth profile. In some embodiments, the first depth profile or the second depth profile is generated using one or more signals selected from the group consisting of second harmonic generation signals, third harmonic generation signals, reflectance confocal microscopy signals, and multi-photon fluorescence signals. In some embodiments, the first depth profile or the second depth profile is a layered depth profile and the first depth profile or the second depth profile is generated using one or more signals selected from the group consisting of second harmonic generation signals, third harmonic generation signals, reflectance confocal microscopy signals, and multi-photon fluorescence signals. In some embodiments, the method further comprises outputting the trained machine learning algorithm. In some embodiments, the method further comprises using one or more additional depth profiles to further train the trained machine learning algorithm.
In another aspect, the present disclosure provides a system comprising one or more computer processors and computer memory coupled thereto, wherein the computer memory comprises machine executable code that, upon execution by the one or more computer processors, implements a method for identifying a tissue characteristic in a subject, the method comprising: (a) accessing a database comprising a first set of data from a first image obtained from a first tissue region of the subject and a second set of data from a second image obtained from a second tissue region of the subject, wherein the first tissue region is suspected of having the tissue characteristic, and wherein the second tissue region is free or suspected of being free from having the tissue characteristic; (b) computer processing the first set of data and the second set of data to (i) identify a presence or absence of one or more features indicative of the tissue characteristic in the first image, and (ii) classify the subject as being positive or negative for the tissue characteristic based on the presence or absence of the one or more features in the first image.
In some embodiments, the method further comprises generating an electronic report which is indicative of the subject being positive or negative for the tissue characteristic. In some embodiments, the electronic report comprises information related to a risk of the tissue characteristic. In some embodiments, the system further comprises an electronic device and wherein method further comprises outputting the electronic report on a user interface of the electronic device used to collect the first image and the second image. In some embodiments, the system comprises an imaging probe, which imaging probe is operatively coupled to the one or more computer processors. In some embodiments, the imaging probe is handheld. In some embodiments, the system comprises an imaging probe, which imaging probe is operatively coupled to the one or more computer processors, and the imaging probe is handheld. In some embodiments, the imaging probe is configured to deliver therapy to the tissue. In some embodiments, the tissue characteristic is a disease or abnormality. In some embodiments, the disease or abnormality is cancer. In some embodiments, the tissue characteristic comprises a beneficial tissue state. In some embodiments, the first image and the second image are obtained in vivo. In some embodiments, the first image and the second image are obtained without removal of the first tissue region or the second tissue region from the subject. In some embodiments, the first tissue region or the second tissue region is not fixed to a slide. In some embodiments, the first image or the second image is generated using at least one non-linear imaging technique. In some embodiments, the first image or the second image is generated using at least one non-linear imaging technique and at least one linear imaging technique. In some embodiments, the first set of data and the second set of data comprise groups of data, and wherein a group of data of the groups of data comprises a plurality of images. In some embodiments, the plurality of images comprises: (i) a positive image, which positive image comprises the one or more features; and (ii) a negative image, which negative image does not comprise the one or more features. In some embodiments, the first image or the second image are real-time images. In some embodiments, the first tissue region is adjacent to the second tissue region. In some embodiments, (i) the first image comprises a first sub-image of a third tissue region adjacent to the first tissue region; or (ii) the second image comprises a second sub-image of a fourth tissue region. In some embodiments, the first image or the second image comprises one or more depth profiles. In some embodiments, the one or more depth profiles are one or more layered depth profiles.
In some embodiments, the one or more depth profiles comprise one or more depth profiles generated from a scanning pattern that moves in one or more slanted directions. In some embodiments, the first image or the second image comprise layered images. In some embodiments, the first image or the second image comprises at least one layer generated using one or more signals selected from the group consisting of second harmonic generation signals, third harmonic generation signals, reflectance confocal microscopy signals, and multi-photon fluorescence signals. In some embodiments, the first image or the second image comprise layered images and first image or the second image comprises at least one layer generated using one or more signals selected from the group consisting of second harmonic generation signals, third harmonic generation signals, reflectance confocal microscopy signals, and multi-photon fluorescence signals. In some embodiments, the first image or the second image comprises one or more depth profiles generated from a scanning pattern that moves in one or more slanted directions. In some embodiments, (b) comprises calculating a first weighted sum of one or more features for the first image and a second weighted sum of one or more features for the second image. In some embodiments, the method further comprises classifying the subject as positive or negative for the tissue characteristic based on a difference between the first weighted sum and the second weighted sum. In some embodiments, the subject is classified as being positive or negative for the tissue characteristic at an accuracy of greater than or equal to about 90%. In some embodiments, the subject is classified as being positive or negative for the tissue characteristic at a sensitivity of greater than or equal to about 90%. In some embodiments, the subject is classified as being positive or negative for the tissue characteristic at a specificity of greater than or equal to about 90%. In some embodiments, (b) further comprises applying a trained machine learning algorithm to the first set of data or the second set of data. In some embodiments, (b) further comprises classifying the subject as being positive or negative for the tissue characteristic based on the presence or absence of the one or more features of the first image at an accuracy of at least about 80%. In some embodiments, (b) further comprises applying a trained machine learning algorithm to the first set of data or the second set of data and (b) further comprises classifying the subject as being positive or negative for the tissue characteristic based on the presence or absence of the one or more features of the first image at an accuracy of at least about 80%. In some embodiments, the first image or the second image has a resolution of at least about 5 micrometers. In some embodiments, (i) the first image extends below a first surface of the first tissue region; or (ii) the second image extends below a second surface of the second tissue region. In some embodiments, (b) further comprises computer processing a third data set from a third image of a third tissue region having the one or more features indicative of the tissue characteristic. In some embodiments, (b) further comprises computer processing a fourth data set from a fourth image of a fourth tissue region lacking the one or more features indicative of the tissue characteristic. In some embodiments, (b) further comprises (i) computer processing a third data set from a third image of a third tissue region having the one or more features indicative of the tissue characteristic; and (ii) computer processing a fourth data set from a fourth image of a fourth tissue region lacking the one or more features indicative of the tissue characteristic. In some embodiments, the third tissue region or the fourth tissue region is of a different subject than the subject. In some embodiments, the third tissue region or the fourth tissue region is of the subject. In some embodiments, the database further comprises one or more images from one or more additional subjects. In some embodiments, at least one of the one or more additional subjects is positive for the tissue characteristic. In some embodiments, at least one of the one or more additional subjects is negative for the tissue characteristic.
In another aspect, the present disclosure provides a system comprising one or more computer processors and computer memory coupled thereto, wherein the computer memory comprises machine executable code that, upon execution by the one or more computer processors, implements a method for generating a trained machine learning algorithm to identify a tissue characteristic in a subject, the method comprising: (a) receiving a data set comprising a plurality of tissue depth profiles, wherein the plurality of tissue depth profiles comprises (i) a first depth profile of a first tissue region positive for the tissue characteristic and (ii) a second depth profile of a second tissue region negative for the characteristic; and (b) using the first depth profile and the second depth profile to train a machine learning algorithm, thereby generating the trained machine learning algorithm.
In some embodiments, the system comprises an imaging probe, which imaging probe is operatively coupled to the one or more computer processors. In some embodiments, the imaging probe is handheld. In some embodiments, the imaging probe is configured to deliver therapy to tissue. In some embodiments, the first depth profile and the second depth profile are obtained from the same subject. In some embodiments, the first depth profile and the second depth profile are obtained from different subjects. In some embodiments, the first tissue region and the second tissue region are tissue regions of the same tissue. In some embodiments, the first tissue region and the second tissue region are tissue regions of different tissues. In some embodiments, the first depth profile or the second depth profile is an in vivo depth profile. In some embodiments, the first depth profile or the second depth profile is a layered depth profile. In some embodiments, the first depth profile or the second depth profile is generated using one or more signals selected from the group consisting of second harmonic generation signals, third harmonic generation signals, reflectance confocal microscopy signals, and multi-photon fluorescence signals. In some embodiments, the first depth profile or the second depth profile is a layered depth profile and the first depth profile or the second depth profile is generated using one or more signals selected from the group consisting of second harmonic generation signals, third harmonic generation signals, reflectance confocal microscopy signals, and multi-photon fluorescence signals. In some embodiments, the system further comprises outputting the trained machine learning algorithm. In some embodiments, the system further comprises using one or more additional depth profiles to further train the trained machine learning algorithm.
In another aspect, the present disclosure provides a system for identifying and treating a tissue, comprising: an optical probe configured to optically obtain an image or depth profile of the tissue; and a radiation source configured to deliver radiation to the tissue; and a housing enclosing the optical imaging probe and the radiation source.
In some embodiments, the housing is handheld. In some embodiments, the radiation source comprises a laser. In some embodiments, in a treatment mode, the radiation source is configured to deliver radiation to the tissue that heats the tissue. In some embodiments, in a treatment mode, the radiation source is configured to activate a beneficial process in the tissue. In some embodiments, in a detection mode, the radiation source is configured to deliver the radiation to tissue that generates optical signals from the tissue, and wherein the optical probe is configured to detect the optical signals. In some embodiments, the system further comprises one or more computer processors operatively coupled to the optical probe and the radiation source. In some embodiments, the radiation source is configured to be operated in detection and treatment modes simultaneously. In some embodiments, the optical probe comprises an additional radiation source separate from the radiation source. In some embodiments, the optical probe comprises optical components separate from the radiation source. In some embodiments, the one or more computer processors are configured to implement a trained machine learning algorithm. In some embodiments, the trained machine learning algorithm is configured to identify a tissue characteristic. In some embodiments, the radiation source is configured to deliver the radiation to the tissue based on the identification of the tissue characteristic. In some embodiments, the one or more computer processors are configured to implement a trained machine learning algorithm, the trained machine learning algorithm is configured to identify a tissue characteristic, and the radiation source is configured to deliver the radiation to the tissue based on the identification of the tissue characteristic.
In an aspect, the present disclosure provides a method for generating a depth profile of a tissue of a subject, comprising (a) using an optical probe to transmit an excitation light beam from a light source to a surface of the tissue, which pulses of the excitation light beam, upon contacting the tissue, yield signals indicative of an intrinsic property of the tissue, wherein the optical probe comprises one or more focusing units that simultaneously adjust a depth and a position of a focal point of the excitation light beam; (b) detecting at least a subset of the signals; and (c) using one or more computer processors programmed to process the at least the subset of the signals detected in (b) to generate the depth profile of the tissue.
In some embodiments, the excitation light beam is a pulsed light beam. In some embodiments, the excitation light beam is a single beam of light. In some embodiments, the single beam of light is a pulsed beam of light. In some embodiments, the excitation light beam comprises multiple beams of light. In some embodiments, the method further comprises (b) comprising simultaneously detecting a plurality of subsets of the signals. In some embodiments, the method further comprises processing the plurality of subsets of the signals to generate a plurality of depth profiles, wherein the plurality of depth profiles is indicative of a probe position at a time of detecting the signals. In some embodiments, the plurality of depth profiles corresponds to a same scanning path. In some embodiments, the scanning path comprises a slanted scanning path. In some embodiments the method further comprises assigning a least one distinct color for each of the plurality of depth profiles. In some embodiments, the method further comprises combining at least a subset of data from the plurality of depth profiles to form a composite depth profile. In some embodiments, the method further comprises displaying, on a display screen, a composite image derived from the composite depth profile. In some embodiments, the composite image is a polychromatic image. In some embodiments, color components of the polychromatic images correspond to multiple depth profiles using subsets of signals that are synchronized in time and location. In some embodiments, each of the plurality of layers comprise data that identifies different characteristics than those of other layers. In some embodiments, the depth profiles comprise a plurality of sub-set depth profiles, wherein the plurality of sub-set depth profiles comprise optical data from processed generated signals. In some embodiments, the plurality of depth profiles comprises a first depth profile and a second depth profile.
In some embodiments, the first depth profile comprises data processed from a signal that is different from data generated from a signal comprised in the second depth profile. In some embodiments, wherein the first depth and the second depth profile comprise one or more processed signals independently selected from the group consisting of a second harmonic generation (SHG) signal, a multi photon fluorescence signal, and a reflectance confocal microscopy (RCM) signal. In some embodiments, the plurality of depth profile comprises a third depth profile comprising data processed from a signal selected from the group consisting of a SHG signal, a multi photon fluorescence signal, and an RCM signal. In some embodiments, the depth profile comprises individual components, images, or depth profiles created from the plurality of subsets of the signals. In some embodiments, the depth profile comprises a plurality of layers created from a plurality of subsets of images collected from a same location and time. In some embodiments, the method further comprises generating a plurality of depth profiles. In some embodiments, each of the plurality of depth profiles corresponds to a different probe position. In some embodiments, the plurality of depth profiles corresponds to different scan patterns at the time of detecting the signals. In some embodiments, the different scan patterns correspond to a same time and probe position. In some embodiments, at least one scanning pattern of the different scan patterns comprises a slanted scanning pattern. In some embodiments, the slanted scanning pattern forms a slanted plane.
In some embodiments, the tissue comprises in vivo tissue. In some embodiments, (c) comprises generating an in vivo depth profile. In some embodiments, the depth profile is an annotated depth profile. In some embodiments, the annotation comprises at least one annotation selected from the group consisting of words and markings. In some embodiments, the signals comprise at least one signal selected from the group consisting of an SHG signal, a multi photon fluorescence signal, and an RCM signal. In some embodiments, the multi photon fluorescence signal comprises a plurality of multi photon fluorescence signals. In some embodiments, the signals comprise at least two signals selected from the group consisting of an SHG signal, a multi photon fluorescence signal, and an RCM signal. In some embodiments, the signals comprise an SHG signal, a multi photon fluorescence signal, and an RCM signal. In some embodiments, the signals further comprise at least one signal selected from the group consisting of third harmonic generation signals, coherent anti-stokes Raman scattering signals, stimulated Raman scattering signals, and fluorescence lifetime imaging signals.
In some embodiments, the signals are generated at a same time and location within the tissue. In some embodiments, the method further comprises prior to (a), contacting the tissue of the subject with the optical probe. In some embodiments, the method further comprises adjusting the depth and the position of the focal point of the excitation light beam along a scanning path. In some embodiments, the scanning path is a slanted scanning path. In some embodiments, the slanted scanning path forms a slanted plane positioned along a direction that is angled with respect to an optical axis of the optical probe. In some embodiments, an angle between the slanted plane and the optical axis is greater than 0 degrees and less than 90 degrees. In some embodiments, (a)-(c) are performed in an absence of administering a contrast enhancing agent to the subject. In some embodiments, the excitation light beam comprises unpolarized light. In some embodiments, the excitation light beam comprises polarized light. In some embodiments, the detecting is performed in a presence of ambient light. In some embodiments, (a) is performed without penetrating the tissue of the subject. In some embodiments, the method further comprises using the one or more computer processors to identify a characteristic of the tissue using the depth profile.
In some embodiments, the method further comprises using the one or more computer processors to identify a disease in the tissue. In some embodiments, the disease is identified with an accuracy of at least about 80%. In some embodiments, the disease is identified with an accuracy of at least about 90%. In some embodiments, the disease is a cancer. In some embodiments, the tissue is a skin of the subject, and wherein the cancer is skin cancer. In some embodiments, the depth profile has a resolution of at least about 0.8 micrometers. In some embodiments, the depth profile has a resolution of at least about 4 micrometers. In some embodiments, the depth profile has a resolution of at least about 10 micrometers. In some embodiments, the method further comprises measuring a power of the excitation light beam. In some embodiments, the method further comprises monitoring the power of the excitation light beam in real-time. In some embodiments, the method further comprises using the one or more computer processors to normalize for the power, thereby generating a normalized depth profile. In some embodiments, the method further comprises displaying a projected cross section image of the tissue generated at least in part from the depth profile. In some embodiments, the method further comprises displaying a composite of a plurality of layers of images. In some embodiments, each of the plurality of layers is generated by a corresponding depth profile of a plurality of depth profiles.
In another aspect, the present disclosure provides a system for generating a depth profile of a tissue of a subject, comprising: an optical probe that is configured to transmit an excitation light beam from a light source to a surface of the tissue, which the excitation light beam, upon contacting the tissue, yield signals indicative of an intrinsic property of the tissue, wherein the optical probe comprises one or more focusing units that are configured to simultaneously adjust a depth and a position of a focal point of the excitation light beam; one or more sensors configured to detect at least a subset of the signals; and one or more computer processors operatively coupled to the one or more sensors, wherein the one or more computer processors are individually or collectively programmed to process the at least the subset of the signals detected by the one or more sensors to generate a depth profile of the tissue.
In some embodiments, the excitation light beam is a pulsed light beam. In some embodiments, the pulsed light beam is a single beam of light. In some embodiments, the one or more focusing units comprise a z-axis scanner and a micro-electro-mechanical-system (MEMS) mirror. In some embodiments, the z-axis scanner comprises one or more lenses. In some embodiments, at least one of the one or more lenses is an afocal lens. In some embodiments, the z-axis scanner comprises an actuator. In some embodiments, the actuator comprises a voice coil. In some embodiments, the z-axis scanner and the MEMS mirror are separately actuated by two or more actuators controlled by the one or more computer processors. In some embodiments, the one or more computer processors are programmed or otherwise configured to synchronize movement of the z-axis scanner and the MEMS mirror. In some embodiments, the synchronized movement of the z-axis scanner and the MEMS mirror provides synchronized movement of one or more focal points at a slant angle.
In some embodiments, the signals comprise at least one signal selected from the group consisting of a second harmonic generation (SHG) signal, a multi photon fluorescence signal, and a reflectance confocal microscopy (RCM) signal. In some embodiments, the multi photon fluorescence signal comprises a plurality of multi photon fluorescence signals. In some embodiments, the signals comprise at least two signals selected from the group consisting of a SHG signal, a multi photon fluorescence signal, and an RCM signal. In some embodiments, the signals comprise a SHG signal, a multi photon fluorescence signal, and an RCM signal. In some embodiments, the tissue is epithelial tissue, and wherein the depth profile facilitates identification of a disease in the epithelial tissue of the subject. In some embodiments, the depth and the position of the focal point of the excitation light beam are adjusted along a scanning path. In some embodiments, the scanning path is a slanted scanning path. In some embodiments, the slanted scanning path is a slanted plane positioned along a direction that is angled with respect to an optical axis of the optical probe. In some embodiments, an angle between the slanted plane and the optical axis is between 0 degrees to 90 degrees.
In some embodiments, the light source comprises an ultra-fast pulse laser with a pulse duration less than about 200 femtoseconds. In some embodiments, during use, the optical probe is in contact with the surface of the tissue. In some embodiments, the system further comprises a sensor that detects a displacement between the optical probe and the surface of the tissue. In some embodiments, the optical probe is configured to receive at least one of the subsets of the signals, wherein the at least one of the subsets of the signals comprises at least one RCM signal. In some embodiments, the optical probe comprises a selective optic configured to send the at least one of the subsets of the signals into a fiber optic element. In some embodiments, the optical probe comprises an alignment arrangement configured to focus and align the at least one of the subsets of signals into the fiber optic element. In some embodiments, the alignment arrangement comprises a focusing lens and an adjustable refractive element between the focusing lens and the fiber optic element. In some embodiments, the focusing lens and the fiber optic element are in a fixed position with respect to the adjustable refractive element. In some embodiments, the adjustable refractive element is angularly movable. In some embodiments, the adjustable refractive element further comprises at least one adjustment element.
In some embodiments, the system further comprises a movable mirror, wherein the focusing lens is positioned between the movable mirror and the refractive element. In some embodiments, the system further comprises a polarizing selective optic positioned between a beam splitter and the focusing lens. In some embodiments, the selective optic comprises an optical filter selected from the group consisting of a beam splitter, a polarizing beam splitter, a notch filter, a dichroic mirror, a long pass filter, a short pass filter, a bandpass filter, and a response flattening filter. In some embodiments, the at least the subset of the signals comprises polarized light. In some embodiments, the optical probe comprises one or more polarization selective optics which select a polarization of the polarized light. In some embodiments, the at least the subset of the signals comprises an RCM signal from a polarization of the polarized light. In some embodiments, the at least the subset of signals comprise unpolarized light. In some embodiments, the optical probe is configured to reject out of focus light.
In some embodiments, the one or more sensors comprises one or more photosensors. In some embodiments, the system further comprises a marking tool for outlining a boundary that is indicative of a location of the disease in the tissue of the subject. In some embodiments, the system is a portable system. In some embodiments, the portable system is less than or equal to 50 pounds. In some embodiments, the optical probe comprises a housing configured to interface with a hand of a user. In some embodiments, the housing further comprises a sensor within the housing. In some embodiments, the sensor is configured to locate the optical probe in space. In some embodiments, the sensor is an image sensor, wherein the image sensor is configured to locate the optical probe in space by tracking one or more features. In some embodiments, the one or more features comprise features of the tissue of the subject. In some embodiments, the one or more features comprise features of a space wherein the optical probe is used. In some embodiments, the image sensor is a video camera. In some embodiments, the system further comprises an image sensor adjacent to the housing. In some embodiments, the image sensor locates the optical probe in space. In some embodiments, the one or more features comprise features of the tissue of the subject. In some embodiments, the one or more features comprise features of a space wherein the optical probe is used.
In some embodiments, the system further comprises a power sensor optically coupled to the excitation light beam. In some embodiments, the depth profile has a resolution of at least about 0.8 micrometers. In some embodiments, the depth profile has a resolution of at least about 4 micrometers. In some embodiments, the depth profile has a resolution of at least about 10 micrometers. In some embodiments, the depth profile is an in vivo depth profile. In some embodiments, the depth profile is an annotated depth profile. In some embodiments, the depth profile comprises a plurality of depth profiles. In some embodiments, the one or more computer processors are programmed to display a projected cross section image of tissue.
In another aspect, the present disclosure provides a method for analyzing tissue of a body of a subject, comprising: directing light to the tissue of the body of the subject; receiving a plurality of signals from the tissue of the body of the subject in response to the light directed thereto in (a), wherein at least a subset of the plurality of signals are from within the tissue; inputting data corresponding to the plurality of signals to a trained machine learning algorithm that processes the data to generate a classification of the tissue of the body of the subject; and outputting the classification on a user interface of an electronic device of a user.
In some embodiments, the data comprises at least one depth profile. In some embodiments, the at least one depth profile comprises one or more layers. In some embodiments, the one or more layers are synchronized in time and location. In some embodiments, the depth profile comprises one or more depth profiles synchronized in time and location. In some embodiments, the plurality of signals is generated substantially simultaneously by the light. In some embodiments, the depth profile comprises an annotated depth profile. In some embodiments, the depth profile comprises an in-vivo depth profile. In some embodiments, the trained machine learning algorithm comprises an input layer, to which the data is presented; one or more internal layers; and an output layer. In some embodiments, the input layer includes a plurality of the depth profiles using data processed from one or more signals that are synchronized in time and location. In some embodiments, the depth profiles are generated using the optical probe. In some embodiments, the depth profiles comprise individual components, images, or depth profiles generated from a plurality of the subsets of signals. In some embodiments, the depth profile comprises a plurality of layers generated from a plurality of subsets of images collected from the same location and time. In some embodiments, each of a plurality of layers comprises data that identifies different characteristics than those of the other layers. In some embodiments, the depth profiles comprise a plurality of sub-set depth profiles.
In some embodiments, the classification identifies a characteristic of the tissue. In some embodiments, the classification identifies features of the tissue in the subject pertaining to a property of the tissue selected from the group consisting of health, function, treatment, and appearance. In some embodiments, the classification identifies the subject as having a disease. In some embodiments, the disease is a cancer. In some embodiments, the tissue is a skin of the subject, and wherein the cancer is skin cancer. In some embodiments, the plurality of signals comprise at least one signal selected from the group consisting of an SHG signal, a multi photon fluorescence signal, and an RCM signal. In some embodiments, the plurality of signals comprise at least two signals selected from the group consisting of a SHG signal, a multi photon fluorescence signal, and an RCM signal. In some embodiments, the plurality of signals comprises a SHG signal, a multi photon fluorescence signal, and an RCM signal. In some embodiments, the multi photon fluorescence signal comprises one or more multi photon fluorescence signals. In some embodiments, (c) comprises identifying one or more features corresponding to the plurality of signals using the trained machine learning algorithm. In some embodiments, the trained machine learning algorithm comprises a neural network. In some embodiments, the neural network comprises an input layer, to which data is presented. In some embodiments, the neural network further comprises one or more internal layers and an output layer.
In some embodiments, the input layer comprises a plurality of depth profiles generated using at least a subset of the plurality of signals synchronized in time and location. In some embodiments, at least one of the plurality of depth profiles is generated using the optical probe, wherein the optical probe comprises one or more focusing units, wherein the one or more focusing units comprise a z-axis scanner and a MEMS mirror. In some embodiments, at least one of the plurality of depth profiles comprises individual components from a plurality of subsets of the plurality of signals. In some embodiments, at least one depth profile of the plurality of depth profiles comprises a plurality of layers generated from optical data collected from the same location and time. In some embodiments, each of the plurality of layers comprises data that identifies a different characteristic than those of another layers. In some embodiments, the depth profile comprises a plurality of sub-set depth profiles. In some embodiments, the neural network comprises a convolutional neural network. In some embodiments, the data is controlled for an illumination power of the optical signal.
In some embodiments, the methods described herein further comprises receiving or using medical data of the subject. In some embodiments, the medical data of the subject comprises at least one medical data selected from the group consisting of a physical condition, medical history, test results, current and past occupations, age, sex, race, skin type, Fitzpatrick skin type, other metrics for skin health and appearance, nationality of the subject, environmental exposure, mental health, and medications. The physical conditions of the subject may be obtained through one or more medical instruments. The one or more medical instruments may include, but not limited to, stethoscopes, suction devices, thermometers, tongue depressors, transfusion kits, tuning forks, ventilators, watches, stopwatches, weighing scales, crocodile forceps, bedpans, cannulas, cardioverters, defibrillators, catheters, dialyzers, electrocardiograph machines, enema equipment, endoscopes, gas cylinders, gauze sponges, hypodermic needles, syringes, infection control equipment, instrument sterilizers, kidney dishes, measuring tapes, medical halogen penlights, nasogastric tubes, nebulizers, ophthalmoscopes, otoscopes, oxygen masks and tubes, pipettes, droppers, proctoscopes, reflex hammers, sphygmomanometers, spectrometers, dermatoscopes, and cameras. In some embodiments, the physical condition comprises vital signs of the subject. The vital signs may be measurements of the patient's basic body functions. The vital signs may include body temperature, pulse rate, respiration rate, and blood pressure.
In some embodiments, the medical data comprises at least one medical data selected from the group consisting of structured data, time-series data, unstructured data, and relational data. In some embodiments, the medical data is uploaded to a cloud-based database. In some embodiments, the data comprises at least one medical data selected from the group consisting of structured data, time-series data, unstructured data, and relational data. In some embodiments, the data is uploaded to a cloud-based database. In some embodiments, the data is kept on a local device. In some embodiments, the data comprises depth profiles obtained of overlapping regions of the tissue.
In another aspect, the present disclosure provides a system for analyzing tissue of a body of a subject, comprising: an optical probe that is configured to (i) direct an excitation light beam to the tissue of the body of the subject, and (ii) receive a plurality of signals from the tissue of the body of the subject in response to the light excitation beam directed thereto in (i), wherein at least a subset of the plurality of signals are from within the tissue; and one or more computer processors operatively coupled to the optical probe, wherein the one or more computer processors are individually or collectively programmed to (i) receive data corresponding to the plurality of signals, (ii) input the data to a trained machine learning algorithm that processes the data to generate a classification of the tissue of the body of the subject, and (iii) output the classification on a user interface of an electronic device of a user.
In some embodiments, the excitation light beam is a pulsed light beam. In some embodiments, the pulsed light beam is a single beam of light. In some embodiments, the data comprises at least one depth profile. In some embodiments, the at least one depth profile comprises one or more layers. In some embodiments, the one or more layers are synchronized in time and location. In some embodiments, the depth profile comprises one or more depth profiles synchronized in time and location. In some embodiments, the depth profile comprises an annotated depth profile. In some embodiments, the depth profile comprises an in-vivo depth profile. In some embodiments, the trained machine learning algorithm comprises an input layer, to which the data is presented; one or more internal layers; and an output layer. In some embodiments, the input layer includes a plurality of the depth profiles using data processed from one or more signals that are synchronized in time and location. In some embodiments, the depth profiles are generated using the optical probe.
In some embodiments, the optical probe comprises one or more focusing units. In some embodiments, the one or more focusing units comprise a z-axis scanner and a micro-electro-mechanical-system (MEMS) mirror. In some embodiments, the z-axis scanner comprises one or more lenses. In some embodiments, at least one of the one or more lenses is an afocal lens. In some embodiments, the z-axis scanner comprises an actuator. In some embodiments, the actuator comprises a voice coil. In some embodiments, the z-axis scanner and the MEMS mirror are separately actuated by two or more actuators controlled by the one or more computer processors. In some embodiments, the one or more computer processors are programmed or otherwise configured to synchronize movement of the z-axis scanner and the MEMS mirror. In some embodiments, the synchronized movement of the z-axis scanner and the MEMS mirror provides synchronized movement of focal points at a slant angle.
In some embodiments, the optical probe and the one or more computer processors are in a same device. In some embodiments, the device is a mobile device. In some embodiments, the optical probe is part of a device, and wherein the one or more computer processors are separate from the device. In some embodiments, the one or more computer processors are part of a computer server. In some embodiments, the one or more computer processors are part of a distributed computing infrastructure. In some embodiments, the data is medical data. In some embodiments, the one or more computer processors are programmed to receive medical data of the subject.
In another aspect, the present disclosure provides a method for generating a trained algorithm for identifying a characteristic in a tissue of a subject, comprising: (a) collecting signals from training tissues of subjects that have been previously or subsequently identified as having the characteristic; (b) processing the signals to generate data corresponding to depth profiles of the training tissues of the subjects; and (c) using the data from (b) to train a machine learning algorithm to yield a trained algorithm in computer memory for identifying the characteristic in the tissue of the subject wherein the tissue is independent of the training tissues.
In some embodiments, the characteristic is a disease. In some embodiments, the characteristic is a characteristic corresponding to a property of the tissue selected from the group consisting of a health, function, treatment, and appearance of the tissue. In some embodiments, the data comprises data having a consistent labeling and consistent properties. In some embodiments, the consistent properties comprise properties selected from the group consisting of illumination intensity, contrast, color, size, and quality. In some embodiments, the data is normalized with respect to an illumination intensity. In some embodiments, the depth profiles correspond to different positions of an optical probe on the tissue. In some embodiments, (a) comprises generating one or more depth profiles using at least a subset of the signals. In some embodiments, (a) further comprises collecting signals from training tissues of subjects that have been previously or subsequently identified as not having the characteristic. In some embodiments, at least one signal collected from training tissues that have been previously or subsequently identified as not having the characteristic is used as a control with the at least one signal collected from the training tissue that has been previously or subsequently identified as not having the characteristic In some embodiments, the data for the control is obtained from the same subject. In some embodiments the data for the control is obtained from the same body part of the same subject. In some embodiments the data for the control is obtained adjacent to the training tissue identified as having the characteristic. In some embodiments, the at least the subset of the signals is synchronized in time and location. In some embodiments, the data correspond to the one or more depth profiles. In some embodiments, at least one of the one or more depth profiles comprises a plurality of layers.
In some embodiments, the plurality of layers is generated from a plurality of subsets of images collected at the same time and location. In some embodiments, each of the plurality of layers comprises data that identifies a different feature or characteristic than that of another layer. In some embodiments, each of the one or more depth profiles comprises a plurality of sub-set depth profiles. In some embodiments, the method further comprises training the machine learning algorithm using each of the plurality of sub-set depth profiles individually. In some embodiments, the method further comprises generating a composite depth profile using the plurality of sub-set depth profiles. In some embodiments, the method further comprises generating a plurality of composite depth profiles using the plurality of sub-set depth profiles. In some embodiments, the method further comprises using the composite depth profile to train the machine learning algorithm. In some embodiments, the method further comprises generating the one or more depth profiles using a first set of signals collected from a first region of a training tissue and a second set of signals from a second region of the training tissue. In some embodiments, the first region of the training tissue is different from the second region of the training tissue. In some embodiments, the first region of the training tissue has the disease. In some embodiments, the first region of training tissue is on the same subject as the second region of training tissue. In some embodiments the first region of training tissue is on the same body part of a subject as the second region of training tissue. In some embodiments the first region of tissue is adjacent the second region of tissue. In some embodiments, the first region is suspected to have the characteristic and the second region does not have the characteristic. In some embodiments the first region has the characteristic and the second region does not. According to some embodiments, the second region is a control sample for the first region. In some embodiments data from the at least one control region is collected within 24 hours, within 12 hours, within 8 hours, within 4 hours, within 2 hours, or within 1 hour from the time the data from the at least one first region is collected. In some embodiments, the signals comprise two or more signals. In some embodiments, the two or more signals are selected from the group consisting of a second harmonic generation (SHG) signal, a multi photon fluorescence signal, and a reflectance confocal microscopy (RCM) signal. In some embodiments, the two or more signals are substantially simultaneous signals of a single region of the tissue. In some embodiments, the two or more signals are processed and combined to generate a composite image.
In another aspect, the present disclosure provides a system for generating a trained algorithm for identifying a characteristic in a tissue of a subject, comprising: a database comprising data corresponding to depth profiles of training tissues of subjects that have been previously or subsequently identified as having the characteristic, which depth profiles are generated from processing signals collected from the training tissues; and one or more computer processors operatively coupled to the database, wherein the one or more computer processors are individually or collectively programmed to (i) retrieve the data from the database and (ii) use the data to train a machine learning algorithm to yield a trained algorithm in computer memory for identifying the characteristic in the tissue of the subject wherein the tissue is independent of the training tissues. In some embodiments, the database further comprises data corresponding to depth profiles of training tissues that have been previously or subsequently identified as not having the characteristic.
In some embodiments, the characteristic is a disease. In some embodiments, the characteristic corresponds to a characteristic of the tissue selected from the group consisting of a health, function, treatment, and appearance. In some embodiments, the one or more computer processors are programmed to receive optical data of one or more depth profiles. In some embodiments, the depth profiles are generated using signals collected from the training tissues. In some embodiments, the signals are synchronized in time and location. In some embodiments, the depth profiles comprise a plurality of layers. In some embodiments, the plurality of layers is generated from a plurality of subsets of images collected at the same time and location. In some embodiments, each of the plurality of layers comprises data that identifies a different characteristic than that of another layer. In some embodiments a plurality of depth profiles comprises data from at least one first region of suspected of having the characteristic and data from at least one second or control region not suspected of having the characteristic. In some embodiments the at least one first region and the at least one control region are of the same subject. In some embodiments, the at least one first region and the at least one control region are of the same body part of a subject. In some embodiments, the at least one first region is adjacent the at least one control region. In some embodiments data from the at least one first region is collected at the same clinical time as the data of the control region. In some embodiments data from the at least on control region is collected within at most about 48 hours, 24 hours, 12 hours, 8 hours, 4 hours, 2 hours, or 1 hour from the time the data from the at least one first region is collected. In some embodiments, the one or more computer processors are programmed to receive medical data of the subject.
In some embodiments, the depth profiles have one or more annotations. In some embodiments, the depth profiles are in vivo depth profiles. In some embodiments the depth profiles are depth profiles of one or more overlapping regions of the tissue. In some embodiments, the characteristic is a disease. In some embodiments, the characteristic is a characteristic corresponding to a property of the tissue selected from the group consisting of a health, function, treatment, and appearance of the tissue. In some embodiments, the data comprises data having a consistent labeling and consistent properties. In some embodiments, the consistent properties comprise properties selected from the group consisting of illumination intensity, contrast, color, size, and quality. In some embodiments, the data is normalized with respect to an illumination intensity. In some embodiments, the depth profiles correspond to different positions of an optical probe on or with respect to the tissue.
In another aspect, the present disclosure provides a method for aligning a light beam, comprising: (a) providing (i) a light beam in optical communication with a lens, wherein the lens is in optical communication with a refractive element, (ii) an optical fiber, and (iii) a detector in optical communication with the optical fiber, wherein the refractive element is positioned between the lens and the optical fiber; and (b) adjusting the refractive element to align the optical path with the optical fiber, wherein the optical path is thereby aligned with the optical fiber.
In some embodiments, a point spread function of the beamlet after interacting with the refractive element is sufficiently small to enable a resolution of the detector to be less than 1 micrometer. In some embodiments, the adjusting the position comprises applying a rotation to the refractive element. In some embodiments, the rotation is at most a 180° rotation. In some embodiments, the rotation is a rotation in at most two dimensions. In some embodiments, the rotation is a rotation in one dimension. In some embodiments, the method further comprises providing an adjustable mirror wherein the lens is fixed between the adjustable mirror and the adjustable refractive element and adjusting the adjustable mirror aligns the optical path prior to using the adjustable refractive element. In some embodiments, the providing the light beam comprises providing a generated light signal from an interaction with a tissue of a subject. In some embodiments, the tissue is an in vivo skin tissue.
In another aspect, the present disclosure provides a system for aligning a light beam, comprising: a light source that is configured to provide a light beam; a focusing lens in optical communication with the light source; an adjustable refractive element in optical communication with the lens; an optical fiber; and a detector in optical communication with the optical fiber, wherein the adjustable refractive element is positioned between the focusing lens and the optical fiber and is movable to align an optical path between the focusing lens and the optical fiber.
In some embodiments, the focusing lens and the optical fiber are fixed with respect to the adjustable refractive element. In some embodiments, the adjustable refractive element is angularly movable. In some embodiments, the system further comprises adjustment elements coupled to the adjustable refractive element, wherein the adjustment elements are configured to adjust a position of the adjustable refractive element. In some embodiments, the adjustment elements angularly move the adjustable refractive element. In some embodiments, the system further comprises a controller operatively coupled to the refractive element, wherein the controller is programmed to direct adjustment of the refractive element to align the optical path with the optical fiber. In some embodiments, the adjustment is performed without an input of a user. In some embodiments, the adjustment is performed by a user. In some embodiments, the system further comprises a beam splitter configured to direct light along the optical path towards the optical fiber. In some embodiments, the system further comprises a movable mirror positioned between the beam splitter and the focusing lens. In some embodiments, the system further comprises a polarization selective optic positioned on the optical path. In some embodiments, the polarization selective optic is positioned between the beam splitter and the focusing lens. In some embodiments, the refractive element is a flat window.
In some embodiments, the refractive element is a glass refractive element. In some embodiments, a point spread function of a beamlet of light after interacting with the refractive element is sufficiently small to enable a resolution of the detector to be less than 1 micrometer. In some embodiments, the refractive element has a footprint of less than 1,000 mm2. In some embodiments, the refractive element is configured to adjust a beamlet of light at most about 10 degrees. In some embodiments, the refractive element has a has a property that permits alignment of a beam of light exiting the lens to a fiber optic. In some embodiments, the diameter is less than about 20 microns. In some embodiments, the diameter is less than about 10 microns. In some embodiments, the fiber optic has a diameter of less than about 5 microns. In some embodiments, the property is at least one property selected from the group consisting of a refractive index, a thickness, and a range of motion. In some embodiments, an aberration introduced by the refractive element is less than 20% of a diffraction limit of the focusing lens. In some embodiments, the aberration is less than 10% of the diffraction limit. In some embodiments, the aberration is less than 5% of the diffraction limit. In some embodiments, the aberration is less than 2% of the diffraction limit. In some embodiments, the aberration is less than 1% of the diffraction limit.
In another aspect, the present disclosure provides a method for aligning a light beam, comprising: (a) providing (i) a light beam in optical communication with a beam splitter, wherein the beam splitter is in optical communication with a lens, wherein the lens is in optical communication with a refractive element, (ii) an optical fiber, and (iii) a detector in optical communication with the optical fiber, wherein an optical path from the refractive element is misaligned with respect to the optical fiber; (b) adjusting the refractive element to align the optical path with the optical fiber; and (c) directing the light beam to the beam splitter that splits the light beam into a beamlet, wherein the beamlet is directed through the lens to the refractive element that directs the beamlet along the optical path to the optical fiber, such that the detector detects the beamlet.
In another aspect, the present disclosure provides a system for aligning a light beam, comprising: a light source that is configured to provide a light beam; a beam splitter in optical communication with the light source; a lens in optical communication with the beam splitter; a refractive element in optical communication with the lens; an optical fiber; and a detector in optical communication with the optical fiber, wherein an optical path from the refractive element is misaligned with respect to the optical fiber, wherein the refractive element is adjustable to align the optical path with the optical fiber, such that, when the optical path is aligned with the optical fiber, the light beam is directed from the light source to the beam splitter that splits the light beam into a beamlet, wherein the beamlet is directed through the lens to the refractive element that directs the beamlet along the optical path to the optical fiber, such that the detector detects the beamlet.
Another aspect of the present disclosure provides a non-transitory computer readable medium comprising machine executable code that, upon execution by one or more computer processors, implements any of the methods above or elsewhere herein.
Another aspect of the present disclosure provides a system comprising one or more computer processors and computer memory coupled thereto. The computer memory comprises machine executable code that, upon execution by the one or more computer processors, implements any of the methods above or elsewhere herein.
Additional aspects and advantages of the present disclosure will become readily apparent to those skilled in this art from the following detailed description, wherein only illustrative embodiments of the present disclosure are shown and described. As will be realized, the present disclosure is capable of other and different embodiments, and its several details are capable of modifications in various obvious respects, all without departing from the disclosure. Accordingly, the drawings and description are to be regarded as illustrative in nature, and not as restrictive.
All publications, patents, and patent applications mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent, or patent application was specifically and individually indicated to be incorporated by reference. To the extent publications and patents or patent applications incorporated by reference contradict the disclosure contained in the specification, the specification is intended to supersede and/or take precedence over any such contradictory material.
The novel features of the invention are set forth with particularity in the appended claims. A better understanding of the features and advantages of the present invention will be obtained by reference to the following detailed description that sets forth illustrative embodiments, in which the principles of the invention are utilized, and the accompanying drawings (also “figure” and “FIG.” herein), of which:
While various embodiments of the invention have been shown and described herein, it will be obvious to those skilled in the art that such embodiments are provided by way of example only. Numerous variations, changes, and substitutions may occur to those skilled in the art without departing from the invention. It should be understood that various alternatives to the embodiments of the invention described herein may be employed.
The term “subject,” as used herein, generally refers to an animal, such as a mammal. A subject may be a human or non-human mammal. A subject may be a plant. A subject may be afflicted with a disease or suspected of being afflicted with or having a disease. The subject may not be suspected of being afflicted with or having the disease. The subject may be symptomatic. Alternatively, the subject may be asymptomatic. In some cases, the subject may be treated to alleviate the symptoms of the disease or cure the subject of the disease. A subject may be a patient undergoing treatment by a healthcare provider, such as a doctor.
The term “tissue characteristic” as used herein generally refers to a state of a tissue. Examples of a tissue characteristic include, but are not limited to a disease, an abnormality, a normality, a condition, a tissue hydration state, a tissue structure state, or a health state of tissue. A characteristic can be a pathology. A characteristic can be benign (e.g., information about a healthy tissue). A tissue characteristic can comprise one or more features that can aid in tissue classification or diagnosis. A tissue characteristic may be eczema, dermatitis, psoriasis, lichen planus, bullous pemphigoid, vasculitis, granuloma annulare, Verruca vulgaris, seborrhoeic keratosis, basal cell carcinoma, actinic keratosis, squamous cell carcinoma in situ (e.g., an intraepidermal carcinoma), squamous cell carcinoma, cysts, lentigo, melanocytic naevus, melanoma, dermatofibroma, scabies, fungal infection, bacterial infection, bums, wounds, and the like, or any combination thereof.
The term “feature,” as used herein, generally refers to an aspect of a tissue or other body part that is indicative of a given tissue characteristic or multiple tissue characteristics. Examples of features include, but are not limited to a property; physiology; anatomy; composition; histology; function; treatment; size; geometry; regularity; irregularity; optical property; chemical property; mechanical property or other property; color; vascularity; appearance; structural element; quality; age of a tissue of a subject; data corresponding to a tissue characteristic; spongiosis in acute eczema with associated lymphocyte exocytosis; acanthosis in chronic eczema; parakeratosis and/or perivascular lymphohistiocytic infiltrate; excoriation and/or signs of rubbing (e.g., irregular acanthosis and perpendicular orientation of collagen in dermal papillae) in chronic cases (e.g., lichen simplex); hyperkeratosis (e.g., parakeratosis), orthokeratosis; neutrophils in stratum corneum and squamous cell layer; hypogranulosis; epidermis is thin over dermal papillae; regular acanthosis, clubbed rete ridges; relatively little spongiosis; dilated capillaries in dermal papillae; perivascular lymphohistiocytic infiltrate; orthokeratosis; hypergranulosis; irregular acanthosis with saw-toothed rete ridges; colloid bodies in lower epidermis and upper dermis; liquefaction degeneration of the basal layer; lichenoid lymphohistiocytic infiltrate in upper dermis (e.g., interface dermatitis) and/or the epidermis; melanin incontinence; subepidermal blister; viable roof over new blister, necrotic over an old blister; variable perivascular infiltrate (e.g., lymphocytes, histiocytes, eosinophils); pre-bullous lesions may show spongiosis with eosinophil exocytosis (e.g., eosinophilic spongiosis); vessel wall damage (e.g., necrosis, hyalinisation, fibrin); invasion of inflammatory cells into vessel walls; red cell extravasation; nuclear dust from leucocytoclasia of neutrophils; ischaemic necrosis of the epidermis; normal epidermis; central foci of dermal collagen degeneration (e.g., necrobiosis), mucin accumulation; palisading of histiocytes; multinucleate giant cells; single-filing of inflammatory cells between collagen bundles (e.g., ‘busy’ dermis); hyperkeratosis, papillomatosis, acanthosis; basaloid keratinocytes; horn cysts; abundant melanin in basal layer and/or throughout epidermis; sharp demarcation of base of epidermal hyperplasia; location; cohesive nests of basaloid tumor cells (e.g., sometimes with a small amount of squamous differentiation); peripheral palisading of nuclei at the margins of cell nests; retraction artefact (e.g., clefts) around cell nests; variable inflammatory infiltrate and ulceration; hyperkeratosis and/or ulceration; columns of parakeratosis optionally overlying atypical keratinocytes optionally separated by areas of orthokeratosis; basal atypical keratinocytes with varying degrees of overlying loss of maturation, hyperchromatism, pleomorphism, increased and abnormal mitoses, dyskeratosis—full thickness change may be called ‘bowenoid actinic keratosis’; variable superficial perivascular or lichenoid chronic inflammatory infiltrate; solar elastosis; hyperkeratosis, parakeratosis; acanthosis; full thickness epidermal involvement by atypical keratinocytes, with pale vacuolated or multinucleated cells; in some lesions, pagetoid spread at the margins; proliferation of atypical keratinocytes; invasion of dermis; variable degrees of keratinisation, optionally squamous eddies or keratin pearls; cyst lined by squamous epithelium, optionally flattened, with a granular layer; lamellated keratin within cyst; hyperpigmented elongated rete ridges; increased melanocytes; squamous lining but no granular layer; dense keratin content; frequent calcification; variable epidermal changes (e.g., atrophy, hyperplasia, papillomatosis, horn cysts); nests of melanocytes/naevus cells at the dermo-epidermal junction (e.g., junctional naevus) and/or in the dermis (e.g., compound naevus, dermal naevus); naevus cells in the epidermis confined to the basal layer, optionally at the tips of the rete ridges; generally round naevus cells that show decreasing size of both the cells and the cell nests with increasing depth in the dermis (e.g., maturation); inflammation, inflammation that varies based on trauma state; asymmetrical proliferation of melanocytes; atypical melanocytes invading upwards through epidermis and downwards into dermis; variable cytological atypia (e.g., loss of maturation, pleomorphism, hyperchromatism, increased mitoses, prominent nucleoli); epidermal hyperplasia (optionally mimicking basal cell carcinoma); hyperpigmented basal layer; circumscribed but poorly demarcated proliferation of spindled fibroblasts; histiocytes and few giant cells; variable amounts of collagen; focal epidermal hyperplasia with hyperkeratosis, parakeratosis and papillomatosis (not verruca plana); trichilemmal keratinization; koilocytes (e.g., keratinocytes in upper squamous layer with vacuoles, large cytoplasmic eosinophilic aggregates, and/or pyknotic nuclei); tangential sections showing squamous cells surrounded by inflamed stroma; older lesions lacking cytoplasmic changes; viral nuclear inclusions, basophilic viral nuclear inclusions; striking papillomatosis (e.g., upward displacement of dermal papillae), stratum corneum exhibits parakeratosis with pointed mounds resembling church spires, extravasated erythrocytes or hemosiderin; granular layer is thickened with prominent keratohyalin granules and keratinocytes displaying perinuclear clearing (e.g., koilocytosis); lymphocytic infiltrate in upper dermis; involuting lesions having chronic inflammatory infiltrates in dermis and epidermis with degenerative epithelial changes; invaginated with numerous coarse, basophilic, intracytoplasmic keratohyalin granules resembling molluscum bodies, or the like, or any combination thereof.
The term “disease,” as used herein, generally refers to an abnormal condition, or a disorder of a biological function or a biological structure such as an organ, that affects part or all of a subject. A disease may be caused by factors originally from an external source, such as infectious disease, or it may be caused by internal dysfunctions, such as autoimmune diseases. A disease can refer to any condition that causes pain, dysfunction, distress, social problems, and/or death to the subject afflicted. A disease may be an acute condition or a chronic condition. A disease may refer to an infectious disease, which may result from the presence of pathogenic microbial agents, including viruses, bacteria, fungi, protozoa, multicellular organisms, and aberrant proteins as prions. A disease may refer to a non-infectious disease, including but not limited to cancer and genetic diseases. In some cases, a disease can be cured. In some cases, a disease cannot be cured. In some cases, the disease is epithelial cancer. An epithelial cancer is a skin cancer including, but not limited to, non-melanoma skin cancers, such as basal cell carcinoma (BCC) and squamous cell carcinoma (SCC), and melanoma skin cancers.
The terms “epithelial tissue” and “epithelium,” as used herein, generally refer to the tissues that line the cavities and surface of blood vessels and organs throughout the body. Epithelial tissue comprises epithelial cells of which there are generally three shapes: squamous, columnar, and cuboidal. Epithelial cells can be arranged in a single layer of cells as simple epithelium comprising either squamous, columnar, or cuboidal cells, or in layers of two or more cells deep as stratified (layered), comprising either squamous, columnar, and/or cuboidal.
The term “cancer,” as used herein, generally refers to a proliferative disorder caused or characterized by a proliferation of cells which may have lost susceptibility to normal growth control. Cancers of the same tissue type usually originate in the same tissue and may be divided into different subtypes based on their biological characteristics. Non-limiting examples of categories of cancer are carcinoma (epithelial cell derived), sarcoma (connective tissue or mesodermal derived), leukemia (blood-forming tissue derived) and lymphoma (lymph tissue derived). Cancer may involve any organ or tissue of the body. Examples of cancer include melanoma, leukemia, astrocytoma, glioblastoma, retinoblastoma, lymphoma, glioma, Hodgkin's lymphoma, and chronic lymphocytic leukemia. Examples of organs and tissues that may be affected by various cancers include the pancreas, breast, thyroid, ovary, uterus, testis, prostate, pituitary gland, adrenal gland, kidney, stomach, esophagus, rectum, small intestine, colon, liver, gall bladder, head and neck, tongue, mouth, eye and orbit, bone, joints, brain, nervous system, skin, blood, nasopharyngeal tissue, lung, larynx, urinary tract, cervix, vagina, exocrine glands, and endocrine glands. In some cases, a cancer can be multi-centric. In some cases, a cancer can be a cancer of unknown primary (CUP).
The term “lesion,” as used herein, generally refers to an area(s) of disease and/or suspected disease, wound, incision, or surgical margin. Wounds may include, but are not limited to, scrapes, abrasions, cuts, tears, breaks, punctures, gashes, slices, and/or any injury resulting in bleeding and/or skin trauma sufficient for foreign organisms to penetrate. Incisions may include those made by a medical professional, such as but not limited to, physicians, nurses, mid-wives, and/or nurse practitioners, and dental professionals during treatment such as a surgical procedure.
The term “light,” as used herein, generally refers to electromagnetic radiation. Light may be in a range of wavelengths from infrared (e.g., about 700 nm to about 1 mm) through the ultraviolet (e.g., about 10 nm to about 380 nm). Light may be visible light. Alternatively, light may be non-visible light. Light may include wavelengths of light in the visible and non-visible wavelengths of the electromagnetic spectrum.
The term “ambient light,” as used herein, generally refers to the light surrounding an environment or subject, such as the light at a location in which devices, methods and systems of the present disclosure are used, such as a point of care location (e.g., a subject's home or office, a medical examination room, or operating room).
The term “optical axis” as used herein, generally refers to a line along which there may be some degree of rotational symmetry in an optical system such as a camera lens or microscope. The optical axis may be a line passing through the center of curvature of a lens or spherical mirror and parallel to the axis of symmetry. The optical axis herein is may also be referred to as the Z axis. For a system of simple lenses and mirrors, the optical axis may pass through the center of curvature of each surface and coincide with the axis of rotational symmetry. The optical axis may be coincident with the system's mechanical axis, as in the case of off-axis optical systems. For an optical fiber, the optical axis (also called as fiber axis) may be along the center of the fiber core.
The term “position,” as used herein, generally refers to a location on a plane perpendicular to the optical axis as opposed to a “depth” which is parallel to the optical axis. For example, a position of a focal point can be a location of the focal point in the x-y plane. Whereas a “depth” position can be a location along a z axis (optical axis). A position of a focal point can be varied throughout the x-y plane. A focal point can also be varied simultaneously along the z axis. The position may be a position of a focal point.
The term “position” can also refer to the position of an optical probe (or housing) which can include: the location in space of the probe; the locations with respect to anatomical features of a subject; and the orientation or angle of the probe and/or its optics or optical axis. Position can mean the location or orientation of the probe in, on or near, tissue or tissue boundaries of a subject. Position can also mean a location with respect to other characteristics or features identified in a subject's tissue or with respect other data collected or observed from a subject's tissue. Position of an optical probe can also mean the location and/or orientation of the probe or its optics with respect to tags, markers, or guides.
The term “focal point” or “focal spot” as used herein generally refers to a point of light on an axis of a lens or mirror of an optical element to which parallel rays of light converge. The focal point or focal spot can be in a tissue sample to be imaged, from which a return signal is generated that can be processed to create depth profiles.
The term “focal plane” as used herein, generally refers a plane formed by focal points directed along a scan path. The focal plane can be where the focal point moves in an X and/or Y direction, along with a movement in a Z direction wherein the Z axis is generally an optical axis. A scan path may also be considered a focal path that comprises at least two focal points that define a path that is non-parallel to the optical axis. For example, a focal path may comprise a plurality of focal points shaped as a spiral. A focal path as used herein may or may not be a plane and may be a plane when projected on an X-Z or Y-Z plane. The focal plane may be a slanted plane. The slanted plane may be a plane that is oriented at an angle with respect to an optical axis of an optical element (e.g., a lens or a mirror). The angle may be between about 0° and about 90°. The slanted plane may be a plane that has non-zero Z axis components.
The term “depth profile,” as used herein, generally refers to information or optical data derived from the generated signals that result from scanning a tissue sample. The scanning a tissue sample can be with imaging focal points extending in a parallel direction to an optical axis or z axis, and with varying positions on an x-y axis. The tissue sample can be, for example, in vivo skin tissue where the depth profile can extend across layers of the skin such as the dermis, epidermis, and subcutaneous layers. A depth profile of a tissue sample can include data that when projected on an X-Z or Y-Z plane creates a vertical planar profile that can translate into a projected vertical cross section image. The vertical cross section image of the tissue sample derived from the depth profile can be vertical or approximately vertical. In some cases, a depth profile provides varied vertical focal point coordinates while the horizontal focal point coordinates may or may not vary. A depth profile may be in the form of at least one plane at an angle to an optical plane (on an optical axis). For example, a depth profile may be parallel to an optical plane or may be at an angle less 90 degrees and greater than 0 degrees with respect to an optical plane. A depth profile may be generated using an optical probe that is contacting a tissue at an angle. For example, a depth profile may not be perpendicular to the optical axis, but rather offset by the same degree as the angle the optical probe is contacting the tissue. A depth profile can provide information at various depths of the sample, for example at various depths of a skin tissue. A depth profile can be provided in real-time. A depth profile may or may not correspond to a planar slice of tissue. A depth profile may correspond to a slice of tissue on a slanted plane. A depth profile may correspond to a tissue region that is not precisely a planar slice (e.g., the slice may have components in all three dimensions). A depth profile can be a virtual slice of tissue or a virtual cross section. A depth profile can be optical data scanned from in-vivo tissue. The data used to create a projected cross section image may be derived from a plurality of focal points distributed along a general shape or pattern. The plurality of distributed points can be in the form of a scanned slanted plane, a plurality of scanned slanted planes, or non-plane scan patterns or shapes (e.g., a spiral pattern, a wave pattern, or other predetermined or random or pseudorandom patterns of focal points.) The location of the focal points used to create a depth profile may be changed or changeable to track an object or region of interest within the tissue, that is detected or identified during scanning or related data processing. A depth profile may be formed from one or more distinct return signals or signals that correspond to anatomical features or characteristics from which distinct layers of a depth profile can be created. The generated signals used to form a depth profile can be generated from an excitation light beam. The generated signals used to form a depth profile can be synchronized in time and location. A depth profile may comprise a plurality of depth profiles where each depth profile corresponds to a particular signal or subset of signals that correspond to anatomical feature(s) or characteristics. The depth profiles can form a composite depth profile generated using signals synchronized in time and location. Depth profiles herein can be in vivo depth profiles wherein the optical data is obtained of in vivo tissue. A depth profile can be a composite of a plurality of depth profiles or layers of optical data generated from different generated signals that are synchronized in time and location. A depth profile can be a depth profile generated from a subset of generated signals that are synchronized in time and location with other subsets of generated signals. A depth profile can include one or more layers of optical data, where each of the layer corresponds to a different subset of signals. A depth profile or depth profile optical data can also include data from processing the depth profile, the optical probe, optical probe position, other sensors, or information identified and corresponding to the time of the depth profile or other pertinent information. Additionally, other data corresponding to subject information such as, for example, medical data, physical conditions, or other data or characteristics, can also be included with optical data of a depth profile. Depth profiles can be annotated depth profiles with annotations or markings.
The term “projected cross section image” as used herein generally refers to an image constructed from depth profile information projected onto the XZ or YZ plane to create an image plane. In this situation, there may be no distortion in depths of structures relative to the surface of the tissue. The projected cross section image may be defined by the portion of the tissue that is scanned. A projected cross section image can extend in a perpendicular direction relative to the surface of the skin tissue. The data used to create a projected cross section image may be derived from a scanned slanted plane or planes, and/or non-plane scan patterns, shapes (e.g., a spiral, a wave, etc.), or predetermined or random patterns of focal points.
The term “fluorescence,” as used herein, generally refers to radiation that can be emitted as the result of the absorption of incident electromagnetic radiation of one or more wavelengths (e.g., a single wavelength or two different wavelengths). In some cases, fluorescence may result from emissions from exogenously provided tags or markers. In some cases, fluorescence may result as an inherent response of one or more endogenous molecules to excitation with electromagnetic radiation.
The term “autofluorescence,” as used herein, generally refers to fluorescence from one or more endogenous molecules due to excitation with electromagnetic radiation.
The term “multi-photon excitation,” as used herein, generally refers to excitation of a fluorophore by more than one photon, resulting in the emission of a fluorescence photon. In some cases, the emitted photon is at a higher energy than the excitatory photons. In some cases, a plurality of multi-photon excitations may be generated within a tissue. The plurality of multi-photon excitations may generate a plurality of multi-photon signals. For example, cell nuclei can undergo a two-photon excitation. As another example, cell walls can undergo a three-photon excitation. At least a subset of the plurality of signals may be different. The different signals may have different wavelengths which may be used for methods described herein. For example, the different signals (e.g., two-photon or three-photon signals) can be used to form a map which may be indicative of different elements of a tissue. In some cases, the map is used to train machine learning based diagnosis algorithms.
The terms “second harmonic generation” and “SHG,” as used herein, generally refer to a nonlinear optical process in which photons interacting with a nonlinear material are effectively “combined” to form new photons with about twice the energy, and therefore about twice the frequency and about half (½) the wavelength of the initial photons.
The terms “third harmonic generation” and “THG,” as used herein, generally refer to a nonlinear optical process in which photons interacting with a nonlinear material are effectively “combined” to form new photons with about three times the energy, and therefore about three times the frequency and about a third (⅓) the wavelength of the initial photons.
The term “reflectance confocal microscopy” or “RCM,” as used herein, generally refers to a process of collecting and/or processing reflected light from a sample (e.g., a tissue or any components thereof). The process may be a non-invasive process where a light beam is directed to a sample and returned light from the focal point within the sample (“RCM signal”) may be collected and/or analyzed. The process may be in vivo or ex vivo. RCM signals may trace a reverse direction of a light beam that generated them. RCM signals may be polarized or unpolarized. RCM signals may be combined with a pinhole, single mode fiber, multimode fiber, intersecting excitation and collection optical pathways, or other confocal arrangements that restrict the light collected to that portion arising from the focal point.
The term “polarized light,” as used herein, generally refers to light with waves oscillating in one plane. Unpolarized light can generally refer to light with waves oscillating in more than one plane.
The term “excitation light beam,” as used herein, generally refers to the focused light beam directed to tissue to create a generated signal. An excitation light beam can be a single beam of light. An excitation light beam can be a pulsed single beam of light. An excitation beam of light can be a plurality of light beams. The plurality of light beams can be synchronized in time and location as described herein. An excitation beam of light can be a pulsed beam or a continuous beam or a combination one or more pulsed and/or continuous beams that are delivered simultaneously to a focal point of tissue to be imaged. The excitation light beam can be selected depending upon the predetermined type of return signal or generated signal as described herein.
The term “generated signal” as used herein generally refers to a signal that is returned from the tissue resulting from direction of focused light, e.g. excitation light, to the tissue and including but not limited to reflected, absorbed, scattered, or refracted light. Generated signals may include, but are not limited to, endogenous signals arising from the tissue itself or signals from exogenously provided tags or markers. Generated signals may arise in either in vivo or ex vivo tissue. Generated signals may be characterized as either single-photon generated signals or multi-photon generated signals as determined by the number of excitation photons that contribute to a signal generation event. Single-photon generated signals may include but are not limited to reflectance confocal microscopy (“RCM”) signals, single-photon fluorescence, and single-photon autofluorescence. Single-photon generated signals, such as RCM, can arise from either a continuous light source, or a pulsed light source, or a combination of light sources that can be either pulsed or continuous. Single-photon generated signals may overlap. Single-photon generated signals may be deconvoluted. Multi-photon generated signals may be generated by at least 2, 3, 4, 5, or more photons. Multi-photon generated signals may include but are not limited to second harmonic generation, two-photon autofluorescence, two-photon fluorescence, third harmonic generation, three-photon autofluorescence, three-photon fluorescence, multi-photon autofluorescence, multi-photon fluorescence, and coherent anti-stokes Raman spectroscopy. Multi-photon generated signals can arise from either a single pulsed light source, or a combination of pulsed light sources as in the case of coherent anti-stokes Raman spectroscopy. Multi-photon generated signals may overlap. Multi-photon generated signals may be deconvoluted. Other generated signals may include but are not limited to Optical Coherence Tomography (OCT), single or multi-photon fluorescence/autofluorescence lifetime imaging, polarized light microscopy signals, additional confocal microscopy signals, and ultrasonography signals. Single-photon and multi-photon generated signals can be combined with polarized light microscopy by selectively detecting the components of said generated signals that are either linearly polarized light, circularly polarized light, unpolarized light, or any combination thereof. Polarized light microscopy may further comprise blocking all or a portion of the generated signal possessing a polarization direction parallel or perpendicular to the polarization direction of the light used to generate the signals or any intermediate polarization direction. Generated signals as described herein may be combined with confocal techniques utilizing a pinhole, single mode fiber, multimode fiber, intersecting excitation and collection optical pathways, or other confocal arrangements that restrict the light detected from the generated signal to that portion of the generated signal arising from the focal point. For example, a pinhole can be placed in a Raman spectroscopy instrument to generate confocal Raman signals. Raman spectroscopy signals may generate different signals based at least in part on different vibrational states present within a sample or tissue. Optical coherence tomography signals may use light comprising a plurality of phases to image a tissue. Optical coherence tomography may be likened to optical ultrasonography. Ultrasonography may generate a signal based at least in part on the reflection of sonic waves from features within a sample (e.g., a tissue).
The term “contrast enhancing agent,” as used herein, generally refers to any agent such as but not limited to fluorophores, metal nanoparticles, nanoshell composites and semiconductor nanocrystals that can be applied to a sample to enhance the contrast of images of the sample obtained using optical imaging techniques. Fluorophores can be antibody targeted fluorophores, peptide targeted fluorophores, and fluorescent probes of metabolic activity. Metallic nanoparticles can comprise metals such as gold and silver that can scatter light. Nanoshell composites can include nanoparticles comprising a dielectric core and metallic shell. Semiconductor nanocrystals can include quantum dots, for example quantum dots containing cadmium selenide or cadmium sulfide. Other contrasting agents can be used herein as well, for example by applying acetic acid to tissue.
The term “in real-time” and “real-time,” as used herein, generally refers to immediate, rapid, not requiring operator intervention, automatic, and/or programmed. Real-time may include, but is not limited to, measurements in femtoseconds, picoseconds, nanoseconds, milliseconds, seconds, as well as longer, and optionally shorter, time intervals.
The term “tissue” as used herein, generally refers to any tissue or content of tissue. A tissue may be a sample that is healthy, benign, or otherwise free of a disease. A tissue may be a sample removed from a subject, such as a tissue biopsy, a tissue resection, an aspirate (such as a fine needle aspirate), a tissue washing, a cytology specimen, a bodily fluid, or any combination thereof. The tissue from which images can be obtained can be any tissue or content of tissue of the subject including but not limited to connective tissue, epithelial tissue, organ tissue, muscle tissue, ligaments, tendons, a skin tissue, breast tissue, bladder, kidney tissue, liver tissue, colon tissue, thyroid tissue, cervical tissue, prostate tissue, lung tissue, cardiac tissue, heart tissue, muscle tissue, pancreas tissue, anal tissue, bile duct tissue, a bone tissue, bone marrow, uterine tissue, ovarian tissue, endometrial tissue, vaginal tissue, vulvar tissue, stomach tissue, ocular tissue, nasal tissue, sinus tissue, penile tissue, salivary gland tissue, gut tissue, gallbladder tissue, gastrointestinal tissue, bladder tissue, brain tissue, spinal tissue, neurons, cells representative of a blood-brain barrier, blood, hair, nails, keratin, collagen, or any combination thereof.
The term “numerical aperture” as used herein, generally refers to a dimensionless number that characterizes the range of angles over which the system can accept or emit light. Numerical aperture may be used in microscopy to describe the acceptance cone of an objective (and hence its light-gathering ability and resolution).
Whenever the term “at least,” “greater than,” or “greater than or equal to” precedes the first numerical value in a series of two or more numerical values, the term “at least,” “greater than” or “greater than or equal to” applies to each of the numerical values in that series of numerical values. For example, greater than or equal to 1, 2, or 3 is equivalent to greater than or equal to 1, greater than or equal to 2, or greater than or equal to 3.
Whenever the term “no more than,” “less than,” or “less than or equal to” precedes the first numerical value in a series of two or more numerical values, the term “no more than,” “less than,” or “less than or equal to” applies to each of the numerical values in that series of numerical values. For example, less than or equal to 3, 2, or 1 is equivalent to less than or equal to 3, less than or equal to 2, or less than or equal to 1.
The methods and systems disclosed herein may be used to form a depth profile of a sample of tissue by utilize scanning patterns that move an imaging beam focal point through the sample in directions that are slanted or angled with respect to the optical axis, in order to improve the resolution of the optical system imaging the samples (e.g., in vivo biologic tissues). The scanner can move its focal points in a line or lines and/or within a plane or planes that are slanted with respect to the optical axis in order to create a depth profile of tissue. The depth profile can provide a projected vertical cross section image generally or approximately representative of a cross section of the tissue that can be used to identify a possible disease state of the tissue. The methods and systems may provide a projected vertical cross section image of an in vivo sample of intact biological tissue formed from depth profile image components (e.g. scanned pattern of focal points). The methods and systems disclosed herein may also produce an image of tissue cross section that is viewed as a tissue slice but may represent different X-Y positions.
According to some embodiments the methods and systems disclosed herein may utilize a slanted plane or planes (or slanted focal plane or planes) formed by a scanning pattern of focal points within the slanted plane or planes. A system that can simultaneously control the X, Y, and Z positions of a focused spot may move the focus through a trajectory in the tissue. The trajectory can be predetermined, modifiable or arbitrary. A substantial increase in resolution may occur when scanning at an angle to the vertical Z axis (e.g., optical axis). The effect may arise, for example, because the intersection between a slanted plane and the point spread function (PSF) is much smaller than the PSF projection in the XZ or YZ plane. Thus, the effective PSF for a focused beam moved along a slanted line or in a slated plane may be smaller as the slant angle increases, approaching the lateral PSF resolution at an angle of 90° (at which point a scan direction line or scan plane can lie within the XY (lateral) plane). Slanted scanning or imaging as described herein, may be used with any type of return signal. Non-limiting examples of return signals can include generated signals described elsewhere herein.
A depth profile through tissue can be scanned at an angle (e.g., more than 0° and less than 90°) with respect to the optical axis, to ensure a portion of the scan trajectory is moving the focus in the Z direction. In some examples, modest slant angles may produce a substantial improvement in resolution. The effective PSF size can be approximated as PSFlateral/sin(θ) for modest angles relative to the Z axis, where θ is the angle between the z axis and the imaging axis. Additional detail may be found in
When the projected cross section image is constructed, the depth profile information derived from the generated signals resulting from the slant scanning, may be projected onto the XZ or YZ plane to create an image plane. In this situation, there may be no distortion in depths of structures relative to the surface of the tissue. This projected cross section image, in some representative embodiments, can comprise data corresponding to a plane optically sliced at one or more angles to the vertical. A projected cross section image can have vastly improved resolution while still representing the depths of imaged structures or tissue.
Disclosed herein are methods for generating a depth profile of a tissue of a subject. In an aspect, a method for generating a depth profile of a tissue of a subject may comprise using an optical probe to transmit an excitation light beam from a light source towards a surface of the tissue, which excitation light beam, upon contacting the tissue, generate signals indicative of an intrinsic property of the tissue; using one or more focusing units in the optical probe to simultaneously adjust a depth and a position of a focal point of the excitation light beam in a scanning pattern; detecting at least a subset of the signals generated upon contacting the tissue with the excitation light beam; and using one or more computer processors programmed to process the at least the subset of the signals detected to generate the depth profile of the tissue. The scanning pattern can comprise a plurality of focal points. The method described herein for generating a depth profile can alternatively utilize a combination of two or more light beams that are either continuous or pulsed and are collocated at the focal point.
The depth profile can be generated by scanning a focal point in a in a scanning pattern that includes one or more slanted directions. The scanning may or may not be in a single plane. The scanning may be in a slanted plane or planes. The scanning may be in a complex shape, such as a spiral, or in a predetermined, variable, or random array of points. A scanning pattern, a scanning plane, a slanted plane, and/or a focal plane may be a different plane from a visual or image cross section that can be created from processed generated signals. The image cross section can be created from processed generated signals resulting from moving imaging focal points across a perpendicular plane, a slanted plane, a non-plane pattern, a shape (e.g., a spiral, a wave, etc.), or a random or pseudorandom assortment of focal points.
The depth profile can be generated in real-time. For example, the depth profile may be generated while the optical probe transmits one or more excitation light beams from the light source towards the surface of the tissue. The depth profile may be generated at a frame rate of at least 1 frame per second (FPS), 2 FPS, 3 FPS, 4 FPS, 5 FPS, 10 FPS, or greater. In some cases, the depth profile may be generated at a frame rate of at most 10 FPS, 5 FPS, 4 FPS, 3 FPS, 2 FPS, or less. Frame rate may refer to the rate at which an imaging device displays consecutive images called frames. An image frame of the depth profile can provide a cross-sectional image of the tissue.
The image frame, or the area of an image, may be a quadrilateral with any suitable dimensions. An image frame may be rectangular, in some cases with equal sides (e.g., square), for example, depicting a 200 μm by 200 μm cross-section of the tissue. The image frame may depict a cross-section of the tissue having dimensions of at least about 50 μm by 50 μm, 100 μm by 100 μm, 150 μm by 150 μm, 200 μm by 200 μm, 250 μm by 250 μm, 300 μm by 300 μm, or greater. In some cases, the image frame may depict a cross-section of the tissue having dimensions of at most about 300 μm by 300 μm, 250 μm by 250 μm, 200 μm by 200 μm, 150 μm by 150 μm, 100 μm by 100 μm, 50 μm by 50 μm, or smaller. The image frame may not have equal sides.
The image frame may be at any angle with respect to the optical axis. For example, the image frame may be at an angle that is greater than about 0°, 5°, 10°, 15°, 20°, 25°, 30°, 35°, 40°, 45°, 50°, 60°, 70°, 80°, 90°, or more, with respect to the optical axis. The image frame may be at an angle that is less than or equal to about 90°, 85°, 80°, 75°, 70°, 65°, 60°, 50°, 40°, 30°, 20°, 10°, 5°, or less, with respect to the optical axis. In some cases, the angle is between any two of the values described above or elsewhere herein, e.g., between 0° and 50°.
The image frame may be in any design, shape, or size. Examples of shapes or designs include but are not limited to: mathematical shapes (e.g., circular, triangular, square, rectangular, pentagonal, or hexagonal), two-dimensional geometric shapes, multi-dimensional geometric shapes, curves, polygons, polyhedral, polytopes, minimal surfaces, ruled surfaces, non-orientable surfaces, quadrics, pseudospherical surfaces, algebraic surfaces, miscellaneous surfaces, Riemann surfaces, box-drawing characters, Cuisenaire rods, geometric shapes, shapes with metaphorical names, symbols, Unicode geometric shapes, other geometric shapes, or partial shapes or combination of shapes thereof. The image frame may be a projected image cross section image as described elsewhere herein.
The excitation light beam may be ultrashort pulses of light. Ultrashort pulses of light can be emitted from an ultrashort pulse laser (herein also referred to as an “ultrafast pulse laser”). Ultrashort pulses of light can have high peak intensities leading to nonlinear interactions in various materials. Ultrashort pulses of light may refer to light having a full width of half maximum (FWHM) on the order of femtoseconds or picoseconds. In some examples, an ultrashort pulse of light has a FWHM of at least about 1 femtosecond, 10 femtoseconds, 100 femtoseconds, 1 picosecond, 100 picoseconds, or 1000 picoseconds or more. In some instances, an ultrashort pulse of light may be a FWHM of at most about 1000 picoseconds, 100 picoseconds, 1 picosecond, 100 femtoseconds, 10 femtoseconds, 1 femtosecond or less. Ultrashort pulses of light can be characterized by several parameters including pulse duration, pulse repetition rate, and average power. Pulse duration can refer to the FWHM of the optical power versus time. Pulse repetition rate can refer to the frequency of the pulses or the number of pulses per second.
The probe can also have other sensors in addition to the power sensor. The information from the sensors can be used or recorded with the depth profile to provide additional enhanced information with respect to the probe and/or the subject. For example, other sensors within the probe can comprise probe position sensors, GPS sensors, temperature sensors, camera or video sensors, dermatoscopes, accelerometers, contact sensors, and humidity sensors.
Non-limiting examples of ultrashort pulse laser technologies include titanium (Ti):Sapphire lasers, mode-locked diode-pumped lasers, mode-locked fiber lasers, and mode-locked dye lasers. A Ti:Sapphire laser may be a tunable laser using a crystal of sapphire (Al2O3) that is doped with titanium ions as a lasing medium (e.g., the active laser medium which is the source of optical gain within a laser). Lasers, for example diode-pumped laser, fiber lasers, and dye lasers, can be mode-locked by active mode locking or passive mode locking, to obtain ultrashort pulses. A diode-pumped laser may be a solid-state laser in which the gain medium comprises a laser crystal or bulk piece of glass (e.g., ytterbium crystal, ytterbium glass, and chromium-doped laser crystals). Although the pulse durations may not be as short as those possible with Ti:Sapphire lasers, diode-pumped ultrafast lasers can cover wide parameter regions in terms of pulse duration, pulse repetition rate, and average power. Fiber lasers based on glass fibers doped with rare-earth elements such as erbium, ytterbium, neodymium, dysprosium, praseodymium, thulium, or combinations thereof can also be used. In some cases, a dye laser comprising an organic dye, such as rhodamine, fluorescein, coumarin, stilbene, umbelliferone, tetracene, malachite green, or others, as the lasing medium, in some cases as a liquid solution, can be used.
The light source providing ultrashort pulses of light can be a wavelength-tunable, ultrashort-pulsed Ti:Sapphire laser. A Ti:Sapphire laser can be a mode-locked oscillator, a chirped-pulse amplifier, or a tunable continuous wave laser. A mode-locked oscillator can generate ultrashort pulses with a duration between about a few picoseconds and about 10 femtoseconds, and in cases about 5 femtoseconds. The pulse repetition frequency can be about 70 to 90 megahertz (MHz). The term ‘chirped-pulse’ generally refers to a special construction that can prevent the pulse from damaging the components in the laser. In a ‘chirped-pulse’ laser, the pulse can be stretched in time so that the energy is not all located at the same point in time and space, preventing damage to the optics in the amplifier. The pulse can then be optically amplified and recompressed in time to form a short, localized pulse.
Ultrashort pulses of light can be produced by gain switching. In gain switching, the laser gain medium is pumped with, e.g., another laser. Gain switching can be applied to various types of lasers including gas lasers (e.g., transversely excited atmospheric (TEA) carbon dioxide lasers). Adjusting the pulse repetition rate can, in some cases, be more easily accomplished with gain-switched lasers than mode-locked lasers, as gain-switching can be controlled with an electronic driver without changing the laser resonator setup. In some cases, a pulsed laser can be used for optically pumping a gain-switched laser. For example, nitrogen ultraviolet lasers or excimer lasers can be used for pulsed pumping of dye lasers. In some cases, Q-switching can be used to produce ultrafast pulses of light.
Tissue and cellular structures in the tissue can interact with the excitation light beam in a wavelength dependent manner and generate signals that relate to intrinsic properties of the tissue. The signals generated can be used to evaluate a normal state, an abnormal state, a cancerous state, or other features of the tissue in a subject pertaining to the health, function, treatment, or appearance of the tissue, such as skin tissue, or of the subject (e.g., the health of the subject). The subset of the signals generated and collected can include at least one of second harmonic generation (SHG) signals, third harmonic generation (THG) signals, polarized light signals, and autofluorescence signals. A slanted plane imaging technique may be used with any generated signals as described elsewhere herein.
Higher harmonic generation microscopy (HHGM) (e.g., second harmonic generation and third harmonic generation), based on nonlinear multiphoton excitation, can be used to examine cellular structures in live and fixed tissues. SHG can generally refer to a nonlinear optical process in which photons with about the same frequency interact with a nonlinear material and effectively “combine” to generate new photons with about twice the energy, and therefore about twice the frequency and about half (½) the wavelength of the initial photons. Similarly, THG can generally refer to a nonlinear optical process in which photons with about the same frequency interact with a nonlinear material and effectively “combine” to generate new photons with about three times the energy, and therefore about three times the frequency and about one-third (⅓) the wavelength of the initial photons. Second harmonic generation (SHG) and third harmonic generation (THG) of ordered endogenous molecules, such as but not limited to collagen, microtubules, and muscle myosin, can be obtained without the use of exogenous labels and provide detailed, real-time optical reconstruction of molecules including fibrillar collagen, myosin, microtubules as well as other cellular information such as membrane potential and cell depolarization. The ordering and organization of proteins and molecules in a tissue, for example collagen type I and II, myosin, and microtubules, can generate, upon interacting with light, signals that can be used to evaluate the cancerous state of a tissue. SHG signals can be used to detect changes such as changes in collagen fibril/fiber structure that may occur in diseases including cancer, fibrosis, and connective tissue disorders. Various biological structures can produce SHG signals. In some cases, the labeling of molecules with exogenous probes and contrast enhancing agents, which can alter the way a biological system functions, may not be used. In some cases, methods herein for identifying a disease in an epithelial tissue of a subject may be performed in the absence of administering a contrast enhancing agent to the subject.
Another type of signal that can be generated and collected for determining a disease in a tissue may be autofluorescence. Autofluorescence can generally refer to light that is naturally emitted by certain biological molecules, such as proteins, small molecules, and/or biological structures. Tissue and cells can comprise various autofluorescent proteins and compounds. Well-defined wavelengths can be absorbed by chromophores, such as endogenous molecules, proteins, water, and adipose that are naturally present in cells and tissue. Non-limiting examples of autofluorescent fluorophores that can be found in tissues include polypeptides and proteins comprising aromatic amino acids such as tryptophan, tyrosine, and phenylalanine which can emit in the UV range and vitamin derivatives which can emit at wavelengths in a range of about 400 nm to 650 nm, including retinol, riboflavin, the nicotinamide ring of NAD(P)H derived from niacin, and the pyridolamine crosslinks found in elastin and some collagens, which are based on pyridoxine (vitamin B6).
The autofluorescence signal may comprise a plurality of autofluorescence signals. One or more filters may be used to separate the plurality of autofluorescence signals into one or more autofluorescence channels. For example, different parts of a tissue can fluoresce at different wavelengths, and wavelength selective filters can be used to direct each fluorescence wavelength to a different detector. One or more monochromators or diffraction gratings may be used to separate the plurality of autofluorescence signals into one or more channels.
Another type of signal that can be generated or collected for determining a disease in a tissue may be reflectance confocal microscopy (RCM) signals. RCM can use light that is reflected of a sample, such as a tissue, when a beam of light from an optical probe is directed to the sample. RCM signals may be a small fraction of the light that is directed to the sample. The RCM signals may be collected by rejecting out of focus light. The out of focus light may or may not be rejected using a pinhole, a single mode fiber optic, or a similar physical filter. The interaction of the sample with the beam of light may or may not alter the polarization of the RCM signal. Different components of the sample may alter the polarization of the RCM signals to different degrees. The use of polarization selective optics in an optical path of the RCM signals may allow a user to select RCM signal from a given component of the sample. The system can select, split, or amplify RCM signals that correspond to different anatomical features or characteristics to provide additional tissue data. For example, based on the changes in polarization detected by the system, the system can select or amplify RCM signal components corresponding to melanin deposits by selecting or amplifying the RCM signal that associated with melanin, using the polarization selective optics. Other tissue components including but are not limited to collagen, keratin, elastin can be identified using the polarization selective optics. Non-limiting examples of generated signals that may be detected are described elsewhere herein.
An ultra-fast pulse laser may produce pulses of light with pulse durations at most 500 femtoseconds, 450 femtoseconds, 400 femtoseconds, 350 femtoseconds, 300 femtoseconds, 250 femtoseconds, 200 femtoseconds, 150 femtoseconds, 100 femtoseconds, or shorter. In some cases, the pulse duration is about 150 femtoseconds. In some cases, an ultra-fast pulse laser may produce pulses of light with pulse durations at least 100 femtoseconds, 150 femtoseconds, 200 femtoseconds, 250 femtoseconds, 300 femtoseconds, 350 femtoseconds, 400 femtoseconds, 450 femtoseconds, 500 femtoseconds, or shorter. The pulse repetition frequency of an ultra-fast pulse laser can be at least 10 MHz, 20 MHz, 30 MHz, 40 MHz, 50 MHz, 60 MHz, 70 MHz, 80 MHz, 90 MHz, 100 MHz, or greater. In some cases, the pulse repetition frequency of an ultra-fast pulse laser can be at most 100 MHz, 90 MHz, 80 MHz, 70 MHz, 60 MHz, 50 MHz, 40 MHz, 30 MHz, 20 MHz, 10 MHz, or less. In some cases, the pulse repetition frequency is about 80 MHz.
The collected signals can be processed by a programmed computer processor to generate a depth profile. The signals can be transmitted wirelessly to a programmed computer processor. As an alternative, the signals may be transmitted through a wired connection to a programmed computer processor. The signals or a subset of the signals relating to an intrinsic property of the tissue can be used to generate a depth profile with the aid of a programmed computer processor. The collected signals and/or generated depth profile can be stored electronically. In some cases, the signals and/or depth profile are stored until deleted by a user, such as a surgeon, physician, nurse, or other healthcare practitioner. When used for diagnosis and/or treatment, the depth profile may be provided to a user in real-time. A depth profile provided in real-time can be used as a pre-surgical image to identify the boundary of a disease, for example skin cancer. The depth profile can provide a visualization of the various layers of tissue, such as skin tissue, including the epidermis, the dermis, and/or the hypodermis. The depth profile can extend at least below the stratum corneum, the stratum lucidum, the stratum granulosum, the stratum spinosum or the squamous cell layer, and/or the basal cell layer. In some cases, the depth profile may extend at least 250 μm, 300 μm, 350 μm, 400 μm, 450 μm, 500 μm, 550 μm, 600 μm, 650 μm, 700 μm, 750 μm, or farther below the surface of the tissue. In some cases, the depth profile may extend at most 750 μm, 700 μm, 650 μm, 600 μm, 550 μm, 500 μm, 450 μm, 400 μm, 350 μm, 300 μm, 250 μm, or less below the surface of the tissue. In some cases, the depth profile extends between about 100 μm and 1 mm, between about 200 μm and 900 μm, between about 300 μm and 800 μm, between about 400 μm and 700 μm, or between about 500 μm and 600 μm below the surface of the tissue.
The method may further comprise processing the depth profile using the one or more computer processors to identify a disease in the tissue. The identification of the disease in the tissue may comprise one or more characteristics. The one or more characteristics may provide a quantitative value or values indicative of one or more of the following: a likelihood of diagnostic accuracy, a likelihood of a presence of a disease in a subject, a likelihood of a subject developing a disease, a likelihood of success of a particular treatment, or any combination thereof. The one or more computer processors may also be configured to predict a risk or likelihood of developing a disease, confirm a diagnosis or a presence of a disease, monitor the progression of a disease, and monitor the efficacy of a treatment for a disease in a subject.
The method may further comprise contacting the tissue of the subject with the optical probe. The contact may be direct or indirect contact. If the contact is a direct contact, performing the contact may comprise placing the optical probe next to the tissue of the subject without an intervening layer. If the contact is an indirect contact, performing the contact may comprise placing the optical probe next to the tissue of the subject with one or more intervening layers. The one or more intervening layers may comprise, but are not limited to, clothes, medical gauzes, and bandages. The contact may be monitored such that when contact between the surface of the epithelial tissue and the optical probe is disrupted, a shutter positioned in front of the detector (e.g., relative to the path of light) can be activated and block incoming light.
According to some representative embodiments, the scanning pattern may follow a slanted plane. The slanted plane may be positioned along a direction that is angled with respect to an optical axis of the optical probe. The angle between the slanted plane and the optical axis may be at most 45°. The angle between the slanted plane and the optical axis may be greater than or equal to about 5°, 10°, 15°, 20°, 25°, 30°, 35°, 40°, 45°, 55°, 60°, 65°, 70°, 75°, 80°, 85°, or greater. In other cases, the angle between the slanted plane and the optical axis may be less than or equal to about 85°, 80°, 75°, 70°, 65°, 60°, 55°, 50°, 45°, 35°, 30°, 25°, 20°, 15°, 10°, 5°, or less. In some cases, the angle between the slanted plane and the optical axis may be between any of the two values described above, for example, between about 5° and 50°.
According to various representative embodiments, the scanning path or pattern may follow one or more patterns that are designed to obtain enhanced, improved, or optimized image resolution. The scanning path or pattern may comprise, for example, one or more perpendicular planes, one or more slanted planes, one or more spiral focal paths, one or more zigzag or sinusoidal focal paths, or any combination thereof. The scanning path or pattern may be configured to maintain the scanning focal points near the optical element's center while moving in slanted directions. The scanning path or pattern may be configured to maintain the scanning focal points near the center of the optical axis (e.g., the focal axis).
The scanning pattern of the plurality of focal points may be selected by an algorithm. For example, a series of images may be obtained using focal points moving at one or more scan angles (with respect to the optical axis). The scanning pattern may include perpendicular scanning and/or slant scanning. Depending upon the quality of the images obtained, one or more additional images may be obtained using different scan angles or combinations thereof, selected by an algorithm. As an example, if an image obtained using a perpendicular scan or a smaller angle slant scan is of low quality, a computer algorithm may direct the system to obtain images using a combination of scan directions or using larger scan angles. If the combination of scan patterns results in an improved image quality, then the imaging session may continue using that combination of scan patterns.
Different scan modalities through the tissue that utilize any cross section of the ellipse can be created by independently controlling the X, Y, Z location of the excitation ellipsoid. Any continuous parametric equation that describes a 3-dimensional volume can be used to scan the structure.
The benefit in resolution may occur when the scan pattern has a component in the X, Y, and Z directions, creating a slanted intersection of the PSF relative to the Z axis. There may be many different patterns, one example of which may be a single slanted plane that moves along a constant angle relative to the XZ plane. For instance, in
The method may be performed in an absence of removing the tissue from the subject. The method may be performed in an absence of administering a contrast enhancing agent to the subject.
The excitation light beam may comprise unpolarized light. In other embodiments, the excitation light beam may comprise polarized light. A wavelength of the excitation light beam can be at least about 400 nanometers (nm), 450 nm, 500 nm, 550 nm, 600 nm, 650 nm, 700 nm, 750 nm, 800 nm, 850 nm, 900 nm, 950 nm or longer. In some cases, a wavelength of the excitation light beam can be at most about 950 nanometers (nm), 900 nm, 850 nm, 800 nm, 750 nm, 700 nm, 650 nm, 600 nm, 550 nm, 500 nm, 450 nm, 400 nm or shorter. The wavelength of the pulses of light may be between about 700 nm and 900 nm, between about 725 nm and 875 nm, between about 750 nm and 850 nm, or between about 775 nm and 825 nm.
Multiple wavelengths may also be used. When multiple wavelengths of light are used, the wavelengths can be centered at least about 400 nm, 450 nm, 500 nm, 550 nm, 600 nm, 650 nm, 700 nm, 750 nm, 800 nm, 850 nm, 900 nm, 950 nm or longer with a bandwidth of at least about 10 nm, 20 nm, 30 nm, 40 nm, 50 nm, 75 nm, 100 nm, 125 nm, 150 nm, 175 nm, 200 nm, 225 nm, 250 nm, 275 nm, 300 nm or longer. For example, the wavelengths can be centered at about 780 nm with a bandwidth of about 50 nm (e.g., about ((780−(50/2))=755 nm) to about ((780+(50/2))=805 nm)). In some cases, the wavelengths can be centered at most about 950 nanometers (nm), 900 nm, 850 nm, 800 nm, 750 nm, 700 nm, 650 nm, 600 nm, 550 nm, 500 nm, 450 nm, 400 nm or shorter with a bandwidth of at least about 10 nm, 20 nm, 30 nm, 40 nm, 50 nm, 75 nm, 100 nm, 125 nm, 150 nm, 175 nm, 200 nm, 225 nm, 250 nm, 275 nm, 300 nm or longer.
The subset of the signals may comprise at least one of signal selected from the group consisting of second harmonic generation (SHG) signal, third harmonic generation (THG) signal, reflectance confocal microscopy (RCM) signal, and autofluorescence signal. SHG, THG, RCM, and autofluorescence are disclosed elsewhere herein. The subset of signals may comprise one or more generated signals as defined herein.
The collecting may be performed in a presence of ambient light. Ambient light can refer to normal room lighting, such as provided by various types of electric lighting sources including incandescent light bulbs or lamps, halogen lamps, gas-discharge lamps, fluorescent lamps, light-emitting diode (LED) lamps, and carbon arc lamps, in a medical examination room or an operating area where a surgical procedure is performed.
The simultaneously adjusting the depth and the position of the focal point of the excitation light beam along the slant scan, scan path or scan pattern may increase a maximum resolution depth of the depth profile. The maximum resolution depth after the increase may be at least about 1.1 times, 1.2 times, 1.5 times, 1.6 times, 1.8 times, 1.9 times, 2 times, 2.1 times, 2.2 times, 2.3 times, 2.4 times, 2.5 times, 2.6 times, 2.7 times, 2.8 times, 2.9 times, 3 times, or greater of the original maximum resolution depth. In other embodiments, the maximum resolution depth after the increase may be at most about 3 times, 2.9 times, 2.8 times, 2.7 times, 2.6 times, 2.5 times, 2.4 times, 2.3 times, 2.2 times, 2.1 times, 2.0 times, 1.9 times, 1.8 times, 1.7 times, 1.6 times, 1.5 times, 1.4 times, or less of the original maximum resolution depth. The increase may be relative to instances in which the depth and the position of the focal point may be not simultaneously adjusted.
The signals indicative of the intrinsic property of the tissue may be detected by a photodetector. A power and gain of the photodetector sensor may be modulated to enhance image quality. The excitation light beam may be synchronized with sensing by the photodetector.
The RCM signals may be detected by a series of optical components in optical communication with a beam splitter. The beam splitter may be a polarization beam splitter, a fixed ratio beam splitter, a reflective beam splitter, or a dichroic beam splitter. The beam splitter may transmit greater than or equal to about 1%, 3%, 5%, 10%, 15%, 20%, 25%, 33%, 50%, 66%, 75%, 80%, 90%, 99% or more of incoming light. The beam splitter may transmit less than or equal to about 99%, 90%, 80%, 75%, 66%, 50%, 33%, 25%, 20%, 15%, 10%, 5%, 3%, 1%, or less of incoming light. The series of optical components may comprise one or more mirrors. The series of optical components may comprise one or more lenses. The one or more lenses may focus the light of the RCM signal onto a fiber optic. The fiber optic may be a single mode, a multi-mode, or a bundle of fiber optics. The focused light of the RCM signal may be aligned to the fiber using an adjustable mirror, a translation stage, or a refractive alignment element. The refractive alignment element may be a refractive alignment element as described elsewhere herein.
The method may be performed without penetrating the tissue of the subject. Methods disclosed herein for identifying a disease in a tissue of a subject can be used during and/or for the treatment of the disease, for example during Mohs surgery to treat skin cancer. In some cases, identifying a disease, for example a skin cancer, in an epithelial tissue of a subject can be performed in the absence of removing the epithelial tissue from the subject. This may advantageously prevent pain and discomfort to the subject and can expedite detection and/or identification of the disease. The location of the disease may be detected in a non-invasive manner, which can enable a user such as a healthcare professional (e.g., surgeon, physician, nurse, or other practitioner) to determine the location and/or boundary of the diseased area prior to surgery. Identifying a disease in an epithelial tissue of a subject, in some cases, can be performed without penetrating the epithelial tissue of the subject, for example by a needle.
The disease or condition may comprise a cancer. In some cases, a cancer may comprise thyroid cancer, adrenal cortical cancer, anal cancer, aplastic anemia, bile duct cancer, bladder cancer, bone cancer, bone metastasis, central nervous system (CNS) cancers, peripheral nervous system (PNS) cancers, breast cancer, Castleman's disease, cervical cancer, childhood Non-Hodgkin's lymphoma, lymphoma, colon and rectum cancer, endometrial cancer, esophagus cancer, Ewing's family of tumors (e.g., Ewing's sarcoma), eye cancer, gallbladder cancer, gastrointestinal carcinoid tumors, gastrointestinal stromal tumors, gestational trophoblastic disease, hairy cell leukemia, Hodgkin's disease, Kaposi's sarcoma, kidney cancer, laryngeal and hypopharyngeal cancer, acute lymphocytic leukemia, acute myeloid leukemia, children's leukemia, chronic lymphocytic leukemia, chronic myeloid leukemia, liver cancer, lung cancer, lung carcinoid tumors, Non-Hodgkin's lymphoma, male breast cancer, malignant mesothelioma, multiple myeloma, myelodysplastic syndrome, myeloproliferative disorders, nasal cavity and paranasal cancer, nasopharyngeal cancer, neuroblastoma, oral cavity and oropharyngeal cancer, osteosarcoma, ovarian cancer, pancreatic cancer, penile cancer, pituitary tumor, prostate cancer, retinoblastoma, rhabdomyosarcoma, salivary gland cancer, sarcoma (adult soft tissue cancer), melanoma skin cancer, non-melanoma skin cancer, stomach cancer, testicular cancer, thymus cancer, uterine cancer (e.g., uterine sarcoma), vaginal cancer, vulvar cancer, or Waldenstrom's macroglobulinemia. The disease may be epithelial cancer. The epithelial cancer may be skin cancer.
The method may further comprise processing the depth profile using the one or more computer processors to classify a disease of the tissue. The classification may identify the tissue as having the disease at an accuracy, selectivity, and/or specificity of at least about 10%, 20%, 30%, 40%, 50%, 60%, 70%, 80%, 90%, 95%, 98%, 99%, 99.9%, or more. The classification may identify the tissue as having the disease at an accuracy, selectivity, and/or specificity of at most about 99.9%, 99%, 98%, 95%, 90%, 80%, 70%, 60%, 50%, 40%, 30%, 20%, 10%, or less. The one or more computer processors may classify the disease using one or more computer programs. The one or more computer programs may comprise one or more machine learning techniques. The one or more machine learning techniques may be trained on a system other than the one or more processors.
The depth profile may have a resolution of at least about 0.5, 0.6, 0.7, 0.8, 0.9, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 15, 20, 25, 30, 40, 50, 75, 100, 150, 200 microns, or more. The depth profile may have a resolution of at most about 200, 150, 100, 75, 50, 40, 30, 25, 20, 15, 10, 9, 8, 7, 6, 5, 4, 3, 2, 1, 0.9, 0.8, 0.7, 0.6, 0.5 microns, or less. For example, the depth profile may be able to resolve an intercellular space of 1 micron.
The method may further comprise measuring a power of the excitation light beam. A power meter may be used to measure the power of the excitation light beam. The power meter may measure the power of the excitation light beam in real time. The one or more computer processors may normalize a signal for the measured power of the excitation light beam. The normalized signal may be normalized with respect to an average power, an instantaneous power (e.g., the power read at the same time as the signal), or a combination thereof. The one or more computer processors may generate a normalized depth profile. The normalized depth profile may be able to be compared across depth profiles generated at different times. The depth profile may also include information related to the illumination power at the time the image was obtained. A power meter may also be referred to herein as a power sensor or a power monitor.
The method may allow for synchronized collection of a plurality of signals. The method may enable collection of a plurality of signals generated by a single excitation event. A depth profile can be generated using signals, as described elsewhere herein, that are generated from the same excitation event. A user may decide which signals to use to generate a depth profile.
The method may generate two or more layers of information. The two or more layers of information may be information generated from data generated from the same light pulse of the single probe system. The two or more layers may be from a same depth profile. Each of the two or more layers may also form separate depth profiles from which a projected cross section image may be created or displayed. For example, each separate layer, or each separate depth profile may correspond to a particular processed signal or signals that correspond to a particular imaging method. For example, a depth profile can be generated by taking two-photon fluorescence signals from melanin and another depth profile can be generated using SHG signals from collagen, and the two depth profiles can be overlaid as two layers of information. Each group of signals can be separately filtered, processed, and used to create individual depth profiles and projected cross section images, combined into a single depth profile with data that can be used to generate a projected cross section image, data from each group of signals can be combined and the combination can be used to generate a single depth profile, or any combination thereof. Each group of signals that correspond to a particular feature or features of the tissue can be assigned a color used to display the individual cross section images of the feature or features or a composite cross section image including data from each group of signals. The cross-sectional images or individual depth profiles can be overlaid to produce a composite image or depth profile. Thus, a multi-color, multi-layer, depth profile or image can be generated.
The present disclosure provides optical techniques that may be used for diagnosing epithelial diseases and skin pathologies. Optical imaging techniques can display nuclear and cellular morphology and may offer the capability of real-time detection of tumors in large areas of freshly excised or biopsied tissue without the need for sample processing, such as that of histology. Optical imaging methods can also facilitate non-invasive, real-time visualization of suspicious tissue without excising, sectioning, and/or staining the tissue sample. Optical imaging may improve the yield of diagnosable tissue (e.g., by avoiding areas with fibrosis or necrosis), minimize unnecessary biopsies or endoscopic resections (e.g., by distinguishing neoplastic from inflammatory lesions), and assess surgical margins in real-time to confirm negative margins (e.g., for performing limited resections). The ability to assess a tissue sample in real-time, without needing to wait for tissue processing, sectioning, and staining, may improve diagnostic turnaround time, especially in time-sensitive contexts, such as during Mohs surgery. Non-limiting examples of optical imaging techniques for diagnosing epithelial diseases and cancers include multiphoton microscopy, autofluorescence microscopy, polarized light microscopy, confocal microscopy, Raman spectroscopy, optical coherence tomography, and ultrasonography. Non-limiting examples of detectable tissue components include keratin, NADPH, melanin, elastin, flavins, protoporphyrin ix, and collagen. Other detectable components can include tissue boundaries. For example, boundaries between stratum corneum, epidermis, and dermis are schematically illustrated in
Multiphoton microscopy (MPM) can be used to image intrinsic molecular signals in living specimens, such as the skin tissue of a patient. In MPM, a sample may be illuminated with light at wavelengths longer than the normal excitation wavelength, for example twice as long or three times as long. MPM can include second harmonic generation microscopy (SHG) and third harmonic generation microscopy (THG). Third harmonic generation may be used to image nerve tissue.
Autofluorescence microscopy can be used to image biological molecules (e.g. fluorophores) that are inherently fluorescent. Non-limiting examples of endogenous biological molecules that are autofluorescent include nicotinamide adenine dinucleotide (NADH), NAD(P)H, flavin adenine dinucleotide (FAD), collagen, retinol, and tryptophan and the indoleamine derivatives of tryptophan. Changes in the fluorescence level of these fluorophores, such as with tumor progression, can be detected optically. Changes may be associated with altered cellular metabolic pathways (NADH, FAD) or altered structural tissue matrix (collagen).
Polarized light can be used to evaluate biological structures and examine parameters such as cell size and refractive index. Refractive index can provide information regarding the composition and organizational structure of cells, for example cells in a tissue sample. Cancer can significantly alter tissue organization, and these changes may be detected optically with polarized light.
Confocal microscopy may also be used to examine epithelial tissue. Exogenous contrast agents may be administered for enhanced visibility. Confocal microscopy can provide non-invasive images of nuclear and cellular morphology in about 2-5 μm thin sections in living human skin with lateral resolution of about 0.5-1.0 μm. Confocal microscopy can be used to visualize in vivo micro-anatomic structures, such as the epidermis, and individual cells, including melanocytes.
Raman spectroscopy may also be used to examine epithelial tissue. Raman spectroscopy may rely on the inelastic scattering (so-called “Raman” scattering) phenomena to detect spectral signatures of disease progression biomarkers such as lipids, proteins, and amino acids.
Optical coherence tomography may also be used to examine epithelial tissue. Optical coherence tomography may be based on interferometry in which a laser light beam is split with a beam splitter, sending some of the light to the sample and some of the light to a reference. The combination of reflected light from the sample and the reference can result in an interference pattern which can be used to determine a reflectivity profile providing information about the spatial dimensions and location of structures within the sample. Current, commercial optical coherence tomography systems have lateral resolutions of about 10 to 15 μm, with depth of imaging of about 1 mm or more. Although this technique can rapidly generate 3-dimensional (3D) image volumes that reflect different layers of tissue components (e.g., cells, connective tissue, etc), the image resolution (e.g., similar to the ×4 objective of a histology microscope) may not be sufficient for routine histopathologic diagnoses.
Ultrasound may also be used to examine epithelial tissue. Ultrasound can be used to assess relevant characteristics of epithelial cancer such as depth and vascularity. While ultrasonography may be limited in detecting pigments such as melanin, it can supplement histological analysis and provide additional detail to assist with treatment decisions. It may be used for noninvasive assessment of characteristics, such as thickness and blood flow, of the primary tumor and may contribute to the modification of critical management decisions.
Methods for diagnosing epithelial diseases and skin pathologies disclosed herein may comprise one or more of multiphoton microscopy, autofluorescence microscopy, polarized light microscopy, confocal microscopy, Raman spectroscopy, optical coherence tomography, and ultrasonography. In some cases, a method for diagnosing an epithelial disease and/or skin pathology comprises autofluorescence microscopy and multiphoton microscopy. As an alternative, a method for diagnosing an epithelial disease and/or skin pathology comprises autofluorescence microscopy, multiphoton microscopy, and polarized light microscopy. Both second harmonic generation microscopy and third harmonic generation microscopy can be used. In some cases, one of second harmonic generation microscopy and third harmonic generation microscopy is used.
Methods for diagnosing epithelial diseases and skin pathologies disclosed herein may comprise using one or more depth profiles to identify anatomical features and/or other tissue properties or characteristics and overlaying the images from the one or more depth profiles to an image from which a skin pathology can be identified.
Disclosed herein are apparatuses for generating depth profiles of tissues. In an aspect, an apparatus for generating a depth profile of a tissue of a subject may comprise an optical probe that transmits an excitation light beam from a light source towards a surface of the tissue, which excitation light beam, upon contacting the tissue, generate signals indicative of an intrinsic property of the tissue; one or more focusing units in the optical probe that simultaneously adjust a depth and a position of a focal point of the excitation light beam along a scan path, scan pattern or in one or more slant directions, one or more sensors configured to detect at least a subset of the signals generated upon contacting the tissue with the excitation light beam; and one or more computer processors operatively coupled to the one or more sensors, wherein the one or more computer processors are individually or collectively programmed to process the at least the subset of the signals detected by the one or more sensors to generate a depth profile of the tissue.
With continued reference to both
The illumination light may be directed to the afocal z-axis scanner 102 and then through MEMS mirror 106. The MEMS mirror scanner may be configured to direct at least a part of the light through one or more relay lenses 107. The one or more relay lenses 107 may be configured to direct the light to a dichroic mirror 108. The dichroic mirror 108 may direct the excitation light into an objective 110. The objective 110 may be configured to direct the light to interact with a tissue of a subject. The objective 110 may be configured to collect one or more signals generated by the light interacting with the tissue of the subject. The generated signals may be either single-photon or multi-photon generated signals. A subset of the one or more signals may be transmitted through dichroic mirror 108 into a collection arrangement 109, and may be detected by one or more photodetectors as described herein, for example of detector block 1108 of
Another subset of the one or more signals generated by light interacting with tissue and collected by the objective 110 may include single-photon signals. The subset of signals may be one or more RCM signals or single-photon fluorescence/autofluorescence signals. An RCM signal may trace a reverse path as the light that generated it. The reflected signal may be reflected by the beam splitter 104 towards an alignment arrangement that may align and focus the reflected signals or RCM signals onto an optical fiber 140. The alignment arrangement may comprise a focusing lens 132 and a refractive alignment element 133 with the refractive alignment element 133 positioned between the focusing lens 132 and optical fiber 140. The alignment arrangement may or may not comprise one or more additional optical elements such as one or more mirrors, lenses, and the like.
The reflected signal may be reflected by beam splitter 104 towards lens 132. The reflected signal may be directed to a focusing lens 132. The focusing lens 132 may be configured to focus the signal into optical fiber 140. The refractive alignment element 133 can be configured to align a focused beam of light from the focusing lens 132 into alignment with the fiber optic 140 for collection. According to some representative embodiments, the refractive alignment element 133 is moveably positioned between the focusing lens 132 and the optical fiber 140 while the focusing lens 132 and optical fiber 140 are fixed in their positions. The refractive element can be angularly or rotationally movable with respect to the focusing lens and optical fiber. The refractive alignment element 133 may be a refractive element as described elsewhere herein. The optical fiber 140 may be a single mode fiber, a multimode fiber, or a bundle of fibers. The optical fiber 140 may be coupled to a photodetector for detecting the reflected signal.
An optional polarizer 135 or polarization selective optical element may be positioned between the beam splitter and the focusing lens. The polarizer may provide further anatomical detail from the reflected signal. A mirror 131 may be used to direct reflected signals from the beam splitter 104 to the alignment arrangement. The mirror 131 can be movable and/or adjustable to provide larger alignment adjustments of the reflected signals before they enter the focusing lens 132. The mirror 131 can be positioned one focal length in front of the refractive alignment element 133. The mirror 131 may also be a beam splitter or may be polarized to split the reflected signal into elements with different polarizations to provide additional tissue detail from the reflected light. Once split, the split reflected signals can be directed through different alignment arrangements and through separate channels for processing.
The focusing lens 132 may focus the light of the RCM signal to a diffraction limited or nearly diffraction limited spot. The refractive alignment element 133 may be used to provide finer alignment of the light of the RCM signal to the fiber optic. The refractive alignment element can have a refractive index, a thickness, and/or a range of motion (e.g., a movement which alters the geometry) that permits alignment of the RCM signal exiting the lens to a fiber optic have a diameter less than about 20 microns, 10 microns, 5 microns, or less. According to some representative embodiments, the refractive alignment element properties (including refractive index, thickness, and range of motion) may be selected so that the aberrations introduced by the refractive alignment element do not increase the size the focused spot by greater than about 0%, 1%, 2%, 5%, 10%, 20%, or more above the focusing lens's diffraction limit. The optical fiber 140 may be coupled to a photodetector as described elsewhere herein. The photodetector may generate an image of a tissue. The refractive alignment element may enable RCM signal detection in a small form factor. The alignment arrangement can be contained within a handheld device.
The at least a subset of signals may comprise polarized light. The optical probe may comprise one or more polarization selective optics (e.g., polarization filters, polarization beam splitters, etc.). The one or more polarization selective optics may select for a particular polarization of RCM signal, such that the RCM signal that is detected is of a particular polarization from a particular portion of the tissue. For example, polarization selective optics can be used to selectively image or amplify different features in tissue.
The at least a subset of signals may comprise unpolarized light. The optical probe may be configured to reject up to all out of focus light. By rejecting out of focus light, a low noise image may be generated from RCM signals.
Multiple refractive lenses, such as relay lenses, collimating lenses, and field lenses, may be used to focus the ultrafast pulses of light from a light source to a small spot within the tissue. The small spot of focused light can, upon contacting the tissue, generate endogenous tissue signals, such as second harmonic generation, 2-photon autofluorescence, third harmonic generation, coherent anti-stokes Raman spectroscopy, reflectance confocal microscopy signals, or other nonlinear multiphoton generated signals. The probe may also transfer the scanning pattern generated by optical elements such as mirrors and translating lenses to a movement of the focal spot within the tissue to scan the focus through the structures and generate a point by point image of the tissue. The probe may comprise multiple lenses to minimize aberrations, optimize the linear mapping of the focal scanning, and maximize resolution and field of view.
The one or more focusing units in the optical probe may comprise, but are not limited to, movable lens, an actuator coupled to an optical element (e.g., an afocal lens), MEMS mirror, relay lenses, dichroic mirror, a fold mirror, a beam splitter, and/or an alignment arrangement. An alignment element may comprise but is not limited to a focusing lens, polarizing lens, refractive element, adjustment element for a refractive element, an angular adjustment element, and/or a movable mirror. The signals indicative of an intrinsic property of the tissue may be signals as described elsewhere herein, such as, for example, second harmonic generation signals, multi photon fluorescence signals, reflectance confocal microscopy signals, other generated signals, or any combination thereof.
Apparatuses consistent with the methods herein may comprise any element of the subject methods including, but not limited to, an optical probe; one or more light sources such as an ultrashort pulse laser; one or more mobile or tunable lenses; one or more optical filters; one or more photodetectors; one or more computer processors; one or more marking tools; and combinations thereof.
The photodetector may comprise, but are not limited to, a photomultiplier tube (PMT), a photodiode, an avalanche photodiode (APD), a charge-coupled device (CCD) detector, a charge-injection device (CID) detector, a complementary-metal-oxide-semiconductor detector (CMOS) detector, a multi-pixel photon counter (MPPC), a silicon photomultiplier (SiPM), light dependent resistors (LDR), a hybrid PMT/avalanche photodiode sensor, and/or other detectors or sensors. The system may comprise one or more photodetectors of one or more types, and each sensor may be used to detect the same or different signals. For example, a system can use both a photodiode and a CCD detector, where the photodiode detects SHG and multi photon fluorescence and the CCD detects reflectance confocal microscopy signals. The photodetector may be operated to provide a framerate, or number of images obtained per second, of at least about 0.5, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 15, 20, 24, or more. The photodetector may be operated to provide a framerate of at most about 60, 50, 40, 30, 24, 20, 15, 10, 9, 8, 7, 6, 5, 4, 3, 2, 1, 0.5, or less.
The optical probe may comprise a photomultiplier tube (PMT) that collects the signals. The PMT may comprise electrical interlocks and/or shutters. The electrical interlocks and/or shutters can protect the PMT when the photomultiplier compartment is exposed to ambient light by activating when contact between the surface of the epithelial tissue and the optical prove has been disrupted. By using activatable interlocks and/or shutters, signals can be collected in the presence of ambient light, thereby allowing a user to generate one or more real-time, pre-surgical depth profiles at the bedside of the patient. The optical probe may comprise other photodetectors as well
The light source providing ultrashort pulses of light can be a wavelength-tunable, ultrashort-pulsed Ti:Sapphire laser. A Ti:Sapphire laser can be a mode-locked oscillator, a chirped-pulse amplifier, or a tunable continuous wave laser. A mode-locked oscillator can generate ultrashort pulses with a duration between about a few picoseconds and about 10 femtoseconds, and in cases about 5 femtoseconds. The pulse repetition frequency can be about 70 to 90 megahertz (MHz). The term ‘chirped-pulse’ generally refers to a special construction that can prevent the pulse from damaging the components in the laser. In a ‘chirped-pulse’ laser, the pulse can be stretched in time so that the energy is not all located at the same point in time and space, preventing damage to the optics in the amplifier. The pulse can then be optically amplified and recompressed in time to form a short, localized pulse.
The mobile lens or movable lens of an apparatus can be translated to yield the plurality of different scan patterns or scan paths. The mobile lens may be coupled to an actuator that translates the lens. The actuator may be controlled by a programmed computer processor. The actuator can be a linear actuator, such as a mechanical actuator, a hydraulic actuator, a pneumatic actuator, a piezoelectric actuator, an electro-mechanical actuator, a linear motor, a linear electric actuator, a voice coil, or combinations thereof. Mechanical actuators can operate by converting rotary motion into linear motion, for example by a screw mechanism, a wheel and axle mechanism, and a cam mechanism. A hydraulic actuator can involve a hollow cylinder comprising a piston and an incompressible liquid. A pneumatic actuator may be similar to a hydraulic actuator but involves a compressed gas instead of a liquid. A piezoelectric actuator can comprise a material which can expand under the application of voltage. As a result, piezoelectric actuators can achieve extremely fine positioning resolution, but may also have a very short range of motion. In some cases, piezoelectric materials can exhibit hysteresis which may make it difficult to control their expansion in a repeatable manner. Electro-mechanical actuators may be similar to mechanical actuators. However, the control knob or handle of the mechanical actuator may be replaced with an electric motor.
Tunable lenses can refer to optical elements whose optical characteristics, such as focal length and/or location of the optical axis, can be adjusted during use, for example by electronic control. Electrically-tunable lenses may contain a thin layer of a suitable electro-optical material (e.g., a material whose local effective index of refraction, or refractive index, changes as a function of the voltage applied across the material). An electrode or array of electrodes can be used to apply voltages to locally adjust the refractive index to the value. The electro-optical material may comprise liquid crystals. Voltage can be applied to modulate the axis of birefringence and the effective refractive index of an electro-optical material comprising liquid crystals. In some cases, polymer gels can be used. A tunable lens may comprise an electrode array that defines a grid of pixels in the liquid crystal, similar to pixel grids used in liquid-crystal displays. The refractive indices of the individual pixels may be electrically controlled to give a phase modulation profile. The phase modulation profile may refer to the distribution of the local phase shifts that are applied to light passing through the layer as the result of the locally-variable effective refractive index over the area of the electro-optical layer of the tunable lens.
In some cases, an electrically or electro-mechanically tunable lens that is in electrical or electro-mechanical communication with the optical probe may be used to yield the plurality of different scan patterns or scan paths. Modulating a curvature of the electrically or electro-mechanically tunable lens can yield a plurality of different scan patterns or scan paths with respect to the epithelial tissue. The curvature of the tunable lens may be modulated by applying current. The apparatus may also comprise a programmed computer processor to control the application of current.
An apparatus for identifying a disease in an epithelial tissue of a subject may comprise an optical probe. The optical probe may transmit an excitation light beam from a light source towards a surface of the epithelial tissue. The excitation light beam, upon contacting the epithelial tissue, can then generate signals that relate to an intrinsic property of the epithelial tissue. The light source may comprise an ultra-fast pulse laser, such as a Ti:Sapphire laser. The ultra-fast pulse laser may generate pulse durations less than 500 femtoseconds, 400 femtoseconds, 300 femtoseconds, 200 femtoseconds, 100 femtoseconds, or less. The pulse repetition frequency of the ultrashort light pulses can be at least 10 MHz, 20 MHz, 30 MHz, 40 MHz, 50 MHz, 60 MHz, 70 MHz, 80 MHz, 90 MHz, 100 MHz, or greater.
The tissue may be epithelial tissue. The depth profile may permit identification of the disease in the epithelial tissue of the subject. The disease in the tissue of the subject is disclosed elsewhere herein.
The scanning path or pattern may be in one or more slant directions and on one or more slanted planes. A slanted plane may be positioned along a direction that is angled with respect to an optical axis of the optical probe. The angle between a slanted plane and the optical axis may be at most 45°. The angle between a slanted plane and the optical axis may be at least about 5°, 10°, 15°, 20°, 25°, 30°, 35°, 40°, 45°, 55°, 60°, 65°, 70°, 75°, 80°, 85°, or greater. In other cases, the angle between a slanted plane and the optical axis may be at most about 85°, 80°, 75°, 70°, 65°, 60°, 55°, 50°, 45°, 35°, 30°, 25°, 20°, 15°, 10°, 5°, or less.
The optical probe may further comprise one or more optical filters, which one or more optical filters may be configured to collect a subset of the signals. Optical filters, as described elsewhere herein, can be used to collect one or more specific subsets of signals that relate to one or more intrinsic properties of the epithelial tissue. The optical filters may be a beam splitter, a polarizing beam splitter, a notch filter, a dichroic filter, a long pass filter, a short pass filter, a bandpass filter, or a response flattening filter. The optical filters may be one or more optical filters. These optical filters can be coated glass or plastic elements which can selectively transmit certain wavelengths of light, such as autofluorescent wavelengths, and/or light with other specific attributes, such as polarized light. The optical filters can collect at least one signal selected from the group consisting of second harmonic generation (SHG) signal, third harmonic generation (THG) signal, polarized light signal, reflectance confocal microscopy (RCM) signal, and autofluorescence signal. The subset of the signals may include at least one of second harmonic generation (SHG) signals, third harmonic generation (THG) signals, and autofluorescence signals.
The light source may comprise an ultra-fast pulse laser with pulse durations less than about 200 femtoseconds. An ultra-fast pulse laser may produce pulses of light with pulse durations at most 500 femtoseconds, 450 femtoseconds, 400 femtoseconds, 350 femtoseconds, 300 femtoseconds, 250 femtoseconds, 200 femtoseconds, 150 femtoseconds, 100 femtoseconds, or shorter. In some cases, the pulse duration is about 150 femtoseconds. In some cases, an ultra-fast pulse laser may produce pulses of light with pulse durations at least 100 femtoseconds, 150 femtoseconds, 200 femtoseconds, 250 femtoseconds, 300 femtoseconds, 350 femtoseconds, 400 femtoseconds, 450 femtoseconds, 500 femtoseconds, or shorter. The pulse repetition frequency of an ultra-fast pulse laser can be at least 10 MHz, 20 MHz, 30 MHz, 40 MHz, 50 MHz, 60 MHz, 70 MHz, 80 MHz, 90 MHz, 100 MHz, or greater. In some cases, the pulse repetition frequency of an ultra-fast pulse laser can be at most 100 MHz, 90 MHz, 80 MHz, 70 MHz, 60 MHz, 50 MHz, 40 MHz, 30 MHz, 20 MHz, 10 MHz, or less. In some cases, the pulse repetition frequency is about 80 MHz.
During use, the optical probe may be in contact with the surface of the tissue. The contact may be direct or indirect contact. If the contact is a direct contact, performing the contact may comprise placing the optical probe next to the tissue of the subject without an intervening layer. If the contact is an indirect contact, performing the contact may comprise placing the optical probe next to the tissue of the subject with one or more intervening layers. The one or more intervening layers may comprise, but are not limited to, clothes, medical gauzes, bandages, and so forth. The contact may be monitored such that when contact between the surface of the epithelial tissue and the optical probe is disrupted, a shutter positioned in front of the detector (e.g., relative to the path of light) can be activated and block incoming light. In some cases, the photodetector comprises electrical interlocks and/or shutters. The electrical interlocks and/or shutters can protect the photodetector when the photomultiplier compartment is exposed to ambient light by activating when contact between the surface of the epithelial tissue and the optical prove has been disrupted. By using activatable interlocks and/or shutters, signals can be collected in the presence of ambient light, thereby allowing a user to generate one or more real-time, pre-surgical depth profiles at the bedside of the patient.
The apparatus may comprise a sensor that detects a displacement between the optical probe and the surface of the tissue. This sensor can protect the photodetector, for example a photodetector, from ambient light by activating a shutter or temporarily deactivating the photodetector to prevent ambient light from reaching and damaging the photodetector, if the ambient light exceeds the detection capacity of the photodetector.
The optical probe may comprise a power meter. The power meter may be optically coupled to the light source. The power meter may be used to correct for fluctuations of the power of the light source. The power meter may be used to control the power of the light source. For example, an integrated power meter can allow for setting a power of the light source depending on how much power is used for a particular imaging session. The power meter may ensure a consistent illumination over a period of time, such that images obtained throughout the period of time have similar illumination conditions. The power meter may provide information regarding the power of the illumination light to the system processing that can be recorded with the depth profile. The power information can be included in the machine learning described elsewhere herein. The power meter may be, for example, a photodiode, a pyroelectric power meter, or a thermal power meter. The power meter may be a plurality of power meters.
The apparatus may further comprise a marking tool for outlining a boundary that is indicative of a location of the disease in the epithelial tissue of the subject. The marking tool can be a pen or other writing instrument comprising skin marking ink that is FDA approved, such as Genetian Violet Ink; prep resistant ink that can be used with aggressive skin prep such as for example CHG/isopropyl alcohol treatment; waterproof permanent ink; or ink that is easily removable such as with an alcohol. A pen can have a fine tip, an ultra-fine tip, or a broad tip. The marking tool can be a sterile pen. As an alternative, the marking tool may be a non-sterile pen.
The apparatus may be a portable apparatus. The portable apparatus may be powered by a battery. The portable apparatus may comprise wheels. The portable apparatus may be contained within a housing. The housing can have a footprint of greater than or equal to about 0.1 ft2, 0.2 ft2, 0.3 ft2, 0.4 ft2, 0.5 ft2, 1 ft2, or more. As an alternative, the housing can have a footprint that is less than or equal to about 1 ft2, 0.5 ft2, 0.4 ft2, 0.3 ft2, 0.2 ft2, or 0.1 ft2. The portable apparatus may comprise a filtered light source that emits light within a range of wavelengths not detectable by the optical probe.
The portable apparatus may be at most 50 lbs, 45 lbs, 40 lbs, 35 lbs, 30 lbs, 25 lbs, 20 lbs, 15 lbs, 10 lbs, 5 lbs or less. In some cases, the portable apparatus may be at least 5 lbs, 10 lbs, 15 lbs, 20 lbs, 25 lbs, 30 lbs, 35 lbs, 40 lbs, 45 lbs, 50 lbs, 55 lbs or more.
The optical probe may comprise a handheld housing configured to interface with a hand of a user. An optical probe that can be translated may comprise a handheld and portable housing. This can allow a surgeon, physician, nurse, or other healthcare practitioner to examine in real-time the location of the disease, for example a cancer in skin tissue, at the bedside of a patient. The portable apparatus can have a footprint of greater than or equal to about 0.1 ft2, 0.2 ft2, 0.3 ft2, 0.4 ft2, 0.5 ft2, or 1 ft2. As an alternative, the portable apparatus can have a footprint that is less than or equal to about 1 ft2, 0.5 ft2, 0.4 ft2, 0.3 ft2, 0.2 ft2, or 0.1 ft2.
The probe may have a tip diameter that is less than about 10 millimeters (mm), 8 mm, 6 mm, 4 mm, or 2 mm. The handheld device may have a mechanism to allow for the disposable probe to be easily connected and disconnected. The mechanism may have an aligning function to enable precise optical alignment between the probe and the handheld device. The handheld device may be shaped like an otoscope or a dermatoscope with a gun-like form factor. The handheld device may have a weight of at most about 8 pounds (lbs), 4 lbs, 2 lbs, 1 lbs, 0.5 lbs, or 0.25 lbs. A screen may be incorporated into the handheld device to give point-of-care viewing. The screen may be detachable and able to change orientation. The handheld device may be attached to a portable system which may include a rolling cart or a briefcase-type configuration. The portable device may comprise a screen. The portable device may comprise a laptop computing device, a tablet computing device, a computing device coupled to an external screen (e.g., a desktop computer with a monitor), or a combination thereof. The portable system may include the laser, electronics, light sensors, and power system. The laser may provide light at an optimal frequency for delivery. The handheld device may include a second harmonic frequency doubler to convert the light from a frequency useful for delivery (e.g., 1,560 nm) to one useful for imaging tissue (e.g., 780 nm). For example, the delivery frequency may be at least about 800 nm, 900 nm, 1,000 nm, 1,100 nm, 1,200 nm, 1,300 nm, 1,400 nm, 1,500 nm, 1,600 nm, 1,700 nm, 1,800 nm, 1,900 nm, or more and the imaging frequency may be at least about 400 nm, 450 nm, 500 nm, 550 nm, 600 nm, 650 nm, 700 nm, 750 nm, 800 nm, 850 nm, 900 nm, 950 nm or more. The laser may be of low enough power to run the system on battery power. The system may further comprise a charging dock or mini-stand to hold the portable unit during operation. There may be many mini-stands in a single medical office and a singly portable system capable of being transported between rooms.
The housing may further comprise an image sensor. Alternatively, the image sensor may be located outside of the housing. In either case, the image sensor may be configured to locate the optical probe housing in space. The image sensor may locate the optical probe housing in space by tracking one or more features around the optical probe. The image sensor may be a video camera. The one or more features may be features of the tissue (e.g., freckles, birthmarks, etc.) or markers on or in the tissue placed by practitioners. The one or more features may be features of the space wherein the optical probe is used (e.g., furniture, walls, etc.). For example, the housing can have a number of cameras integrated into it that use a computer algorithm to track the position of the housing by tracking the movement of the furniture of the room the optical probe is being used in, and the tracking can be used to help generate a complete 3D image of a section of a tissue. By simultaneously tracking the position of the housing or optical probe position while recording images of tissue, a computer can reconstruct the location of the image within the tissue as the housing translates. In this way a larger mosaic region of the tissue can be imaged and digitally reconstructed. Such a region can be a 3D volume, or a 2D mosaic, or an arbitrary surface within the tissue. The image sensor may be configured to detect light in the near infrared. The housing may be configured to project a plurality of points to generate a map for the image sensor to use for tracking. In addition to using an image sensor, one or more position sensors, one or more other guides, or one or more sensors may be used with or by the optical probe or housing to locate the probe position with respect to the location of tissue features or tissue characteristics. A processor can identify the optical probe position with respect to currently or previously collected data. For example, identified features of the tissue can be used to identify, mark, or notate optical probe position. Current or previously placed tags or markers can also be used to identify optical probe position with respect to the tissue. Such tags or markers can include, without limitation, dyes, wires, fluorescent tracers, stickers, inked marks, incisions, sutures, mechanical fiducials, mechanical anchors, or other elements that can be sensed. A guide can be used with an optical probe to direct, mechanically reference, and/or track optical probe position. Optical probe position data can be incorporated into image data that is collected to create a depth profile.
The housing may contain optical elements configured to direct the at least a subset of the signals to one or more detectors. The one or more detectors may be optically coupled to the housing via one or more fiber optics. The housing may contain the one or more detectors as well as a light source, thus having an entirely handheld imaging system.
The one or more computer processors may be operatively coupled to the one or more sensors. The one or more sensors may comprise an infrared sensor, optical sensor, microwave sensor, ultrasonic sensor, radio-frequency sensors, magnetic sensor, vibration sensor, acceleration sensor, gyroscopic sensor, tilt sensor, piezoelectric sensor, pressure sensor, strain sensor, flex sensor, electromyographic sensor, electrocardiographic sensor, electroencephalographic sensor, thermal sensor, capacitive touch sensor, or resistive touch sensor.
According to some embodiments of the methods and apparatuses herein, an image can be depth profile as described herein and can include additional data as described herein. The depth profile may be an image. The images can also be portions of depth profiles as described herein and can be in the form of tiles or portions of image data. The images can be obtained in vivo. The first image and the second image can be captured with a time interval less than about 5 minutes, 15 minutes, 30 minutes, 45 minutes, 1 hour, 2 hours, 4 hours, 8 hours, 24 hours, or more. The first image and the second image can be captured with a time interval of greater than about 24 hours, 8 hours, 4 hours, 2 hours, 1 hour, 45 minutes, 30 minutes, 15 minutes, 5 minutes, or less. The signals can be collected and images, depth profiles, tiles, or datasets can be created without removing tissue from the body of the subject or fixing the tissue to a slide. The images can extend below a surface of the tissue. The images can have a resolution of at least about 1, 5, 10, 25, 50, 75, 100, 150, 200, 250, 300, 400, 500, 600, 700, 800, 900, 1,000 or more micrometers. The images can have a resolution of at most about 1,000, 900, 800, 700, 600, 500, 400, 300, 250, 200, 150, 100, 75, 50, 25, 10, 5, 1, or fewer micrometers. The images can comprise optical images. The images can be of a same size as one another. For example, the first image and the second image may both be 1024×1024 pixels.
Disclosed herein are methods for detecting or identifying a tissue characteristic in a subject, detecting or identifying a characteristic of tissue, generating a data set for a trained algorithm and generating a trained algorithm for classifying images of tissues from a subject. Classifying images of tissues may aid in identifying a disease in a tissue of a subject or in assessing, analyzing, or identifying other features of the tissue in a subject, for example, pertaining to the health, function, treatment, or appearance of the tissues or of the subject.
In an aspect, a method for generating a trained algorithm for identifying a disease in a tissue of a subject may comprise (a) collecting signals from training tissues of subjects that have been previously or subsequently identified as having the disease, which signals are selected from the group consisting of second harmonic generation signal, third harmonic generation signal, reflectance confocal microscopy signal, autofluorescence signal, and other generated signals as defined herein; (b) processing the signals to generate data corresponding to depth profiles of the training tissues of the subjects; and (c) using the data from (b) to train a machine learning algorithm to yield a trained algorithm in computer memory for identifying the disease in the tissue of the subject wherein the tissue is independent of the training tissues. Collecting the signals from training tissues of subjects in operation (a) above may comprise collecting signals from the training tissues of subjects to generate one or more depth profiles using signals that are synchronized in time and location. Such depth profiles, for example, may be generated using the optical probe as described elsewhere herein. Such depth profiles can comprise individual components, images or depth profiles created from a plurality of subsets of gathered and processed generated signals. The depth profile may comprise a plurality of layers created from a plurality of subsets of images collected from the same location and time. Each of the plurality of layers may comprise data that identifies different anatomical structures, tissue characteristics, and/or features than those of the other layer(s). Such depth profiles may comprise a plurality of sub-set depth profiles. Each of the subset of depth profiles may be individually trained and/or a composite depth profile of subset depth profiles may be trained. The subset of signals that form a subset of layers or depth profiles may comprise second harmonic generation signal, third harmonic generation signal, autofluorescence signal, RCM signals, other generated signals, and/or subsets or split sets of any of the foregoing as described elsewhere herein. A plurality of depths profiles can be generated in the training tissues of the subject by translating the optical probe. A portion of the plurality of depth profiles can be generated in a region of the training tissue with the suspected disease while a portion of the depth profiles can be generated outside of the region. For example, a portion of the plurality of depth profiles generated outside of the region may be used to collect subject control data. A method for generating a trained algorithm for identifying and classifying features of the tissue in a subject pertaining to the health, function, treatment, or appearance of the tissues or of a subject can proceed in a similar manner by collecting signals from training tissues of subjects that have been previously or subsequently identified as having the respective features. The respective features can include features used to identify disease and/or disfunction in tissue and/or to assess health, function or appearance of skin or tissue or of a subject.
A method for generating a trained algorithm for identifying and classifying features of the tissue in a subject pertaining to the health, function, treatment, or appearance of the tissue or of a subject can further proceed in a similar manner by collecting signals from training tissues of subjects that have a tissue characteristic and control tissue not having the tissue characteristic. Images, datasets, or tiles can be created from the collected signals from the tissue regions. The tissue, images, datasets, or tiles can be identified as having or not having the tissue characteristic, positive or negative, present or absent, or normal or abnormal. The images, datasets, or tiles that have been previously or subsequently identified as having the tissue characteristic and not having the tissue characteristic can be used to train an algorithm. The algorithm can then be used to classify tissue. The images, datasets, or tiles can be given scores, grades, or categories. The signals collected from training tissues can comprise a plurality of pairs or sets of data with present and absent features and/or tissue characteristics where each pair or group is from a single subject and has at least one positive and one control image, tile, or data set. The plurality of pairs or groups can be collected from a plurality of subjects or a single subject. The single subject may or may not be a subject to be treated. The positive and the control tissue can be on the same body part of the subject. The positive and control tissue can be adjacent normal and abnormal tissue.
A method of training a machine learning algorithm using images from both tissue with a tissue characteristic and tissue without the tissue characteristic can include collecting signals from training tissues of at least one subject that have a tissue characteristic (e.g., positive or present) and control tissue not having the tissue characteristic (e.g., negative or absent) and using the data sets to improve machine learning. The method can include obtaining first (positive) and second (control) images and repeating; and training a machine learning algorithm using at least a part of the data. The method can include hard negative mining and/or hard positive mining with images from either the tissue with the suspected tissue characteristic or the control tissue that are incorrectly classified. The method can utilize multiple instance learning where the images from the tissue with a tissue characteristic or suspected tissue characteristic and images from the control tissue are grouped into labeled “bags” each containing multiple images. The data sets can be obtained from a single individual or multiple individuals. The data sets or a portion of the data sets can be utilized to initialize parameters of a machine learning algorithm prior to training the algorithm. These methods can use imaging techniques described herein including collecting signals in vivo to create depth profiles or layered data. The methods can also include using movable optical probe tip to at one or more locations. The methods can also include altering and/or tracking the location and/or orientation of the optical probe to obtain collected signals, and using location data with collected data to train the algorithm. The methods can also include use of other subject data/information (e.g., medical data).
A method for generating a dataset comprising a plurality of images of tissue can include obtaining, via a handheld optical electronic device, a first image from a first tissue region of the subject and a second image from a second tissue region of the subject, wherein the first region is suspected of having or has a tissue characteristic, and wherein the second part is free or suspected of being free from the tissue characteristic; and storing data corresponding to the first image and the second image in a database. The first image and second image can be on the same body part of the subject. The first image and second image can be of adjacent tissue. The operations of obtaining the images and storing the data can be repeated to generate the dataset comprising a plurality of first images of the first tissue region. The operations of obtaining the images and storing the data can be repeated to generate the dataset comprising a plurality of second images of the second tissue region. The dataset can comprise a plurality of datasets from different subjects. The method can further comprise training a machine learning algorithm using at least part of the data.
A method of identifying tissue characteristics according to some methods and systems described herein can include imaging suspected tissue and control tissue of a subject and applying a trained algorithm to identify presence or absence of a tissue characteristic of tissue. Generated signals can be collected from a first tissue region of a subject having a suspected tissue characteristic and from a second tissue region of the subject without the tissue characteristic wherein the first tissue region and the second tissue region are from the same subject. The method can include collecting signals from the same body part of a subject and can also include collecting signals from adjacent tissue. The collected signals from both regions can be used to train an algorithm to detect or identify the tissue characteristic for example as described herein. A trained algorithm, for example as described herein, can be applied to the collected signals from both regions to detect or identify the tissue characteristic. Trained algorithms can be used to identify suspected tissue and can guide movement of the optical probe to identify additional tissue characteristics.
According to some embodiments, the first and second images can be obtained in vivo. The suspected tissue and control tissue can be of a same tissue type. The first and second images can be obtained on the same body part of a subject. The images can be obtained in adjacent tissue. The images can be depth profiles formed at the different locations or regions. The depth profiles can be layered images, or layered depth profiles as described herein. A subset of signals that form a subset of layers or depth profiles can comprise second harmonic generation signal, third harmonic generation signal, autofluorescence signal, RCM signals, other generated signals, and/or subsets or split sets of any of the foregoing as described elsewhere herein. The depth profile can be formed using imaging techniques described elsewhere herein. The optical scanning pattern can be set or determined by a trained algorithm, and can be modified during use, for example as different features are identified and used to model the data file(s)
The depth profiles can be obtained from different locations in real time or at a closely spaced times as described herein. The generated signals or data sets from the depth profiles can be created using a handheld optical probe and moving it to first and second regions or at different orientations. The handheld optical probe can also be moved to different locations or orientations with respect to a single region. The location and orientation of the handheld probe can be tracked during use and such tracking information can be added to the data files forming the depth profile, data sets, or tiles.
The classification can be determined by calculating a weighted sum of the one or more features for each of the first image and second image. The tissue of the subject under examination can be classified as positive or negative for the tissue characteristic based on a difference between said weighted sum of the one or more features for said first image and the weighted sum of the one or more features for the second image. The subject tissue can be classified as being positive or negative for the tissue disease or abnormality at an accuracy, specificity, and/or sensitivity of greater than or equal to about 40%, 50%, 60%, 70%, 80%, 90%, 91%, 92%, 93%, 94%, 95%, 96%, 97%, 98%, 99%, 99.9% or more. The subject tissue can be classified as being positive or negative for the tissue disease or abnormality at an accuracy, specificity, and/or sensitivity or less than or equal to about 99.9%, 99%, 98%, 97%, 96%, 95%, 94%, 93%, 92%, 91%, 90%, 80%, 70%, 60%, 50%, 40%, or less.
In addition to a positive or negative classification, other classifications can be identified. A trained algorithm can be applied to collected data from an examined subject to identify a likelihood of a presence or absence of a tissue characteristic. To identify a tissue characteristic or its likelihood or risk, different types of data sets can be created from the collected signal from tissue with and without a variety or plurality of different characteristics. The data sets can be derived from different subjects or a single subject. The single subject may or may not be the subject to be examined or diagnosed.
The trained algorithm can also use or identify markers of tissue health and function of a subject within control as well as suspected tissue. For example, markers of skin health and function can be used or identified such as, collagen content, hydration, cell, topology, proximity of cells, density, intercellular space, tissue geometry, cell nucleus features, microscale geometry, biological age of skin. This information can be combined with other medical information or data of the subject. The markers can be used to weight the risk or probability of a disease, condition, or other tissue characteristic existing. This can be used by the algorithm to detect tissue characteristics. Other features can be detected and used by a trained algorithm, such as, for example, features and types of tumor or stages of tumors.
Data derived from the first image and the second image can be transmitted to a computer system. The computer system can process the data and classify the tissue as described herein. A computer processor can be used to apply the trained algorithm to data to identify presence of absence of one or more features corresponding to the tissue characteristic. Using the trained algorithm, the computer processor can classify the subject as being positive or negative for the tissue characteristic based on the presence or absence of the one or more features in the image. The computer processor can be used to identify one or more features associated with the subject. An electronic report can be generated which is indicative of the subject being positive or negative for the tissue characteristic. The electronic report can be on a user interface of an electronic device used to collect the first image and the second image.
The computer processor can classify the tissue at an accuracy, specificity, and/or sensitivity as described elsewhere herein. The computer processor can also be used to identify a subject's risk for a disease, condition, or other tissue characteristic.
A method may comprise providing a treatment to the subject upon classifying tissue of the subject. The treatment may be provided in contemporaneous clinical visit as the imaging and classification. The treatment may be guided using the collected signals and the depth profiles as described herein. For example, the methods and devices herein can be used to identify disease boundaries and can guide medical procedures. Depth profiles can be obtained at several locations or orientations to identify disease margins during medical procedures to remove disease. Two- and/or three-dimensional images can be used for this purpose. Trained algorithms can determine whether to image in two or three dimensions depending upon what information or features are sought by a practitioner. Trained algorithms can be used to identify suspected tissue during a procedure and can guide movement of the optical probe to identify additional tissue to be treated. An example of a therapeutic procedure that can use an optical probe includes photo dynamic therapy where diseased cells can be eliminated while using an optical probe to identify diseased tissue or boundaries before, during, and after treatment. Real time feedback can be provided of an extent to which the treatment has eliminated diseased cells. According to some embodiments, a system or device may have a treatment function and imaging function that can be combined in a single handheld probe. A handheld probe can include an imaging element such as are described elsewhere herein and further comprise a treatment element. For example, a handheld probe may comprise a laser system configured to apply a laser treatment to a subject. In another example, the handheld probe can comprise a surgical knife for making an incision and removing a portion of a tissue.
In another aspect, the present disclosure provides a system for identifying and treating a tissue that may comprise an optical probe configured to optically obtain an image and/or a depth profile of the tissue and a treatment element configured to deliver treatment to the tissue. The treatment element may comprise a radiation source configured to deliver radiation to the tissue and a housing enclosing the optical imaging probe and the treatment element.
The housing may be handheld. The radiation source may comprise a light source. The radiation source may comprise one or more lasers. The radiation source may comprise one or more ionizing radiation sources (e.g., x-ray tubes, gamma ray sources). For example, a laser and a copper x-ray tube can be used to supply radiation. In a treatment mode, the radiation source may be configured to deliver radiation to the tissue. The radiation may heat the tissue. For example, a near-infrared laser can be used to supply heating radiation to the tissue. In a treatment mode, the radiation source may be configured to activate a beneficial process in the tissue. For example, the radiation source may be configured to promote a growth of the tissue. In another example, the radiation source may be configured to active a heat sensitive medicine in the tissue to impart a therapeutic effect. The radiation source may be configured to apply radiation to a limited area of the tissue. For example, the radiation source can apply laser light to ablate cancerous tissue while leaving benign tissue unharmed. In a detection mode, the radiation source may be configured to deliver the radiation to tissue that generates optical signals from the tissue. The optical probe may be configured to detect the optical signals. The optical signals may be generated signals as described elsewhere herein. One or more computer processors may be operatively coupled to the optical probe and the radiation source. The one or more computer processors may be configured to control a detection and/or a treatment mode of the system. The radiation source may be configured to be operated in detection and treatment modes simultaneously. For example, a laser can be configured to generate optical signals for detection as well as stimulate a beneficial response within the tissue. The optical probe may comprise an additional radiation source separate from the radiation source. For example, a first laser can be used to generate signals and image the tissue while a second laser can be used to provide treatment to the tissue. In another example, a laser can be configured to generate images of the tissue and an ionizing radiation source can be configured to supply ionizing radiation to the tissue to destroy a cancerous mass. The optical probe may comprise optical components separate from the radiation source. For example, the optical probe can comprise detection optics for detecting one or more signals. In another example, the optical probe can comprise a camera. The one or more computer processors may be configured to implement a trained machine learning algorithm. The trained machine learning algorithm may be a trained machine learning algorithm as described elsewhere herein. The trained machine learning algorithm may be configured to identify a tissue characteristic. The radiation source may be configured to deliver the radiation to the tissue based on the identification of the tissue characteristic. For example, the machine learning algorithm can intake signals generated by the optical probe, identify a tissue characteristic in the tissue, and direct a laser to apply laser radiation to the tissue region comprising the tissue characteristic.
The present disclosure provides methods and systems for identifying a tissue characteristic in a subject. In one aspect, the present disclosure provides a method of identifying a tissue characteristic in a subject that may comprise accessing a database comprising a first set of data from a first image obtained from a first tissue region of the subject and a second set of data from a second image obtained from a second tissue region of the subject. The first tissue region may be suspected of having the tissue characteristic. The second tissue region may be free or suspected of being free from having the tissue characteristic. The first set of data and the second set of data may be computer processed to (i) identify a presence or absence of one or more features indicative of the tissue characteristic in the first image, and (ii) classify the subject as being positive or negative for the tissue characteristic based on the presence or absence of the one or more features in the first image. An electronic report which is indicative of the subject being positive or negative for the tissue characteristic may be generated.
The tissue characteristic may be a disease or abnormality. The disease or abnormality may be cancer. The tissue characteristic may be a beneficial state. The first image and/or the second image may be obtained in vivo. The in vivo image may be obtained from a living tissue of the subject. For example, a first image of the skin of a subject can be an in vivo image. The first image and/or the second image may be obtained without removal of the first tissue region or the second tissue region from the subject. The first tissue region and/or the second tissue region may not be fixed to a slide. Not fixing the tissue to a slide may improve the speed of the image acquisition, as well as preserve fine features that may be destroyed in fixing the tissue to a slide. The first image and/or the second image may be generated using at least one non-linear imaging technique (e.g., second harmonic generation (SHG) signals, multiphoton autofluorescence, multiphoton fluorescence, coherent anti-Stokes Raman scattering, etc.). The first image and/or the second image may be generated using at least one linear imaging technique (e.g., optical coherence tomography, single photon fluorescence, reflectance confocal microscopy, brightfield microscopy, polarized microscopy, ultrasonic imaging, etc.). The first image and/or the second image may be generated using at least one non-linear imaging technique and at least one linear imaging technique. The image may be a depth profile as described elsewhere herein. The depth profile may be an image.
The first set of data and/or the second set of data may comprise groups of data. A group of data may comprise a plurality of images. The plurality of images may comprise (i) a positive image, and (ii) a negative image. The positive image may comprise one or more features. The negative image may not comprise the one or more features. The first set of data and/or the second set of data may comprise one or more sets of at least about 2 (e.g., pairs), 3, 4, 5, 6, 7, 8, 9, 10, or more instances of data. For example, the first data set can comprise a pair of instances of data with a first and second image. In another example, the second data set can have five sets each containing 4 images. The instances of data may be data as described elsewhere herein (e.g., images, signals, depth profiles). The electronic report may comprise information related to a risk of said tissue characteristic. For example, the electronic report can include information regarding the risk to the subject associated with the presence of the tissue characteristic. For example, the electronic report can include a general prognosis related to the presence of the tissue characteristic. The first image and/or the second image may be real-time depth profiles or layers of depth profiles as described elsewhere herein. For example, the first image can be a real time depth profile of a subject's skin layers. The first image and/or the second image may comprise one or more images of a tissue region adjacent to the first tissue region or the second tissue region. The first tissue region may be adjacent to the second tissue region. For example, a first image can be of the border of a suspected carcinoma and a second image can be of the suspected healthy skin on the other side of the border. In another example, the first image can be of a muscle tissue and the second image can be an image of the adjacent subcutaneous tissue. In another example, a user of a handheld probe can obtain a first image of a first tissue regions, lift the probe and place it onto the adjacent second tissue region, and obtain a second image. The user may additionally or alternatively change the orientation of the probe and obtain a second image. The first image may comprise a first sub-image of a third tissue region adjacent to the first tissue region. The second image may comprise a second sub-image of a fourth tissue region. For example, the first image can comprise both an image of a tissue region positive for a characteristic and an adjacent tissue region without the characteristic. In another example, the second image can comprise both an image of a tissue region free from the characteristic as well as a tissue region positive for a different characteristic. The first image and/or the second image may have a resolution of at least about 0.5, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 20, 30, 40, 50, 60, 70, 80, 90, 100, 250, 500, 1,000 or more micrometers. The first image and/or the second image may have a resolution of at most about 1,000, 500, 250, 100, 90, 80, 70, 60, 50, 40, 30, 20, 10, 9, 8, 7, 6, 5, 4, 3, 2, 1, 0.5, or fewer micrometers.
The first image and/or the second image may comprise one or more depth profiles. The depth profiles may be depth profiles as described elsewhere herein. The depth profiles may be one or more layered depth profiles of generated signals as described elsewhere herein. For example, a series of depth profiles which comprise layers generated from second harmonic generation (SHG) signals, reflectance confocal microscopy (RCM) signals, and multi-photon fluorescence signals can be used as first or second images. The depth profiles may be generated from a scanning pattern that moves in one or more slanted directions. The first image and/or the second image may comprise one or more layered images. Each layer of the first and/or second images may comprise at least one layer from different generated signals as described elsewhere herein (e.g., second harmonic generation (SHG) signals, third harmonic generation (THG) signals, reflectance confocal microscopy signals (RCM) signals, multi-photon fluorescence signals, multi-photon signals, etc.). For example, one layer of the layered image can be generated from a multi-photon fluorescence signal, and another layer can be generated from a second harmonic generation signal. Multiple layers of the layered image can be from a same type of generated signal. For example, two second harmonic generation signals collected at different wavelengths can each generate a layer of the layered image. The first image and/or the second image may be formed by one or more scanning patterns that move in one or more slanted directions as described elsewhere herein. The signals generated by the tissue may form depth profiles of the tissue in the first region and/or the second region. For example, a beam of light interacting with the tissue can generate a plurality of depth profiles. In this example, the beam of light can interact with both tissue in the first region and the second region to form depth profiles in the first and second regions. The first image may extend below a first surface of the first tissue region. The second image may extend below a second surface of the second tissue region. For example, a depth profile or an image can extend below the surface of a subject's skin.
The electronic report may be output on a user interface of an electronic device used to collect the first image and/or the second image. For example, user who used a handheld scanning device as described elsewhere herein can receive an electronic report on a screen coupled to the device. In another example, the electronic report can be displayed on a computer monitor coupled to the device. The electronic report may be sent as an electronic communication (e.g., email, short message service message, multimedia message service message). The electronic report may be stored on a local device (e.g., a computer, a mobile phone, a tablet, an imaging device) and/or the electronic report may be stored on a remote device (e.g., a server, a cloud storage device). The electronic report may be associated with the subject. For example, the electronic report can be included in a subject's medical record. The electronic report may comprise one or more determined characteristics, associated features, analyses, probabilities, likelihoods, frequencies, risks, severities of one or more of the forgoing, or the like, or any combination thereof.
The computer processing may comprise calculating a first weighted sum of one or more features for the first image and/or a second weighted sum of one or more features for the second image. The calculating the weighted sum may be a part of a machine learning algorithm. The computer processing may further comprise calculating a weighted sum for one or more additional images. For example, 10 images of the first tissue region can be obtained and each image can be processed. The computer processing may comprise classifying the subject as positive or negative for the tissue characteristic based at least in part on a difference between the first weighted sum and the second weighted sum.
The subject may be classified as being positive or negative for the tissue characteristic at an accuracy, sensitivity, and/or a specificity of at least about 40%, 50%, 60%, 70%, 80%, 90%, 91%, 92%, 93%, 94%, 95%, 96%, 97%, 98%, 99%, 99.9% or more. The subject may be classified as being positive or negative for the tissue characteristic at an accuracy, sensitivity, and/or a specificity of at most about 99.9%, 99%, 98%, 97%, 96%, 95%, 94%, 93%, 92%, 91%, 90%, 80%, 70%, 60%, 50%, 40%, or less. The subject may be classified as being positive or negative for the tissue characteristic at an accuracy, sensitivity, and/or a specificity of a range as defined by any two of the previous numbers. For example, the subject can be classified as having a skin cancer with an accuracy of about 90%-95% and a sensitivity of about 93%-94%.
The computer processing may comprise applying a trained machine learning algorithm. The machine learning algorithm may be trained as described elsewhere herein. The machine learning algorithm may be an algorithm as described elsewhere herein. The machine learning algorithm may be applied to the first set of data or the second set of data. The machine learning algorithm may have an accuracy, sensitivity, and/or a specificity of at least about 40%, 50%, 60%, 70%, 80%, 90%, 91%, 92%, 93%, 94%, 95%, 96%, 97%, 98%, 99%, 99.9% or more. The machine learning algorithm may have an accuracy, sensitivity, and/or a specificity of at most about 99.9%, 99%, 98%, 97%, 96%, 95%, 94%, 93%, 92%, 91%, 90%, 80%, 70%, 60%, 50%, 40%, or less. The machine learning algorithm may have an accuracy, sensitivity, and/or a specificity of a range as defined by any two of the previous numbers. The computer processing may comprise classifying the subject as being positive or negative for the tissue characteristic based on the presence or absence of the one or more features of the first image at an accuracy as described above. For example, the accuracy may be at least about 80%.
The first set of data may be data collected from one or more tissues having or suspected of having the tissue characteristic. The second set of data may be data collected from one or more tissues without the tissue characteristic. The first and/or second data set may be sorted, labeled, or otherwise marked to show the presence or absence of the tissue characteristic. For example, the first dataset can be annotated with indications of the presence of the tissue characteristic. The sets of data may be groups of data from one or more subjects having images positive and/or negative for the tissue characteristic. The one or more subjects may be different subjects. For example, the one or more subjects can comprise the subject being tested as well as an additional subject who is not currently being tested for the tissue characteristic. The one or more subjects may be the subject being tested for the characteristic. For example, images from another part of the subject being tested can be used in addition to the images of the area being tested. The database may further comprise one or more images from one or more additional subjects. The database may be a bank of a plurality of images collected over a period of time of different tissues both having and not having the characteristic. At least one of the one or more additional subjects may be positive for the tissue characteristic. At least one of the one or more additional subjects may be negative for the tissue characteristic. For example, the database can comprise a plurality of images of tissues of users who do not have the tissue characteristic, and the plurality of images can be used as a control for a machine learning algorithm. In another example, the database can comprise a plurality of images of tissues of users who are positive for the tissue characteristic that can be used as known positives to train a machine learning algorithm.
The computer processing may comprise computer processing a third data set from a third image of a third tissue region having the one or more features indicative of the tissue characteristic. For example, an additional image of the tissue having the characteristic can be obtained from the subject and processed. The computer processing may comprise computer processing a fourth data set from a fourth image of a fourth tissue region lacking the one or more features indicative of the tissue characteristic. The computer processing may comprise (i) computer processing a third data set from a third image of a third tissue region having the one or more features indicative of the tissue characteristic; and (ii) computer processing a fourth data set from a fourth image of a fourth tissue region lacking the one or more features indicative of the tissue characteristic. The third tissue region and/or the fourth tissue region may be of a different subject than the subject. For example, a bank of images comprising images of the third and fourth tissue regions can be used to improve the quality of the computer processing. The third tissue region and/or the fourth tissue region may be of the subject. For example, images of additional tissue regions of interest can be obtained to characterize those additional regions. In another example, multiple regions free from the characteristic can be used to generate a more general control group.
The first image may be obtained at least about 1 second (s), 5 s, 10 s, 30 s, 1 minute (m), 5 m, 10 m, 15 m, 30 m, 1 hour (h), 2 h, 3 h, 4 h, 5 h, 6 h, 7 h, 8 h, 9 h, 10 h, 12 h, 18 h, 24 h, 48 h, 72 h, 96 h, 120 h, 144 h, 168 h, or more prior to obtaining the second image. The first image may be obtained at most about 168 h, 144 h, 120 h, 96 h, 72 h, 48 h, 24 h, 18 h, 12 h, 10 h, 9 h, 8 h, 7 h, 6 h, 5 h, 4 h, 3 h, 2 h, 1 h, 30 m, 15 m, 10 m, 5 m, 1 m, 30 s, 10 s, 5 s, 1 s, or less prior to obtaining the second image. The first image may be obtained within at least about 1 second (s), 5 s, 10 s, 30 s, 1 minute (m), 5 m, 10 m, 15 m, 30 m, 1 hour (h), 2 h, 3 h, 4 h, 5 h, 6 h, 7 h, 8 h, 9 h, 10 h, 12 h, 18 h, 24 h, 48 h, 72 h, 96 h, 120 h, 144 h, 168 h, or more of obtaining the second image. The first image may be obtained within at most about 168 h, 144 h, 120 h, 96 h, 72 h, 48 h, 24 h, 18 h, 12 h, 10 h, 9 h, 8 h, 7 h, 6 h, 5 h, 4 h, 3 h, 2 h, 1 h, 30 m, 15 m, 10 m, 5 m, 1 m, 30 s, 10 s, 5 s, 1 s, or less of obtaining the second image. The first image may extend below a first surface of the first tissue region. The second image may extend below a second surface of the second tissue region. For example, the first image can be an image of the epidermis, the dermis, and the subcutaneous tissue. In another example, the second image can be an image of the dermis.
The present disclosure provides methods and systems of identifying a tissue characteristic in a subject. In another aspect, the present disclosure provides a method of identifying a tissue characteristic in a subject may comprise using an imaging probe, such as to obtain a first image from a first tissue region of the subject and a second image from a second tissue region. The first tissue region may be suspected of having the tissue characteristic. The second tissue region may be free or suspected of being free from the tissue characteristic. The data derived from the first image and the second image may be transmitted to a computer system. The computer system may process the data to (i) identify a presence or absence of the characteristic in the first image, and (ii) classify the subject as being positive or negative for the tissue characteristic based on the presence or absence of the one or more characteristics in the first image. A treatment may be provided to the subject upon classifying the subject as being positive for the characteristic.
The imaging probe may be configured to measure one or more electronic signals. The electronic signal may be or may be indicative of a current, a voltage, a charge, a resistance, a capacitance, a conductivity, an impedance, any combination thereof, or a change thereof. The imaging probe may comprise imaging optics. The imaging probe may be configured to measure one or more optical signals. Examples of imaging probes, including handheld optical probes, are provided elsewhere herein. Signals received by the imaging probe can be used to generate images of tissue regions from which signals were received. The imaging probe may be handheld. The imaging probe may be translated, lifted, or the orientation may be changed. For example, an imaging probe can be placed at an angle on a subject's skin and rotated to view tissue in a different location.
Before or after a treatment may be provided, the method may further comprise receiving an electronic report indicative of the tissue characteristic. The electronic report may be an electronic report as described elsewhere herein. The electronic report may comprise an indication of a risk associated with the characteristic. For example, a report can indicate how aggressive a carcinoma is expected to be. The electronic report may be displayed on a user interface of the imaging probe. The electronic report may be usable by a medical professional to form at least a part of a diagnosis related to the tissue characteristic. The electronic report may comprise suggested treatments. For example, an electronic report for a skin feature with a high likelihood of malignancy can suggest surgical removal of the skin feature. The electronic report may comprise other elements as described elsewhere herein.
The computer system may be a cloud-based computer system. For example, the first image and the second image can be processed on a system operatively coupled to the imaging probe to generate the data derived from the first image and the second image, and the data can be transmitted to a server for further processing. The computer system may be a computer system local to a user. For example, the transmitting can be transmitting within a computer system operatively coupled to the imaging probe. The computer system may comprise one or more machine learning algorithms. The one or more machine learning algorithms may be machine learning algorithms as described elsewhere herein. The one or more machine learning algorithms may be used to process the data. The data from the second image may be used as a control. For example, the second image can be used in part to develop a model of the appearance of a healthy tissue, which can improve the accuracy of the machine learning algorithm in determining the presence of the tissue characteristic in the first region.
The imaging probe may be a handheld imaging probe. The handheld imaging probe may be a handheld imaging probe as described elsewhere herein, including an optical probe described elsewhere herein. For example, the handheld imaging probe may be configured to generate depth profiles from a scanning pattern that moves in one or more slanted directions as described elsewhere herein. The handheld imaging probe may be translatable across a surface of the tissue. For example, the handheld imaging probe can be slid along the surface of the subject's skin to image a larger area. The handheld imaging probe may be translated between the first tissue region and the second tissue region, or from the second tissue region to the first tissue region. The orientation of the imaging probe may be directed to different regions. For example, the handheld imaging probe can be placed on a suspected carcinoma and drawn across the surface of the skin, recording depth profile through the carcinoma, the border of the carcinoma, and the surrounding health tissue. Translating the handheld imaging probe across the first and second tissue regions, changing the orientation of the probe, or otherwise moving the probe from one location to another location on the subject can generate a dataset comprising depth profiles and/or images of a tissue suspected of having or having the tissue characteristic, images of the border of the tissue suspected of having or having the tissue characteristic, as well as images of the tissue free from the tissue characteristic. The presence of all three of these image types can significantly improve the performance of a machine learning algorithm trained by or applied to the images. The position of the handheld imaging probe can be tracked during the obtaining the first and/or second images. The tracking may be tracking as described elsewhere herein. For example, one or more camera modules within or on the handheld imaging probe can record the locations of one or more of tracking markers to determine a three-dimensional position of the handheld imaging probe. In another example, the camera module can record a location of a one or more tracking markers and/or can record information from an internal sensor array comprising an accelerometer and a gyroscope.
The present disclosure provides methods and systems for identifying a tissue characteristic in a subject. In another aspect, the present disclosure provides a method of identifying a tissue characteristic in a subject may comprise accessing a database comprising data from an image obtained from a tissue region of the subject. The tissue region may be suspected of having the tissue characteristic. A trained algorithm may be applied to the data to (i) identify a presence or absence of one or more features indicative of the tissue characteristic in the image, and (ii) classify the subject as being positive or negative for the tissue characteristic based on the presence or absence of one or more features in the image at an accuracy of at least about 80%. An electronic report may be generated which is indicative of the subject being positive or negative for the tissue characteristic. The tissue characteristic may be indicative of a disease or an abnormality. The disease or abnormality may be cancer.
The present disclosure provides methods and systems for detecting a tissue characteristic in a subject. In another aspect, the present disclosure provides a method of detecting a tissue characteristic in a subject may comprise accessing a database comprising data from an image obtained from a tissue region of the subject. The tissue region may be suspected of having the tissue characteristic. The image may have a resolution of at least about 5 micrometers. A trained algorithm may be applied to the data to (i) identify a presence or absence of one or more features indicative of the tissue characteristic in the image, and (ii) classify the subject as being positive or negative for the tissue characteristic based on the presence or absence of the one or more features in the image. An electronic report may be generated which is indicative of the subject being positive or negative for the tissue characteristic. The tissue characteristic may be indicative of a disease or an abnormality. The disease or abnormality may be cancer.
The present disclosure provides methods and systems for generating a dataset comprising a plurality of images of a tissue of a subject. In another aspect, the present disclosure provides a method for generating a dataset comprising a plurality of images of a tissue of a subject may comprise obtaining, via a handheld imaging probe, a first image from a first part of said tissue of said subject and a second set of images from a second part of said tissue of said subject. The first part may be suspected of having a tissue characteristic. The second part may be free or suspected of being free from said tissue characteristic. Data corresponding to the first image and the second image may be stored in a database.
The handheld imaging probe may comprise imaging optics. The handheld imaging probe may be a handheld imaging probe as described elsewhere herein. For example, the handheld imaging probe can detect second harmonic generation signals, reflectance confocal microscopy signals, and multiphoton fluorescence signals and comprise a refractive alignment element. The handheld imaging probe may be translatable across a surface of the tissue. The handheld imaging probe may be rotated to change the orientation of the optical or sensing elements. The handheld imaging probe may be configured to be lifted from the surface of the tissue and placed at a different point on the tissue. For example, a user can place the handheld imaging probe onto a skin region suspected of having a melanoma, obtain one or more images, move the handheld imaging probe to image a skin region clear of any melanoma, and obtain an additional one or more images.
The obtaining may be repeated one or more times to generate the dataset comprising a plurality of first sets of images of the first part of the tissue of the subject and a plurality of second sets of images of the second part of the second tissue of the subject. The obtaining may be repeated at least about 1, 5, 10, 15, 20, 25, 50, 75, 100, 150, 200, 250, 500, 750, 1,000, or more times. The obtaining may be repeated at most about 1,000, 750, 500, 250, 200, 150, 100, 75, 50, 25, 20, 15, 10, 5, 1, or fewer times. The first set of images and the second set of images may be images of one or more tissues as described elsewhere herein. The method may comprise training a machine learning algorithm using at least a part of the plurality of signals. The training may be training as described elsewhere herein. The training may be performed on a remote computer system (e.g., a cloud server). The training may generate a trained machine learning algorithm. The trained machine learning algorithm may be implemented on a computer operatively coupled to the handheld imaging probe. The data derived from the second set of signals may be used as a control. The tissue of the subject may not be removed from the subject. For example, the tissue can be in the subject's let during the obtaining. The tissue of the subject may not be fixed to a slide. Not fixing the tissue to a slide may enable in vivo imaging, which can be faster and less invasive than methods that fix tissue to slides. The first part and the second part may be adjacent parts of the tissue. For example, the first part can be a mole and the second part can be the skin surrounding the mole. The first image or the second image may comprise a depth profile of the tissue as described elsewhere herein. The first image or the second image may be collected from a depth profile of the tissue. For example, the first image can be an image derived from signals in the depth profile. The first image and/or the second image may be collected in substantially real-time. The first image and/or the second image may be collected in real-time. The first image may be obtained within at least about 1 second (s), 5 s, 10 s, 30 s, 1 minute (m), 5 m, 10 m, 15 m, 30 m, 1 hour (h), 2 h, 3 h, 4 h, 5 h, 6 h, 7 h, 8 h, 9 h, 10 h, 12 h, 18 h, 24 h, 48 h, 72 h, 96 h, 120 h, 144 h, 168 h, or more of obtaining the second image. The first image may be obtained within at most about 168 h, 144 h, 120 h, 96 h, 72 h, 48 h, 24 h, 18 h, 12 h, 10 h, 9 h, 8 h, 7 h, 6 h, 5 h, 4 h, 3 h, 2 h, 1 h, 30 m, 15 m, 10 m, 5 m, 1 m, 30 s, 10 s, 5 s, 1 s, or less of obtaining the second image.
The present disclosure provides methods and systems for generating a trained machine learning algorithm to identify a tissue characteristic in a subject. In another aspect, the present disclosure provides a method for generating a trained machine learning algorithm to identify a tissue characteristic in a subject may comprise providing a data set comprising a plurality of tissue depth profiles. The plurality of tissue depth profiles may comprise (i) a first depth profile of a first tissue region positive for the tissue characteristic and (ii) a second depth profile of a second tissue region negative for the characteristic. The first depth profile and the second depth profile may be used to train a machine learning algorithm, thereby generating the trained machine learning algorithm. The method can include hard negative mining and/or hard positive mining with images from either the tissue region positive for the suspected tissue characteristic or the control tissue region negative for the suspected tissue characteristic that are incorrectly classified. Hard positive or negative mining can be either supervised or unsupervised. Unsupervised mining can be accomplished by identifying intermittent misclassifications straddled by a series of correct classifications from an image sequence within a tissue region. The method can utilize multiple instance learning where the images from the tissue with a tissue characteristic or suspected tissue characteristic and images from the control tissue are grouped into labeled “bags” each containing multiple images. Additional images from both the first and second regions can be collected to augment the data by providing a multitude of similar but individually unique images that can improve training of the model. Images from the region negative for the suspected characteristic can be used to build a feature vector to parameterize tissue images that lack a particular tissue characteristic. The feature vector can be used to identify tissue that differs from the non-characteristic tissue which may be indicative of the presence of one or more tissue characteristics of interest. Collecting images from multiple regions in multiple subjects that are not suspected of possessing a particular tissue characteristic may help train the machine learning algorithm to recognize non-characteristic tissue. In one example, the non-characteristic tissues can be control tissue regions or control regions that are suspected to be normal or absent of a particular characteristic.
The first depth profile and the second depth profile may be obtained from the same subject. For example, a depth profile of a skin region with a rash and a depth profile of a skin region without a rash can be obtained from a single subject. The first depth profile and the second depth profile may be obtained from different subjects. For example, a depth profile of a Basel cell carcinoma can be obtained from a first subject and a depth profile of healthy skin can be obtained from a second subject. The first tissue region and the second tissue region can be tissue regions of the same tissue. For example, the first tissue region and the second tissue region can both be tissue regions on the left arm of the subjects. The first tissue region and the second tissue region can be tissue regions of different tissues. For example, the first tissue region can be a tissue region on a leg while the second tissue region is a tissue region on a neck. In another example, the first tissue region can be in epithelium while the second tissue region is in stroma. The first depth profile and/or the second depth profile may be an in vivo depth profile. The in vivo depth profile may be a depth profile obtained of a tissue in a subject. The first depth profile and/or the second depth profile can be a layered depth profile. The layered depth profile may be a layered depth profile as described elsewhere herein.
The first depth profile and/or the second depth profile may be generated using one or more generated signals as described elsewhere herein. The method may further comprise outputting a trained machine learning algorithm. The trained machine learning algorithm may be output to be usable on a computer system of a user. For example, the trained machine learning algorithm can be a program on a computer. The trained machine learning algorithm may be hosted on a remote computing system (e.g., a cloud server). One or more additional depth profile may be used to further train the trained machine learning algorithm. For example, additional depth profile can be input into the machine learning algorithm to classify, and the results can be used to improve the machine learning algorithm. The one or more additional depth profiles may be used in a reinforcement learning scheme. Additional examples of machine learning algorithms and methods and systems for generating and training such machine learning algorithms are provided elsewhere herein. Such examples could be combined with the abovementioned method to generate additional machine learning algorithms and train them.
In another aspect, the present disclosure provides a system comprising one or more computer processors and computer memory coupled thereto, wherein the computer memory comprises machine executable code that, upon execution by the one or more computer processors, implements a method for identifying a tissue characteristic in a subject. The method may comprise accessing a database comprising a first set of data from a first image obtained from a first tissue region of the subject and a second set of data from a second image obtained from a second tissue region of the subject. The first tissue region may be suspected of having the tissue characteristic. The second tissue region may be free or suspected of being free from having the tissue characteristic. The first set of data and the second set of data may be computer processed to (i) identify a presence or absence of one or more features indicative of the tissue characteristic in the first image, and (ii) classify the subject as being positive or negative for the tissue characteristic based on the presence or absence of the one or more features in the first image. An electronic report which is indicative of the subject being positive or negative for the tissue characteristic may be generated.
The electronic report may comprise information related to a risk of the tissue characteristic. For example, the electronic report can have information about a prognosis of the subject based on the identified tissue characteristic. In another example, the electronic report can have information about the likelihood of the identified tissue characteristic being present in the tissue. The system may comprise an electronic device. The electronic device may have a screen. The electronic device may be a computer, tablet, cell phone, or the like. The electronic report may be output on a user interface of the electronic device. The electronic device may be used at least in part to collect the first image and/or the second image. For example, a handheld optical probe used to take the first and second images that is connected to a computer can have the electronic report displayed on a screen of the computer. The system may comprise an imaging probe. The imaging probe may be an imaging probe as described elsewhere herein. The imaging probe may be operatively coupled to the one or more computer processors. For example, the computer processors can be of a computer connected to the imaging probe. The imaging probe may be handheld. The imaging probe may be configured to deliver one or more therapies to the tissue. In another example, the imaging probe may comprise a surgical blade configured to excise a portion of the tissue.
The tissue characteristic may be a disease or abnormality. The disease or abnormality may be cancer. The tissue characteristic may comprise a beneficial tissue state. The first image and/or the second image may be obtained in vivo. The first image and/or the second image may be obtained without removal of the first tissue and/or the second tissue from the subject. The first image and/or the second image may extend below a surface of the tissue. The first tissue region and/or the second tissue region may not be fixed to a slide.
The first image and/or the second image may be generated using at least one non-linear imaging technique as described elsewhere herein. The image may be a depth profile as described elsewhere herein. The first image and/or the second image may be generated using at least one non-linear imaging technique and/or at least one linear imaging technique as described elsewhere herein. The first set of data and/or the second set of data may comprise groups of data. A group of data may comprise a plurality of images. The plurality of images may comprise (i) a positive image, and (ii) a negative image. The positive image may comprise one or more features. The negative image may not comprise the one or more features. The first set of data and/or the second set of data may comprise one or more sets of at least about 2 (e.g., pairs), 3, 4, 5, 6, 7, 8, 9, 10, or more instances of data. For example, the first data set can comprise a pair of instances of data with a first and second image. In another example, the second data set can have five sets each containing 4 images. The instances of data may be data as described elsewhere herein (e.g., images, signals, depth profiles). The plurality of images may comprise a positive image. The positive image may comprise the one or more features. The positive image may comprise the tissue characteristic. The plurality of images may comprise a negative image. The negative image may not comprise the one or more features. The negative image may not comprise the tissue characteristic. The first and/or second images may be real-time images. The first tissue region may be adjacent to the second tissue region. The first image may comprise a first sub-image of a third tissue region adjacent to the first tissue region. The second image may comprise a second sub-image of a fourth tissue region. The first image and/or the second image may comprise one or more depth profiles. The depth profiles may be images. The depth profiles may be depth profiles as described elsewhere herein. The one or more depth profiles may be one or more layered depth profiles. For example, a depth profile can comprise three layers each generated from a different signal. The one or more depth profiles may comprise one or more depth profiles generated from a scanning pattern that moves in one or more slanted directions as described elsewhere herein. The first image and/or the second image may comprise layered images. Each layer of the layered image may be of a different signal. For example, the layered image can comprise images generated from second harmonic generation signals, multi-photon fluorescence signals, and/or a reflectance confocal microscopy signal. The first image and/or the second image may comprise at least one layer generated using one or more generated signals (e.g., second harmonic generation signals, third harmonic generation signals, reflectance confocal microscopy signals, and multi-photon fluorescence signals, etc.). The first image or the second image may comprise one or more depth profiles generated from a scanning pattern that moves in one or more slanted directions as described elsewhere herein.
The computer processing may comprise calculating a first weighted sum of one or more features for the first image and /or a second weighted sum of one or more features for the second image. The subject may be classified as positive or negative for the tissue characteristic based on a difference between the first weighted sum and the second weighted sum. For example, a subject with images having a weighted sum less than that of the first image may be classified as free from the characteristic. The subject may be classified as being positive or negative for the tissue characteristic at an accuracy, sensitivity, and/or a specificity of at least about 40%, 50%, 60%, 70%, 80%, 90%, 91%, 92%, 93%, 94%, 95%, 96%, 97%, 98%, 99%, 99.9% or more. The subject may be classified as being positive or negative for the tissue characteristic at an accuracy, sensitivity, and/or a specificity of at most about 99.9%, 99%, 98%, 97%, 96%, 95%, 94%, 93%, 92%, 91%, 90%, 80%, 70%, 60%, 50%, 40%, or less. The subject may be classified as being positive or negative for the tissue characteristic at an accuracy, sensitivity, and/or a specificity of a range as defined by any two of the previous numbers. For example, the subject can be classified as having a skin cancer with an accuracy of about 90%-95% and a sensitivity of about 85%-90%. The computer processing may comprise applying a trained machine learning algorithm to the first set of data and/or the second set of data. The trained machine learning algorithm may be a trained machine learning algorithm as described elsewhere herein. The subject may be classified as being positive or negative for the tissue characteristic based on the presence or absence of the one or more features of the first image at an accuracy of at least about 40%, 50%, 60%, 70%, 80%, 90%, 91%, 92%, 93%, 94%, 95%, 96%, 97%, 98%, 99%, 99.9% or more. The subject may be classified as being positive or negative for the tissue characteristic based on the presence or absence of the one or more features of the first image at an accuracy of at most about 99.9%, 99%, 98%, 97%, 96%, 95%, 94%, 93%, 92%, 91%, 90%, 80%, 70%, 60%, 50%, 40%, or less. The first image and/or the second image may have a resolution of at least about 0.5, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 20, 30, 40, 50, 60, 70, 80, 90, 100, 250, 500, 1,000 or more micrometers. The first image and/or the second image may have a resolution of at most about 1,000, 500, 250, 100, 90, 80, 70, 60, 50, 40, 30, 20, 10, 9, 8, 7, 6, 5, 4, 3, 2, 1, 0.5, or fewer micrometers. The first image may extend below a first surface of the first tissue region. The second image may extend below a second surface of the second tissue region. For example, the first image can be of tissue below the epithelium of the subject. A third data set from a third image of a third tissue region having the one or more features indicative of the tissue characteristic may be computer processed. A fourth data set from a fourth image of a fourth tissue region lacking the one or more features indicative of the tissue characteristic may be computer processed. The third and/or fourth tissue region may be of a different subject than the subject. The third and/or fourth tissue region may be of the same subject. The addition of the third and/or fourth data sets may improve the quality of the computer processing by adding additional data points. The computer processing may comprise computer processing a third data set from a third image of a third tissue region having the one or more features indicative of the tissue characteristic; and (ii) computer processing a fourth data set from a fourth image of a fourth tissue region lacking the one or more features indicative of the tissue characteristic. The database may comprise one or more images from one or more additional subjects. The one or more subjects may be positive and/or negative for the tissue characteristic. For example, the database can comprise images from additional subjects that are free from the tissue characteristic as well as images from the same additional subjects that are positive for the tissue characteristic. In another example, the database can comprise images free from the tissue characteristic from subjects who are entirely free from the tissue characteristic.
In another aspect, the present disclosure provides a system comprising one or more computer processors and computer memory coupled thereto, wherein the computer memory comprises machine executable code that, upon execution by the one or more computer processors, implements a method for generating a trained machine learning algorithm to identify a tissue characteristic in a subject. The method may comprise receiving a data set comprising a plurality of tissue depth profiles. The plurality of tissue depth profiles may comprise (i) a first depth profile of a first tissue region positive for the tissue characteristic and (ii) a second depth profile of a second tissue region negative for the characteristic. The first depth profile and the second depth profile may be used to train a machine learning algorithm, thereby generating the trained machine learning algorithm. The trained machine learning algorithm may be output.
The system may comprise an imaging probe. The imaging probe may be operatively coupled to the one or more computer processors. For example, the imaging probe may be plugged into a computer comprising the processors. In another example, the imaging probe may be connected to the one or more computer processors via a network. The imaging probe may be handheld. The imaging probe may be configured to deliver therapy to the tissue as described elsewhere herein.
The first depth profile and/or the second depth profile may be obtained from the same subject. The first depth profile and/or the second depth profile may be obtained from different subjects. The first tissue region and the second tissue region may be tissue regions of the same tissue. For example, the first and second tissue regions may both be tissue regions on the skin of an arm of a subject. In another example, the first and second tissue regions may both be tissue regions in a leg of a subject. The first and/or second tissue regions may be tissue regions of different tissues. For example, the first tissue region can be on a subject's face while the second tissue region can be on a subject's foot. The first depth profile and/or the second depth profile may be in vivo depth profiles. The first depth profile and/or the second depth profile may be a layered depth profile as described elsewhere herein. The first depth profile and/or the second depth profile may be an image. The first depth profile and/or the second depth profile may be a depth profile of a generated signal as described elsewhere herein (e.g., second harmonic generation signals, third harmonic generation signals, reflectance confocal microscopy signals, multi-photon fluorescence signals). One or more additional depth profiles may be used to further train the trained machine learning algorithm. For example, the trained machine learning algorithm can be applied to a plurality of different depth profiles to improve the quality of the trained machine learning algorithm.
The signals may be substantially simultaneously (e.g., signals generated within a time period less than or equal to about 30 seconds (s), 20 s, 10 s, 1 s, 0.5 s, 0.4 s, 0.3 s, 0.2 s, 0.1 s, 0.01 s, 0.005 s, 0.001 s, or less; signals generated by the same pulse or beam of light, etc.) generated within a single region of the tissue (e.g., signals generated within less than or equal to about 1, 1E-1, 1E-2, 1E-3, 1E-4, 1E-5, 1E-6, 1E-7, 1E-8, 1E-9, 1E-10, 1E-11, 1E-12, 1E-13 or less cubic centimeters). The signals may be generated by the same pulse or beam of light. The signals may be generated by multiple beams of light synchronized in time and location as described elsewhere herein. Two or more of the signals may be combined to generate a composite image. The signals or subset of signals may be generated within a single region of the tissue using the same or similar scanning pattern or scanning plane. Each signal of a plurality of signals may be independent from the other signals of the plurality of signals. A user can decide which subset(s) of signals to use. For example, when both RCM and SHG signals are collected in a scan, a user can decide whether to use only the RCM signals. The substantially simultaneous generation of the signals may make the signals ideal signals for use with a trained algorithm. Additionally, video tracking of the housing or optical probe position as described previously herein can be recorded simultaneously with the generated signals.
The optical data may comprise structured data, time-series data, unstructured data, relational data, or any combination thereof. Unstructured data may comprise text, audio data, image data and/or video. Time-series data may comprise data from one or more of a smart meters, a smart appliance, a smart device, a monitoring system, a telemetry device, or a sensor. Relational data may comprise data from one or more of a customer system, an enterprise system, an operational system, a website, or web accessible application program interface (API). This may be done by a user through any method of inputting files or other data formats into software or systems.
The data can be stored in a database. A database can be stored in computer readable format. A computer processor may be configured to access the data stored in the computer readable memory. A computer system may be used to analyze the data to obtain a result. The result may be stored remotely or internally on storage medium and communicated to personnel such as medication professionals. The computer system may be operatively coupled with components for transmitting the result. Components for transmitting can include wired and wireless components. Examples of wired communication components can include a Universal Serial Bus (USB) connection, a coaxial cable connection, an Ethernet cable such as a Cat5 or Cat6 cable, a fiber optic cable, or a telephone line. Examples or wireless communication components can include a Wi-Fi receiver, a component for accessing a mobile data standard such as a 3G or 4G LTE data signal, or a Bluetooth receiver. All these data in the storage medium may be collected and archived to build a data warehouse.
The database may comprise an external database. The external database may be a medical database, for example, but not limited to, Adverse Drug Effects Database, AHFS Supplemental File, Allergen Picklist File, Average WAC Pricing File, Brand Probability File, Canadian Drug File v2, Comprehensive Price History, Controlled Substances File, Drug Allergy Cross-Reference File, Drug Application File, Drug Dosing & Administration Database, Drug Image Database v2.0/Drug Imprint Database v2.0, Drug Inactive Date File, Drug Indications Database, Drug Lab Conflict Database, Drug Therapy Monitoring System (DTMS) v2.2/DTMS Consumer Monographs, Duplicate Therapy Database, Federal Government Pricing File, Healthcare Common Procedure Coding System Codes (HCPCS) Database, ICD-10 Mapping Files, Immunization Cross-Reference File, Integrated A to Z Drug Facts Module, Integrated Patient Education, Master Parameters Database, Medi-Span Electronic Drug File (MED-File) v2, Medicaid Rebate File, Medicare Plans File, Medical Condition Picklist File, Medical Conditions Master Database, Medication Order Management Database (MOMD), Parameters to Monitor Database, Patient Safety Programs File, Payment Allowance Limit-Part B (PAL-B) v2.0, Precautions Database, RxNorm Cross-Reference File, Standard Drug Identifiers Database, Substitution Groups File, Supplemental Names File, Uniform System of Classification Cross-Reference File, or Warning Label Database.
The optical data may also be obtained through data sources other than the optical probe. The data sources may include sensors or smart devices, such as appliances, smart meters, wearables, monitoring systems, video or camera systems, data stores, customer systems, billing systems, financial systems, crowd source data, weather data, social networks, or any other sensor, enterprise system or data store. Example of smart meters or sensors may include meters or sensors located at a customer site, or meters or sensors located between customers and a generation or source location. By incorporating data from a broad array of sources, the system may be capable of performing complex and detailed analyses. The data sources may include sensors or databases for other medical platforms without limitation.
The optical probe may transmit an excitation light beam from a light source towards a surface of a reference tissue, which excitation light beam, upon contacting the tissue, generate the optical data of the tissue. The optical probe may comprise one or more focusing units to simultaneously adjust a depth and a position of a focal point of the excitation light beam along a scan path or scan pattern. The one or more focusing units in the optical probe may comprise, but are not limited to, movable lens, voice coil coupled to an optical element (e.g., an afocal lens), MEMS mirror, relay lenses, dichroic mirror, and fold mirror.
The scan path or scan pattern may comprise a path or pattern in at least one slant direction (“slanted path” or “slanted pattern”). The at least one slanted path or slanted pattern may be angled with respect to an optical axis. The angle between a slanted path or slanted pattern and the optical axis may be at most 45°. The angle between a slanted path or slanted pattern and the optical axis may be at least about 5°, 10°, 15°, 20°, 25°, 30°, 35°, 40°, 45°, 55°, 60°, 65°, 70°, 75°, 80°, 85°, or greater. In other cases, the angle between the slanted path or slanted pattern and the optical axis may be at most about 85°, 80°, 75°, 70°, 65°, 60°, 55°, 50°, 45°, 35°, 30°, 25°, 20°, 15°, 10°, 5°, or less.
The scan path or scan pattern may form a focal plane and/or may form or lie on at least one slanted plane. The at least one slanted plane may be positioned along a direction that is angled with respect to an optical axis. The angle between a slanted plane and the optical axis may be at most 45°. The angle between a slanted plane and the optical axis may be at least about 5°, 10°, 15°, 20°, 25°, 30°, 35°, 40°, 45°, 55°, 60°, 65°, 70°, 75°, 80°, 85°, or greater. In other cases, the angle between the slanted plane and the optical axis may be at most about 85°, 80°, 75°, 70°, 65°, 60°, 55°, 50°, 45°, 35°, 30°, 25°, 20°, 15°, 10°, 5°, or less.
The disease may be epithelial cancer.
The method may further comprise receiving medical data of the subject. The medical data of the subject may be obtained from a data receiver. The data receiver may be configured to either retrieve or receive data from one or more data sources, wherein retrieving data comprises a data extraction process and receiving data comprises receiving transmitted data from an electronic source of data.
Medical data or optical data of a subject may be paired with the subject through surgical a subject identity, so that a subject can retrieve his/her own information from a storage or a server through a subject identity. A subject identity may comprise patient's photo, name, address, social security number, birthday, telephone number, zip code, or any combination thereof. A patient identity may be encrypted and encoded in a visual graphical code. A visual graphical code may be a one-time barcode that can be uniquely associated with a patient identity. A barcode may be a UPC barcode, EAN barcode, Code 39 barcode, Code 128 barcode, ITF barcode, CodaBar barcode, GS1 DataBar barcode, MSI Plessey barcode, QR barcode, Datamatrix code, PDF417 code, or an Aztec barcode. A visual graphical code may be configured to be displayed on a display screen. A barcode may comprise QR that can be optically captured and read by a machine. A barcode may define an element such as a version, format, position, alignment, or timing of the barcode to enable reading and decoding of the barcode. A barcode can encode various types of information in any type of suitable format, such as binary or alphanumeric information. A QR code can have various symbol sizes as long as the QR code can be scanned from a reasonable distance by an imaging device. A QR code can be of any image file format (e.g., EPS or SVG vector graphs, PNG, TIF, GIF, or JPEG raster graphics format).
The process of generating datasets based on the optical data may comprise using one or more algorithms. The datasets may be selected optical data that represents one or more intrinsic properties of the tissue. The datasets can correspond to one or more depth profiles, images, layers of images or depth profiles indicating one or more intrinsic properties, characteristics, or structures of tissue. The datasets can include a plurality of depth profiles corresponding to different locations within the tissue of interest gathered by translating the optical probe while imaging. The datasets can include a plurality of depth profiles. At least one dataset can correspond to a control tissue at a first location and at least one dataset can correspond to positive (e.g., characteristic present) tissue at a second location. The one or more algorithms may be configured to select optical data, transfer optical data, and modify optical data. The one or more algorithms may comprise dimension reduction algorithms. Dimension reduction algorithms may comprise principal component regression and partial least squares. The principal component regression may be used to derive a low-dimensional set of features from a large set of variables. For instance, whether the tissue is at risk of cancer (a low-dimensional set of features) can be derived from all the intrinsic properties of the tissue (a large set of variables). The principal components used in the principal component regression may capture the most variance in the data using linear combinations of the data in subsequently orthogonal directions. The partial least squares may be a supervised alternative to principal component regression that makes use of the response variable in order to identify the new features.
The optical data may be uploaded to a cloud-based database, a database otherwise attached to a network, and the like. The datasets may be uploaded to a cloud-based database. The cloud-based database may be accessible from local and/or remote computer systems on which the machine learning-based sensor signal processing algorithms are running. The cloud-based database and associated software may be used for archiving electronic data, sharing electronic data, and analyzing electronic data. The optical data or datasets generated locally may be uploaded to a cloud-based database, from which it may be accessed and used to train other machine learning-based detection systems at the same site or a different site. Sensor device and system test results generated locally may be uploaded to a cloud-based database and used to update the training data set in real time for continuous improvement of sensor device and detection system test performance.
The trained algorithm may comprise one or more neural networks. A neural network may be a type of computational system that can learn the relationships between an input data set and a target data set. A neural network may be a software representation of a human neural system (e.g., cognitive system), intended to capture “learning” and “generalization” abilities as used by a human. A neural network may comprise a series of layers termed “neurons” or “nodes.” A neural network may comprise an input layer, to which data is presented; one or more internal, and/or “hidden,” layers; and an output layer. The input layer can include multiple depth profiles using signals that are synchronized in time and location. Such depth profiles, for example, can be generated using the optical probe as described elsewhere herein. Such depth profiles can comprise individual components, images, or depth profiles created from a plurality of subsets of gathered and processed signals. The depth profile may comprise a plurality of layers created from a plurality of subsets of images collected from the same location and time. Each of the plurality of layers may comprise data that identifies different anatomical structures and/or characteristics than those of the other layer(s). Such depth profile may comprise a plurality of sub-set depth profiles.
A neuron may be connected to neurons in other layers via connections that have weights, which are parameters that control the strength of a connection. The number of neurons in each layer may be related to the complexity of a problem to be solved. The minimum number of neurons required in a layer may be determined by the problem complexity, and the maximum number may be limited by the ability of a neural network to generalize. Input neurons may receive data being presented and then transmit that data to the first hidden layer through connections' weights, which are modified during training. The node may sum up the products of all pairs of inputs and their associated weights. The weighted sum may be offset with a bias. The output of a node or neuron may be gated using a threshold or activation function. An activation function may be a linear or non-linear function. An activation function may be, for example, a rectified linear unit (ReLU) activation function, a Leaky ReLU activation function, or other function such as a saturating hyperbolic tangent, identity, binary step, logistic, arcTan, softsign, parametric rectified linear unit, exponential linear unit, softPlus, bent identity, softExponential, Sinusoid, Sinc, Gaussian, or sigmoid function, or any combination thereof.
A first hidden layer may process data and transmit its result to the next layer through a second set of weighted connections. Each subsequent layer may “pool” results from previous layers into more complex relationships. Neural networks may be programmed by training them with a sample set (data collected from one or more sensors) and allowing them to modify themselves during (and after) training so as to provide an output such as an output value. A trained algorithm may comprise convolutional neural networks, recurrent neural networks, dilated convolutional neural networks, fully connected neural networks, deep generative models, generative adversarial networks, deep convolutional inverse graphics networks, encoder-decoder convolutional neural networks, residual neural networks, echo state network, a long/short term memory network, gated recurrent units, and Boltzmann machines. A trained algorithm may combine elements of the neural networks or Boltzmann machines in full or in part.
Weighting factors, bias values, and threshold values, or other computational parameters of a neural network, may be “taught” or “learned” in a training phase using one or more sets of training data. For example, parameters may be trained using input data from a training data set and a gradient descent or backward propagation method so that output value(s) that a neural network computes are consistent with examples included in training data set.
The number of nodes used in an input layer of a neural network may be at least about 10, 50, 100, 200, 300, 400, 500, 600, 700, 800, 900, 1000, 2000, 3000, 4000, 5000, 6000, 7000, 8000, 9000, 10,000, 20,000, 30,000, 40,000, 50,000, 60,000, 70,000, 80,000, 90,000, 100,000 or greater. In other instances, the number of node used in an input layer may be at most about 100,000, 90,000, 80,000, 70,000, 60,000, 50,000, 40,000, 30,000, 20,000, 10,000, 9000, 8000, 7000, 6000, 5000, 4000, 3000, 2000, 1000, 900, 800, 700, 600, 500, 400, 300, 200, 100, 50, or 10 or smaller. In some instance, the total number of layers used in a neural network (including input and output layers) may be at least about 3, 4, 5, 10, 15, 20, or greater. In other instances, the total number of layers may be at most about 20, 15, 10, 5, 4, 3 or less.
In some instances, the total number of learnable or trainable parameters, e.g., weighting factors, biases, or threshold values, used in a neural network may be at least about 10, 50, 100, 200, 300, 400, 500, 600, 700, 800, 900, 1000, 2000, 3000, 4000, 5000, 6000, 7000, 8000, 9000, 10,000, 20,000, 30,000, 40,000, 50,000, 60,000, 70,000, 80,000, 90,000, 100,000 or greater. In other instances, the number of learnable parameters may be at most about 100,000, 90,000, 80,000, 70,000, 60,000, 50,000, 40,000, 30,000, 20,000, 10,000, 9000, 8000, 7000, 6000, 5000, 4000, 3000, 2000, 1000, 900, 800, 700, 600, 500, 400, 300, 200, 100, 50, or 10 or smaller.
A neural network may comprise a convolutional neural network. A convolutional neural network may comprise one or more convolutional layers, dilated layers, or fully connected layers. The number of convolutional layers may be between 1-10 and dilated layers between 0-10. The total number of convolutional layers (including input and output layers) may be at least about 1,2, 3, 4, 5, 10, 15, 20, or greater, and the total number of dilated layers may be at least about 1,2, 3, 4, 5, 10, 15, 20, or greater. The total number of convolutional layers may be at most about 20, 15, 10, 5, 4, 3 or less, and the total number of dilated layers may be at most about 20, 15, 10, 5, 4, 3 or less. In some embodiments, the number of convolutional layers is between 1-10 and fully connected layers between 0-10. The total number of convolutional layers (including input and output layers) may be at least about 1,2, 3, 4, 5, 10, 15, 20, or greater, and the total number of fully connected layers may be at least about 1,2, 3, 4, 5, 10, 15, 20, or greater. The total number of convolutional layers may be at most about 20, 15, 10, 5, 4, 3 or less, and the total number of fully connected layers may be at most about 20, 15, 10, 5, 4, 3 or less.
A convolutional neural network (CNN) may be deep and feed-forward artificial neural networks. A CNN may be applicable to analyzing visual imagery. A CNN may comprise an input, an output layer, and multiple hidden layers. Hidden layers of a CNN may comprise convolutional layers, pooling layers, fully connected layers, and normalization layers. Layers may be organized in 3 dimensions: width, height, and depth.
Convolutional layers may apply a convolution operation to an input and pass results of a convolution operation to a next layer. For processing images, a convolution operation may reduce the number of free parameters, allowing a network to be deeper with fewer parameters. In a convolutional layer, neurons may receive input from a restricted subarea of a previous layer. Convolutional layer's parameters may comprise a set of learnable filters (or kernels). Learnable filters may have a small receptive field and extend through the full depth of an input volume. During a forward pass, each filter may be convolved across the width and height of an input volume, compute a dot product between entries of a filter and an input, and produce a 2-dimensional activation map of that filter. As a result, a network may learn filters that activate when it detects some specific type of feature at some spatial position in an input.
Pooling layers may comprise global pooling layers. Global pooling layers may combine outputs of neuron clusters at one layer into a single neuron in the next layer. For example, max pooling layers may use the maximum value from each of a cluster of neurons at a prior layer; and average pooling layers may use an average value from each of a cluster of neurons at the prior layer. Fully connected layers may connect every neuron in one layer to every neuron in another layer. In a fully-connected layer, each neuron may receive input from every element of a previous layer. A normalization layer may be a batch normalization layer. A batch normalization layer may improve a performance and stability of neural networks. A batch normalization layer may provide any layer in a neural network with inputs that are zero mean/unit variance. Advantages of using batch normalization layer may include faster trained networks, higher learning rates, easier to initialize weights, more activation functions viable, and simpler process of creating deep networks.
A neural network may comprise a recurrent neural network. A recurrent neural network may be configured to receive sequential data as an input, such as consecutive data inputs, and a recurrent neural network software module may update an internal state at every time step. A recurrent neural network can use internal state (memory) to process sequences of inputs. A recurrent neural network may be applicable to tasks such as handwriting recognition or speech recognition, next word prediction, music composition, image captioning, time series anomaly detection, machine translation, scene labeling, and stock market prediction. A recurrent neural network may comprise fully recurrent neural network, independently recurrent neural network, Elman networks, Jordan networks, Echo state, neural history compressor, long short-term memory, gated recurrent unit, multiple timescales model, neural Turing machines, differentiable neural computer, neural network pushdown automata, or any combination thereof.
A trained algorithm may comprise a supervised, partially supervised, or unsupervised learning method such as, for example, SVM, random forests, clustering algorithm (or software module), gradient boosting, logistic regression, generative adversarial networks, recurrent neural networks, and/or decision trees. It is possible according to some representative embodiments herein, to use a combination of supervised, partially supervised, or unsupervised learning methods to classify images. Supervised learning algorithms may be algorithms that rely on the use of a set of labeled, paired training data examples to infer the relationship between an input data and output data. An example of a labeled data set for supervised learning can be annotated depth profiles generated as described elsewhere herein. The annotated depth profiles can include user indicated regions of pixels within the depth profiles displaying known anatomical features. The known anatomical features can be of diseased or non-diseased tissues or elements of tissues. A partially supervised data set may include a plurality of depth profiles generated by translating the optical probe as described elsewhere herein. The plurality of profiles may be labeled as belonging to a tissue of subjects that have been previously or subsequently identified as having a disease or feature or not having a disease or feature without annotating regions of pixels within the individual profiles. Unsupervised learning algorithms may be algorithms used to draw inferences from training data sets to output data. Unsupervised learning algorithm may comprise cluster analysis, which may be used for exploratory data analysis to find hidden patterns or groupings in process data. One example of unsupervised learning method may comprise principal component analysis. Principal component analysis may comprise reducing the dimensionality of one or more variables. The dimensionality of a given variables may be at least 1, 5, 10, 50, 100, 200, 300, 400, 500, 600, 700, 800, 900, 1000, 1100, 1200 1300, 1400, 1500, 1600, 1700, 1800, or greater. The dimensionality of a given variables may be at most 1800, 1600, 1500, 1400, 1300, 1200, 1100, 1000, 900, 800, 700, 600, 500, 400, 300, 200, 100, 50, 10 or less.
A trained algorithm may be obtained through statistical techniques. In some embodiments, statistical techniques may comprise linear regression, classification, resampling methods, subset selection, shrinkage, dimension reduction, nonlinear models, tree-based methods, support vector machines, unsupervised learning, or any combination thereof.
A linear regression may be a method to predict a target variable by fitting the best linear relationship between a dependent and independent variable. The best fit may mean that the sum of all distances between a shape and actual observations at each point is the least. Linear regression may comprise simple linear regression and multiple linear regression. A simple linear regression may use a single independent variable to predict a dependent variable. A multiple linear regression may use more than one independent variable to predict a dependent variable by fitting a best linear relationship.
A classification may be a data mining technique that assigns categories to a collection of data in order to achieve accurate predictions and analysis. Classification techniques may comprise logistic regression and discriminant analysis. Logistic Regression may be used when a dependent variable is dichotomous (binary). Logistic regression may be used to discover and describe a relationship between one dependent binary variable and one or more nominal, ordinal, interval, or ratio-level independent variables. A resampling may be a method comprising drawing repeated samples from original data samples. A resampling may not involve a utilization of a generic distribution tables in order to compute approximate probability values. A resampling may generate a unique sampling distribution on a basis of an actual data. In some embodiments, a resampling may use experimental methods, rather than analytical methods, to generate a unique sampling distribution. Resampling techniques may comprise bootstrapping and cross-validation. Bootstrapping may be performed by sampling with replacement from original data, and take “not chosen” data points as test cases. Cross validation may be performed by split training data into a plurality of parts.
A subset selection may identify a subset of predictors related to a response. A subset selection may comprise best-subset selection, forward stepwise selection, backward stepwise selection, hybrid method, or any combination thereof. In some embodiments, shrinkage fits a model involving all predictors, but estimated coefficients are shrunken towards zero relative to the least squares estimates. This shrinkage may reduce variance. A shrinkage may comprise ridge regression and a lasso. A dimension reduction may reduce a problem of estimating n+1 coefficients to a simple problem of m+1 coefficients, where n<m. It may be attained by computing n different linear combinations, or projections, of variables. Then these n projections are used as predictors to fit a linear regression model by least squares. Dimension reduction may comprise principal component regression and partial least squares. A principal component regression may be used to derive a low-dimensional set of features from a large set of variables. A principal component used in a principal component regression may capture the most variance in data using linear combinations of data in subsequently orthogonal directions. The partial least squares may be a supervised alternative to principal component regression because partial least squares may make use of a response variable in order to identify new features.
A nonlinear regression may be a form of regression analysis in which observational data are modeled by a function which is a nonlinear combination of model parameters and depends on one or more independent variables. A nonlinear regression may comprise step function, piecewise function, spline, generalized additive model, or any combination thereof.
Tree-based methods may be used for both regression and classification problems. Regression and classification problems may involve stratifying or segmenting the predictor space into a number of simple regions. Tree-based methods may comprise bagging, boosting, random forest, or any combination thereof. Bagging may decrease a variance of prediction by generating additional data for training from original dataset using combinations with repetitions to produce multistep of the same carnality/size as original data. Boosting may calculate an output using several different models and then average a result using a weighted average approach. A random forest algorithm may draw random bootstrap samples of a training set. Support vector machines may be classification techniques. Support vector machines may comprise finding a hyperplane that best separates two classes of points with the maximum margin. Support vector machines may be constrained optimization problem where a margin is maximized subject to a constraint that it perfectly classifies data.
Unsupervised methods may be methods to draw inferences from datasets comprising input data without labeled responses. Unsupervised methods may comprise clustering, principal component analysis, k-Mean clustering, hierarchical clustering, or any combination thereof.
The method may train a machine learning algorithm to yield a trained algorithm in computer memory for identifying the disease in the tissue of the subject at an accuracy of at least 90%, wherein the tissue is independent of the training tissues. The method may train a machine learning algorithm to yield a trained algorithm in computer memory for identifying the disease in the tissue of the subject at an accuracy of at least 50%, 60%, 70%, 80%, 90% or greater. In some cases, the method may train a machine learning algorithm to yield a trained algorithm in computer memory for identifying the disease in the tissue of the subject at an accuracy of at most 90%, 80%, 70%, 60%, 50% or greater.
A method may train using a plurality of virtual cross-sections. The virtual cross sections may comprise a plurality of layers, images and/or depth profiles that were obtained using an excitation light beam directed at tissue at a synchronized time and location. A virtual cross-section may comprise depth profiles from an in vivo sample. Examples of a virtual cross section that can be used is illustrated as an image derived from one or more synchronized depth profiles in
Disclosed herein are systems for generating a trained algorithm for identifying a disease, condition, or other characteristic in a tissue of a subject. A system for generating a trained algorithm for identifying a disease, condition, or other characteristic in a tissue of a subject may comprise a database comprising data corresponding to depth profiles, related images, and or layers thereof, of training tissues of subjects that have been previously identified as having the disease condition, or other characteristic, which depth profiles related images, and or layers thereof, are generated signals and data synchronized or correlated in time and location; which depth profiles, related images, and or layers thereof are generated from signals generated from an excitation light beam; and/or which depth profiles, related images, and or layers thereof are generated from signals selected from the group consisting of second harmonic generation signal, third harmonic generation signal, reflectance confocal microscopy signal, autofluorescence signal and other generated signals described herein; and one or more computer processors operatively coupled to the database, wherein the one or more computer processors are individually or collectively programmed to (i) retrieve the data from the database and (ii) use the data to train a machine learning algorithm to yield a trained algorithm in computer memory for identifying the disease, condition or other characteristic in the tissue of the subject, wherein the tissue is independent of the training tissues. The database can additionally comprise similar data that corresponds to depth profiles, related images, and or layers thereof, of, training tissues of a subject that have been previously identified as not having the disease condition, or other characteristic. The datasets can include a plurality of depth profiles wherein at least one dataset corresponds to a control tissue at a first location and at least one dataset corresponds to positive (characteristic present) tissue at a second location. The datasets that have been previously or subsequently identified as having the characteristic and not having the characteristic can be used to train an algorithm. The algorithm can then be used to classify tissue. The database can comprise a plurality of pairs or sets of data with present and absent characteristics where each pair or group is from a single subject and has at least one positive and one control data set. The data forming the plurality of pairs or groups can comprise data collected from a plurality of subjects or a single subject. The single subject may or may not be a subject to be treated. The database comprising positive and the control tissue can comprise data collected from the same body part of the subject and /or adjacent normal and abnormal tissue.
The optical data may be described elsewhere herein. The optical data may comprise second harmonic generation signal, third harmonic generation signal, reflectance confocal microscopy signal, and autofluorescence signal and/or other generated signals as defined herein. The apparatus may be connected to a database. The optical data may be stored in the database. The database may be a centralized database. The database may be connected with the one or more processors. The one or more processors may analyze the data stored in the database through one or more algorithms. The analysis performed by the one or more processors may include, but not limited to, selecting optical data, creating datasets based on optical data, obtaining the patient health status from one or more databases, and yield a training algorithm based on data obtained. The one or more processors may provide one or more instructions based on the analysis.
The one or more instructions may be displayed on a display screen. The display screen may be a detachable display screen. The display screen may have a zoom function. The display screen may comprise editable feature that allows for marking of the epithelial features on the display screen. The display screen may be split and comprises the macroscopic image and the polychromatic image created from the depth profile. The display screen may be a liquid crystal display, similar to a tablet computer. The display screen may be accompanied by one or more speakers, and may be configured for providing visual and audial instructions to a user. The one or more instructions may comprise showing whether the subject has the rick of certain types of cancer, requesting the subject to take a given medication or go through a given treatment based on whether the subject has the risk of cancer. The one or more instructions may also comprise requesting the subject to provide his/her health status.
The depth profile can comprise a monochromatic image displaying colors derived from a single base hue. Alternatively or additionally, the depth profile can comprise a polychromatic image displaying more than one color. In a polychromatic image, color components may correspond to multiple depth profiles using signals or subsets of signals that are synchronized in time and location. Such depth profiles, for example, may be generated using the optical probe as described elsewhere herein. Such depth profiles can comprise individual components, images or depth profiles created from a plurality of subsets of gathered and processed generated signals. The depth profile may comprise a plurality of layers created from a plurality of subsets of images collected from the same location and time. Each of the plurality of layers may comprise data that identifies different anatomical structures and/or characteristics than those of the other layer(s). Such depth profiles may comprise a plurality of sub-set depth profiles. In this manner multiple colors can be used to highlight different elements of the tissue such as cells, nuclei, cytoplasm, connective tissues, vasculature, pigment, and tissue layer boundaries. The contrast can be adjusted in real-time to provide and/or enhance structure specific contrast. The contrast can be adjusted by a user (e.g. surgeon, physician, nurse, or other healthcare practitioner) or a programmed computer processor may automatically optimize the contrast in real-time. In a polychromatic image, each color may be used to represent a specific subset of the signals collected, such as second harmonic generation signals, third harmonic generation signals, signals resulting from polarized light, and autofluorescence signals. The colors of a polychromatic depth profile can be customized to reflect the image patterns a surgeon and/or pathologist may see when using standard histopathology. A pathologist may more easily interpret the results of a depth profile when the depth profile is displayed similar to how a traditional histological sample, for example a sample stained with hematoxylin and eosin, may be seen.
The optical probe may transmit an excitation light beam from a light source towards a surface of a reference tissue, which excitation light beam, upon contacting the tissue, generate the optical data of the tissue. The optical probe may comprise one or more focusing units to simultaneously adjust a depth and a position of a focal point of the excitation light beam along a scanning path or scanning pattern or at a different depth and position.
The scan path or scan pattern may comprise a path or pattern in at least one slant direction (“slanted path” or “slanted pattern”). The at least one slanted path or slanted pattern may be angled with respect to an optical axis. The angle between a slanted path or slanted pattern and the optical axis may be at most 45°. The angle between a slanted path or slanted pattern and the optical axis may be at least about 5°, 10°, 15°, 20°, 25°, 30°, 35°, 40°, 45°, 55°, 60°, 65°, 70°, 75°, 80°, 85°, or greater. In other cases, the angle between the slanted path or slanted pattern and the optical axis may be at most about 85°, 80°, 75°, 70°, 65°, 60°, 55°, 50°, 45°, 35°, 30°, 25°, 20°, 15°, 10°, 5°, or less.
The scan path or scan pattern may form a focal plane and/or lie on at least one slanted plane. The at least one slanted plane may be positioned along a direction that is angled with respect to an optical axis. The angle between a slanted plane and the optical axis may be at most 45°. The angle between a slanted plane and the optical axis may be at least about 5°, 10°, 15°, 20°, 25°, 30°, 35°, 40°, 45°, 55°, 60°, 65°, 70°, 75°, 80°, 85°, or greater. In other cases, the angle between the slanted plane and the optical axis may be at most about 85°, 80°, 75°, 70°, 65°, 60°, 55°, 50°, 45°, 35°, 30°, 25°, 20°, 15°, 10°, 5°, or less.
The identifying the disease may be at an accuracy of at least about 50%, 60%, 70%, 80%, 90%, 95%, 99%, 99.9%, or more. The identifying the disease may be at an accuracy of at most about 99.9%, 99%, 95%, 90%, 80%, 70%, 60%, 50%, or less.
The disease may be epithelial cancer.
The optical data may further comprise structured data, time-series data, unstructured data, and relational data. The unstructured data may comprise text, audio data, image data and/or video. The relational data may comprise data from one or more of a customer system, an enterprise system, an operational system, a website, or web accessible application program interface (API). This may be done by a user through any method of inputting files or other data formats into software or systems.
The optical data may be uploaded to, for example, a cloud-based database or other remote or networked database. The datasets may be uploaded to, for example, a cloud-based database or other remote or networked database. The cloud-based database may be accessible from local and/or remote computer systems on which the machine learning-based sensor signal processing algorithms are running. The cloud-based database and associated software may be used for archiving electronic data, sharing electronic data, and analyzing electronic data. The optical data or datasets generated locally may be uploaded to a cloud-based database, from which it may be accessed and used to train other machine learning-based detection systems at the same site or a different site. Sensor device and system test results generated locally may be uploaded to a cloud-based database and used to update the training data set in real time for continuous improvement of sensor device and detection system test performance.
The data may be stored in a database. A database can be stored in computer readable format. A computer processor may be configured to access the data stored in the computer readable memory. A computer system may be used to analyze the data to obtain a result. The result may be stored remotely or internally on storage medium, and communicated to personnel such as medication professionals. The computer system may be operatively coupled with components for transmitting the result. Components for transmitting can include wired and wireless components. Examples of wired communication components can include a Universal Serial Bus (USB) connection, a coaxial cable connection, an Ethernet cable such as a Cat5 or Cat6 cable, a fiber optic cable, or a telephone line. Examples or wireless communication components can include a Wi-Fi receiver, a component for accessing a mobile data standard such as a 3G or 4G LTE data signal, or a Bluetooth receiver. In some embodiments, all these data in the storage medium is collected and archived to build a data warehouse.
The training of a machine learning algorithm may yield a trained algorithm in computer memory for identifying the disease, condition, or other characteristic in the tissue of the subject wherein the tissue is independent of the training tissues. The training of a machine learning algorithm may yield a trained algorithm in computer memory for identifying the disease in the tissue of the subject at an accuracy of at least 50%, 60%, 70%, 80%, 90% or greater. In some cases, the training of a machine learning algorithm may yield a trained algorithm in computer memory for identifying the disease in the tissue of the subject at an accuracy of at most 90%, 80%, 70%, 60%, 50% or greater.
Disclosed herein are methods for analyzing tissue of a body of a subject. In an aspect, a method for analyzing tissue of a body of a subject may comprise (a) directing light to the tissue of the body of the subject; (b)receiving a plurality of signals from the tissue of the body of the subject in response to the light directed thereto in (a), wherein at least a subset of the plurality of signals are from within the tissue; (c) inputting data corresponding to the plurality of signals to a trained machine learning algorithm that processes the data to generate a classification of the tissue of the body of the subject; and outputting the classification on a user interface of an electronic device of a user.
The classification may identify the subject as having a disease, condition, or other characteristic. The disease may be a disease as described elsewhere herein. The disease may be a cancer. The tissue of the subject may be a skin of the subject, and the cancer may be skin cancer. The cancer may be benign or malignant. The classification may identify the tissue as having the disease at an accuracy of at least about 50%, 60%, 70%, 80%, 90%, 95%, 98%, 99%, 99.9%, or more.
The plurality of signals may comprise a second harmonic generation (SHG) signal, a multi photon fluorescence signal, a reflectance confocal microscopy (RCM) signal, any other generated signals described herein, or any combination thereof. The multi photon fluorescence signal may be a plurality of multi photon fluorescence signals. The plurality of multi photon fluorescence signals may be at a plurality of wavelengths. The plurality of multi photon fluorescence signals may be generated by a plurality of components of the tissue. The method may comprise identifying one or more features corresponding to the plurality of signals using the trained machine learning algorithm. A plurality of signals may be filtered such that fewer signals than are recorded are used. A plurality of generated signals may be used to generate a plurality of depth profiles.
The trained machine learning algorithm may comprise a neural network. The neural network may be a convolutional neural network. The data may be controlled for an illumination power of the optical signal. The control may be normalization. The data may be controlled for an illumination power by the trained machine learning algorithm. The data may be controlled for an illumination power before the trained machine learning algorithm is applied. The convolutional neural network may be configured to use colorized data as an input of the neural network.
The method may comprise receiving medical data of the subject. The medical data may be as described elsewhere herein. The medical data may be uploaded to a cloud or network attached device. The data may be kept on a local device.
The method may be configured to use data augmentation to improve the trained machine learning algorithm. For example, an augmented data set can be a data set where a fast image capture created a dataset with a number of similar, but not the same, images from a tissue.
The method may be configured to improve the trained machine learning algorithm by comparing control tissue (e.g., tissue not having a characteristic) with positive tissue (e.g., tissue having the characteristic). The control tissue and positive tissue data can be obtained from a single subject. The control tissue data and positive tissue data can be obtained from the same body part of a subject. The control tissue data and positive tissue data can be obtained from adjacent tissue of a subject. The control tissue data and positive tissue data can be obtained in vivo. The control tissue data and positive tissue data can be obtained in real time.
The method may be configured to use images obtained using a controlled power of illumination. The controlled power of illumination may improve the performance of the trained machine learning algorithm. For example, a controlled illumination can enable a trained machine learning algorithm to attribute differences between two images to differences in a tissue rather than differences in the conditions used to obtain the images, thus improving the accuracy of the trained machine learning algorithm.
The method may be configured to use data with minimal variations to improve the trained machine learning algorithm. For example, due to the low variation in image parameters generated by optical probes described herein the trained machine learning algorithm can more accurately determine if a lesion is cancerous, if tissue is normal or abnormal, or other features of the tissue in a subject pertaining to the health, function, treatment, or appearance of the tissues or of a subject as all images used by the trained machine learning algorithm use the same labeling and coloring scheme. The method may be configured to use data from the same subject that is characteristic positive tissue and control tissue that is characteristic negative to improve machine learning. The positive and control tissue data can both be obtained in a time period as described elsewhere herein. The tissue can also be obtained from the same body party or from adjacent tissue. The method may be configured to use data generated from an excitation light beam interacting with a tissue. The excitation light beam may generate a plurality of depth profiles for use in a trained machine learning algorithm. The excitation light beam may generate a plurality of depth profiles to train a machine learning algorithm. The excitation light beam may generate a depth profile from a subset of a plurality of return signals.
The trained machine learning algorithm may be trained to generate a spatial map of the tissue. The spatial map may be a three-dimensional model of the tissue. The spatial map may be annotated by a user and/or the trained machine learning algorithm.
Disclosed herein are systems for analyzing tissue of a body of a subject. In an aspect, a system for analyzing tissue of a body of a subject may comprise an optical probe that is configured to (i) direct light to the tissue of the body of the subject, and (ii) receive a plurality of signals from the tissue of the body of the subject in response to the light directed thereto in (i), wherein at least a subset of the plurality of signals are from within the tissue; and one or more computer processors operatively coupled to the optical probe, wherein the one or more computer processors are individually or collectively programmed to (i) receive data corresponding to the plurality of signals, (ii) input the data to a trained machine learning algorithm that processes the data to generate a classification of the tissue of the body of the subject, and (iii) output the classification on a user interface of an electronic device of a user.
The optical probe and the one or more computer processors may comprise a same device. The device may be a mobile device. The device may be a plurality of devices that may be operatively coupled to one another. For example, the system can be a handheld optical probe optically connected to a laser and detection box, and the box can also contain a computer.
The optical probe may be part of a device, and the one or more computer processors may be separate from the device. The one or more computer processors may be part of a computer server. The one or more processors may be part of a distributed computing infrastructure. For example, the system can be a handheld optical probe containing all of the optical components that is wirelessly connected to a remote server that processes the data from the optical probe.
The system may be configured to receive medical data of the subject. The medical data may be as described elsewhere herein. The medical data may be uploaded to a cloud or network attached device. The data may be kept on a local device. The machine learning algorithm may be applied remotely, through a cloud or other network, or may be applied on a local device.
The present disclosure provides computer systems that are programmed to implement methods of the disclosure.
The computer system 601 includes a central processing unit (CPU, also “processor” and “computer processor” herein) 605, which can be a single core or multi core processor, or a plurality of processors for parallel processing. The computer system 601 also includes memory or memory location 610 (e.g., random-access memory, read-only memory, flash memory), electronic storage unit 615 (e.g., hard disk), communication interface 620 (e.g., network adapter) for communicating with one or more other systems, and peripheral devices 625, such as cache, other memory, data storage and/or electronic display adapters. The memory 610, storage unit 615, interface 620 and peripheral devices 625 are in communication with the CPU 605 through a communication bus (solid lines), such as a motherboard. The storage unit 615 can be a data storage unit (or data repository) for storing data. The computer system 601 can be operatively coupled to a computer network (“network”) 630 with the aid of the communication interface 620. The network 630 can be the Internet, an internet and/or extranet, or an intranet and/or extranet that is in communication with the Internet. The network 630 in some cases is a telecommunication and/or data network. The network 630 can include one or more computer servers, which can enable distributed computing, such as cloud computing. The network 630, in some cases with the aid of the computer system 601, can implement a peer-to-peer network, which may enable devices coupled to the computer system 601 to behave as a client or a server.
The CPU 605 can execute a sequence of machine-readable instructions, which can be embodied in a program or software. The instructions may be stored in a memory location, such as the memory 610. The instructions can be directed to the CPU 605, which can subsequently program or otherwise configure the CPU 605 to implement methods of the present disclosure. Examples of operations performed by the CPU 605 can include fetch, decode, execute, and writeback.
The CPU 605 can be part of a circuit, such as an integrated circuit. One or more other components of the system 601 can be included in the circuit. In some cases, the circuit is an application specific integrated circuit (ASIC).
The storage unit 615 can store files, such as drivers, libraries, and saved programs. The storage unit 615 can store user data, e.g., user preferences and user programs. The computer system 601 in some cases can include one or more additional data storage units that are external to the computer system 601, such as located on a remote server that is in communication with the computer system 601 through an intranet or the Internet.
The computer system 601 can communicate with one or more remote computer systems through the network 630. For instance, the computer system 601 can communicate with a remote computer system of a user (e.g., phone). Examples of remote computer systems include personal computers (e.g., portable PC), slate or tablet PC's (e.g., Apple® iPad, Samsung® Galaxy Tab), telephones, Smart phones (e.g., Apple® iPhone, Android-enabled device, Blackberry®), or personal digital assistants. The user can access the computer system 601 via the network 630.
Methods as described herein can be implemented by way of machine (e.g., computer processor) executable code stored on an electronic storage location of the computer system 601, such as, for example, on the memory 610 or electronic storage unit 615. The machine executable or machine-readable code can be provided in the form of software. During use, the code can be executed by the processor 605. In some cases, the code can be retrieved from the storage unit 615 and stored on the memory 610 for ready access by the processor 605. In some situations, the electronic storage unit 615 can be precluded, and machine-executable instructions are stored on memory 610.
The code can be pre-compiled and configured for use with a machine having a processer adapted to execute the code, or can be compiled during runtime. The code can be supplied in a programming language that can be selected to enable the code to execute in a pre-compiled or as-compiled fashion.
Aspects of the systems and methods provided herein, such as the computer system 601, can be embodied in programming. Various aspects of the technology may be thought of as “products” or “articles of manufacture” typically in the form of machine (or processor) executable code and/or associated data that is carried on or embodied in a type of machine readable medium. Machine-executable code can be stored on an electronic storage unit, such as memory (e.g., read-only memory, random-access memory, flash memory) or a hard disk. “Storage” type media can include any or all of the tangible memory of the computers, processors or the like, or associated modules thereof, such as various semiconductor memories, tape drives, disk drives and the like, which may provide non-transitory storage at any time for the software programming. All or portions of the software may at times be communicated through the Internet or various other telecommunication networks. Such communications, for example, may enable loading of the software from one computer or processor into another, for example, from a management server or host computer into the computer platform of an application server. Thus, another type of media that may bear the software elements includes optical, electrical, and electromagnetic waves, such as used across physical interfaces between local devices, through wired and optical landline networks and over various air-links. The physical elements that carry such waves, such as wired or wireless links, optical links, or the like, also may be considered as media bearing the software. As used herein, unless restricted to non-transitory, tangible “storage” media, terms such as computer or machine “readable medium” refer to any medium that participates in providing instructions to a processor for execution.
Hence, a machine readable medium, such as computer-executable code, may take many forms, including but not limited to, a tangible storage medium, a carrier wave medium or physical transmission medium. Non-volatile storage media include, for example, optical or magnetic disks, such as any of the storage devices in any computer(s) or the like, such as may be used to implement the databases, etc. shown in the drawings. Volatile storage media include dynamic memory, such as main memory of such a computer platform. Tangible transmission media include coaxial cables; copper wire and fiber optics, including the wires that comprise a bus within a computer system. Carrier-wave transmission media may take the form of electric or electromagnetic signals, or acoustic or light waves such as those generated during radio frequency (RF) and infrared (IR) data communications. Common forms of computer-readable media therefore include for example: a floppy disk, a flexible disk, hard disk, magnetic tape, any other magnetic medium, a CD-ROM, DVD or DVD-ROM, any other optical medium, punch cards paper tape, any other physical storage medium with patterns of holes, a RAM, a ROM, a PROM and EPROM, a FLASH-EPROM, any other memory chip or cartridge, a carrier wave transporting data or instructions, cables or links transporting such a carrier wave, or any other medium from which a computer may read programming code and/or data. Many of these forms of computer readable media may be involved in carrying one or more sequences of one or more instructions to a processor for execution.
The computer system 601 can include or be in communication with an electronic display 635 that comprises a user interface (UI) 640 for providing, for example, results of the optical data analysis to the user. Examples of UI's include, without limitation, a graphical user interface (GUI) and web-based user interface.
Methods and systems of the present disclosure can be implemented by way of one or more algorithms. An algorithm can be implemented by way of software upon execution by the central processing unit 605. The algorithm can, for example, be used for selecting data, identifying features in the data, and/or classifying the data.
Computer processors or systems may comprise or be configured to train machine learning algorithm using collected or gathered data. Computer processors or systems may comprise or be configured to apply a machine learning algorithm to collected data to classify tissue.
Also provided herein a method for aligning a light beam (e.g., aligning a light beam between a beam splitter and an optical fiber). In some cases, the method of aligning a light beam can be used to align a beam of light between any two components. For example, a focused beam of light can be aligned between a lens and a pinhole using a refractive element. In another example, a beam of light can be aligned to a specific region of a sample using the methods and systems described herein.
In an aspect, a method of the present disclosure may comprise providing (i) a light beam in optical communication with a beam splitter. The beam splitter is in optical communication with a lens. The lens may be in optical communication with a refractive element, (ii) an optical fiber, and (iii) a detector in optical communication with the optical fiber. An optical path from the refractive element may be misaligned with respect to the optical fiber. In an aspect, the method may further comprise adjusting the refractive element to align the optical path with the optical fiber. In an aspect, the method may further comprise directing the light beam to the beam splitter that splits the light beam into a beamlet. The beamlet may be directed through the lens to the refractive element that directs the beamlet along the optical path to the optical fiber, such that the detector detects the beamlet.
The method of aligning a light beam using a refractive element may allow for significantly faster and easier alignment of a beam of light to a fiber optic. The method may allow for a single mode fiber optic to be aligned in less than about 60, 45, 30, 15, 5, or less minutes with high long-term stability. The method may allow for a small alignment adjustment to be performed by a large adjustment to the refractive element, which may give fine control of the alignment adjustment.
The beamlet may be directed to an additional element that reflects the beamlet to the beam splitter, which beam splitter directs the beamlet through the lens to the refractive element. The additional element may be a mirror. The mirror may be used in the alignment process by providing a strong signal to align with. The beamlet may be directed from the beam splitter through one or more additional elements prior to being reflected by the refractive element. The additional elements may be the elements of the optical probe described elsewhere herein. The additional elements may be a mirror scanner, a focus lens pair, a plurality of relay lenses, a dichroic mirror, an objective, a lens, or any combination thereof. The refractive element may be operatively coupled to a lens. The refractive element and a lens may be on the same or different mounts.
The point spread function of the beamlet after interacting with the refractive element may be sufficiently small to enable a resolution of the detector to be less than about 200, 150, 100, 75, 50, 40, 30, 25, 20, 15, 10, 9, 8, 7, 6, 5, 4, 3, 2, 1, 0.9, 0.8, 0.7, 0.6, 0.5, or less microns. For example, the refractive element may introduce astigmatism or defocus into the beamlet, but the astigmatism or defocus is sufficiently small as to not impact the overall resolution of the detector (e.g., the astigmatism or defocus can be less than the diffraction point spread function). The refractive element may be a flat window, a curved window, a window with surface patterning, or the like.
The adjusting the position may comprise applying a rotation of the refractive element. The adjusting the position may comprise a translation of the refractive element. The rotation may be at most about 180, 170, 160, 150, 125, 100, 90, 80, 70, 60, 50, 40, 30, 20, 10, 9, 8, 7, 6, 5, 4, 3, 2, 1 degree, or less. The rotation may be at most about 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 20, 30, 40, 50, 60, 70, 80, 90, 100, 125, 150, 179 degrees, or more. The rotation or translation or both may be in at most three, two, or one dimensions. An adjustment ratio of the refractive alignment can be defined as the degree of misalignment divided by the deflection of the refractive element that corrects the misalignment. For example, a beam of light that is 0.05 degrees out of alignment that is corrected by a rotation of 20 degrees of the refractive element can have an adjustment ratio of 0.05/20=0.0025 or 2.5E-3. The adjustment ratio may be at least about 1E-5, 5E-5, 1E-4, 5E-4, 1E-3, 5E-3, 1E-2, 5E-2, 1E-1, 1, 5, or more. The adjustment ratio may be at most about 5, 1, 5E-1, 1E-1, 5E-2, 1E-2, 5E-3, 1E-3, 5E-4, 1E-4, 5E-5, 1E-5, or less.
Also disclosed herein are systems for aligning a light beam. In an aspect, a system for aligning a light beam may comprise a light source that is configured to provide a light beam; a focusing lens in optical communication with the light beam; a movable refractive element in optical communication with the lens; an optical fiber; and a detector in optical communication with the optical fiber wherein the refractive element is positioned between the focusing lens and the optical fiber. The refractive alignment element may be adjustable to align the optical path with the optical fiber, such that, when the optical path is aligned with the optical fiber, the light beam may be directed through the lens to the refractive element that directs the beam along the optical path to the optical fiber, such that the detector detects the beam. The refractive alignment element may be rotationally or angularly movable with respect to the optical fiber and/or the optical fiber mount.
The light beam can be a beamlet split from a beam splitter prior to directing the beamlet to the alignment arrangement. The alignment arrangement can further comprise a movable mirror positioned between the beam splitter and the focusing lens (for example, as shown in
The focusing lens may focus the light of the beamlet to a diffraction limited or nearly diffraction limited spot. The refractive alignment element may be used to correct any additional fine misalignment of the beamlet to the fiber optic. The refractive alignment element can have a refractive index, thickness and/or range of motion (e.g., a movement which alters the geometry) that permits alignment of the beamlet exiting the lens to a fiber optic have a diameter less than about 20 microns, 10 microns, 5 microns, or less. According to some representative embodiments, the refractive alignment element properties (including refractive index, thickness, and range of motion) may be selected so that the aberrations introduced by the refractive alignment element do not increase the size the beamlet focused on the optical fiber by more than 0%, 1%, 2%, 5%, 10%, 20%, or more above the focusing lens's diffraction limit. The alignment arrangement can be contained within a handheld device.
The beamlet may comprise polarized light. The optical probe may comprise one or more polarization selective optics (e.g., polarization filters, polarization beam splitters, etc.). The one or more polarization selective optics may be selected for a particular polarization of the beamlet, such that the beamlet that is detected is of a particular polarization.
The system may comprise a controller operatively coupled to the refractive element. The controller may be programmed to direct adjustment of the refractive element to align the optical path with the optical fiber. The adjustment may also be performed with an input of a user or manually. The adjustment may be performed by an actuator operatively coupled to the refractive element. The actuator may be an actuator as described elsewhere herein. For example, a piezoelectric motor can be attached to a three-axis optical mount holding a flat plate of quartz, and the piezoelectric motor can be controlled by an alignment algorithm programmed to maximize signal of the detector. The adjustment may be performed by a user. For example, a user can adjust a micrometer that is attached to a three-axis optical mount holding a flat plate of glass, moving the stage until an acceptable level of signal is read out on the detector.
The refractive element may be a flat window, a curved window, a flat window with a patterned surface, a curved window with a patterned surface, a photonic structure, or the like. The refractive element may be made of glass, quartz, calcium fluoride, germanium, barium, fused silica, sapphire, silicon, zinc selenide, magnesium fluoride, and a plastic. The refractive element may have an index of refraction greater than 2.
The point spread function of the beam after interacting with the refractive element may be sufficiently small to enable a resolution of the detector to be less than about 200, 150, 100, 75, 50, 40, 30, 25, 20, 15, 10, 9, 8, 7, 6, 5, 4, 3, 2, 1, 0.9, 0.8, 0.7, 0.6, 0.5 microns, or less. The refractive element may be configured to adjust the beam at most about 45, 40, 35, 30, 25, 20, 15, 10, 5, 4, 3, 2, 1, 0.5, 0.1, 0.01 degrees, or less. The refractive element may be configured to adjust the beam at least about 0.01, 0.1, 0.5, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 15, 20, 25, 30, 35, 40, 45 degrees, or more. The refractive element may be adjusted to change the amount of adjustment. For example, the refractive element was set to a deflection of 60 degrees, but the system has fallen out of alignment. In this example, the refractive element can be adjusted to generate an adjustment of 15 degrees to bring the system back into alignment.
The refractive element may have a footprint of at most about 100, 90, 80, 70, 60, 50, 40, 30, 20, 10, 5, 4, 3, 2, 1, 0.5, 0.1 square inches, or less. The refractive element and an associated housing may have a footprint of at most about 100, 90, 80, 70, 60, 50, 40, 30, 20, 10, 5, 4, 3, 2, 1, 0.5, 0.1 square inches, or less. The refractive element may have a footprint of at least about 0.1, 0.5, 1, 2, 3, 4, 5, 10, 20, 30, 40, 50, 60, 70, 80, 90, 100 square inches, or more. The refractive element and an associated housing may have a footprint of at least about 0.1, 0.5, 1, 2, 3, 4, 5, 10, 20, 30, 40, 50, 60, 70, 80, 90, 100 square inches, or more.
While preferred embodiments of the present invention have been shown and described herein, it will be obvious to those skilled in the art that such embodiments are provided by way of example only. It is not intended that the invention be limited by the specific examples provided within the specification. While the invention has been described with reference to the aforementioned specification, the descriptions and illustrations of the embodiments herein are not meant to be construed in a limiting sense. Numerous variations, changes, and substitutions will now occur to those skilled in the art without departing from the invention. Furthermore, it shall be understood that all aspects of the invention are not limited to the specific depictions, configurations or relative proportions set forth herein which depend upon a variety of conditions and variables. It should be understood that various alternatives to the embodiments of the invention described herein may be employed in practicing the invention. It is therefore contemplated that the invention shall also cover any such alternatives, modifications, variations, or equivalents. It is intended that the following claims define the scope of the invention and that methods and structures within the scope of these claims and their equivalents be covered thereby.
This application claims the benefit of U.S. Provisional Patent Application No. 63/023,727, filed May 12, 2020, and is a continuation-in-part of International Application No. PCT/US2019/061306, filed Nov. 13, 2019, which claims the benefit of U.S. Provisional Patent Application No. 62/760,620, filed Nov. 13, 2018, each of which is entirely incorporated herein by reference.
The invention was made with U.S. Government support under Small Business innovation Research (SBIR) grant number 2R44CA221591-02A1 awarded by the Department of Health and Human Services, National Institutes of Health, National Cancer Institute. The U.S. Government has certain rights in the invention.
| Number | Date | Country | |
|---|---|---|---|
| 63023727 | May 2020 | US | |
| 62760620 | Nov 2018 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | PCT/US2019/061306 | Nov 2019 | US |
| Child | 17096602 | US |
