Patent Application
20240225451
All patents, patent applications and publications cited herein are hereby incorporated by reference in their entirety. The disclosures of these publications in their entireties are hereby incorporated by reference into this application in order to more fully describe the state of the art as known to those skilled therein as of the date of the invention described and claimed herein.
This patent disclosure contains material that is subject to copyright protection. The copyright owner has no objection to the facsimile reproduction by anyone of the patent document or the patent disclosure as it appears in the U.S. Patent and Trademark Office patent file or records, but otherwise reserves any and all copyright rights.
Aspects of the invention are drawn to a Raman spectroscopy system that incorporates a tissue probe into a thin sheath that is adapted to fit any flexible or rigid laryngoscope.
Laryngeal carcinoma is the second most common cancer of the respiratory tract with an overall 5-year survival rate of 60%. Clinical stage at the time of diagnosis and margin status during surgical resection are the two most important factors correlated with disease prognosis. Five-year survival drops from approximately 90% for tumors confined to the larynx, to 60% and 27% for regional and distant disease, respectively. Early diagnosis depends upon careful assessment of suspicious laryngeal lesions noted on diagnostic laryngoscopy. Confirmation requires biopsy and histologic evaluation. Laryngeal tissue sampling is complicated by the need for endoscopic instrument placement in the airway and the heterogeneous appearance of premalignant and malignant lesions. Margin detection during surgical resection of laryngeal carcinoma also requires histologic assessment that is highly dependent on surgeon selected sample sites and the tendency for submucosal tumor extension. Nationally, the incidence of positive margins following endoscopic resection of laryngeal carcinoma is 22%.
The present invention provides Raman spectroscopy system, components thereof, and methods of using the same.
In embodiments, the invention comprises an endoscopy sheath-probe device. The device can comprise a sheath, a Raman probe system, an endoscope, or a combination thereof. In embodiments, the sheath comprises a Raman probe channel and an endoscope channel.
The endoscope can comprise a rigid endoscope or a flexible endoscope. In embodiments, the flexible endoscope comprises a fiberoptic endoscope.
The Raman probe system can comprise a probe, a laser source, an excitation signal filter, a collection filter, a charge couple device detector, a signal collection system, a housing unit, a computer, a display, or a combination thereof. In certain embodiments, the signal collection system comprises a spectrum collection range of about 200 cm−1 to about 4000 cm−1. The laser source can comprise a wavelength of about 532 nm, about 638 nm, about 785 nm, or about 1064 nm. In embodiments, the computer is configured for signal analysis and display capabilities.
In certain embodiments, the probe comprises a flexible, fiberoptic probe. The probe can be configured to control an incidence laser angle. In one embodiment, probe comprises an offset distal tip, wherein the offset distal tip is configured to allow contact with a tissue site and control of an optimal incident laser distance. The probe can be configured to control ambient lighting. In embodiments, the excitation signal filter comprises a high-optical density (OD) band-pass filter. The collection filter can comprise a high-optical density long-pass filter. In embodiments, the filter is configured to filter out elastic scattering. In embodiments, the probe comprises an outer diameter of up to about 2.5 mm, a length of up to about 2 m, or a combination thereof.
The probe can be configured to permit laser light delivery and signal collection in real time during an endoscopic procedure. In certain embodiments, the endoscopic procedure comprises an upper airway endoscopic procedure. The upper airway endoscopic procedure can comprise a laryngoscopy.
In embodiments, the endoscope comprises an endoscopic light source and the Raman probe system housing unit further comprises a switch, wherein the switch is configured to permit a user to toggle between the endoscopic light source and the laser source. In certain embodiments, the endoscope comprises an arthroscope, a bronchoscope, a colonoscope, a hysteroscope, a laparoscope, a laryngoscope, a mediastinoscope, sigmoidoscope, thoracoscope, ureteroscope, or an endoscope for use in operative endoscopy. In one embodiment, the endoscopy comprises laryngoscopy. The operative endoscopy can comprise pancreatic laparoscopy.
In one embodiment, the sheath is disposable.
The device can be configured to permit collection of data in real time during an endoscopic procedure. In embodiments, the device is configured to produce a maximum energy exposure of a target tissue, wherein the maximum energy exposure is sufficient to permit collection of Raman data without damaging the tissue. In embodiments, the maximum energy exposure is less than about 5 joules/cm2.
In another aspect, the present invention is directed to a method of optical biopsy, IN embodiments, the method comprises the use of any one or more of the devices disclosed herein in a subject. The method can further comprise: imaging a target tissue with the endoscope; subjecting the target tissue to Raman spectroscopy using the Raman probe system to generate Raman spectral data; analyzing the Raman spectral data; and classifying the target tissue as cancer or non-cancer based up on a peak at any one or more wavenumbers from the Raman spectral data. The method can further comprise obtaining a dot product of each normalized Raman signal to generate a 2D Raman image. In embodiments, the target tissue comprises pancreatic tissue, brain tissue, lung tissue, oral tissue, or laryngeal tissue. The cancer can comprise pancreatic cancer, laryngeal cancer, or oral cancer. Non-cancer can comprise non-cancer pancreatic tissue or non-cancer laryngeal tissue.
Another aspect of the present invention includes a computer-aided method of diagnostic assessment of a tissue. In embodiments, the method comprises receiving Raman spectral input data of the tissue; subjecting the Raman spectral input data to at least one data analysis model, wherein the at least one data analysis model uses the Raman spectral input data to classify the tissue as cancer tissue or non-cancer tissue. The data analysis model can comprise principal component (PCA) analysis. The data analysis model can comprises random forest (RF) analysis. In one embodiment, the data analysis model comprises convolutional neural network (CNN) methods.
The Raman spectral input data can comprise 1D Raman spectra or 2D Raman spectra. In certain embodiments, the Raman spectral input data comprises a mean Raman spectrum. The tissue classification can comprise pancreatic cancer tissue or non-cancer pancreatic tissue. In one embodiment, the tissue classification comprises laryngeal cancer tissue or non-cancer laryngeal tissue.
An additional aspect of the present invention comprises a method of diagnosing cancerous tissues. In embodiments, the method comprises positioning patent for the according to a standard of care for a procedure; deploying the device according to any one or more of the various device embodiments disclosed herein, wherein deploying the device comprises inserting the device into the patient; identifying an anatomical tissue of interest; subjecting the anatomical tissue of interest to Raman spectroscopy; and classifying the anatomical tissue as cancerous or non-cancerous.
In embodiments, the sheath comprises one or more channels, such as a Raman probe channel and/or a Laryngoscope channel. “Channel” can refer to a lumen running all or part of the length of the sheath, for example, to support a functional element.
In embodiments, the sheath is a component of a Raman spectroscopy system. For example, the Raman spectroscopy system comprises the system of
Aspects of the invention are drawn to a Raman spectroscopy system. For example, the Raman spectroscopy system as displayed in
Aspects of the invention are further drawn to methods of using the Raman spectroscopy system as described herein. For example, a method of identifying a lesion as premalignant or malignant comprising using the Raman spectroscopy system as described herein. For example, the lesion comprises a vocal fold lesion, a laryngeal lesion, a pancreatic cancer, or an oral cancer.
Other objects and advantages of this invention will become readily apparent from the ensuing description.
This view shows the tip of the Raman probe extending from the housing and body of the probe.
Detailed descriptions of one or more embodiments are provided herein. It is to be understood, however, that the present invention can be embodied in various forms. Therefore, specific details disclosed herein are not to be interpreted as limiting, but rather as a basis for the claims and as a representative basis for teaching one skilled in the art to employ the present invention in any appropriate manner.
The singular forms “a”, “an” and “the” include plural reference unless the context clearly dictates otherwise. The use of the word “a” or “an” when used in conjunction with the term “comprising” in the claims and/or the specification can mean “one,” but it is also consistent with the meaning of “one or more,” “at least one,” and “one or more than one.”
Wherever any of the phrases “for example,” “such as,” “including” and the like are used herein, the phrase “and without limitation” is understood to follow unless explicitly stated otherwise. Similarly, “an example,” “exemplary” and the like are understood to be nonlimiting.
The term “substantially” allows for deviations from the descriptor that do not negatively impact the intended purpose. Descriptive terms are understood to be modified by the term “substantially” even if the word “substantially” is not explicitly recited.
The terms “comprising” and “including” and “having” and “involving” (and similarly “comprises”, “includes,” “has,” and “involves”) and the like are used interchangeably and have the same meaning. Specifically, each of the terms is defined consistent with the common United States patent law definition of “comprising” and is therefore interpreted to be an open term meaning “at least the following,” and is also interpreted not to exclude additional features, limitations, aspects, etc. Thus, for example, “a process involving steps a, b, and c” means that the process includes at least steps a, b and c. Wherever the terms “a” or “an” are used, “one or more” is understood, unless such interpretation is nonsensical in context.
As used herein, the term “about” can refer to approximately, roughly, around, or in the region of. When the term “about” is used in conjunction with a numerical range, it modifies that range by extending the boundaries above and below the numerical values set forth. In general, the term “about” is used herein to modify a numerical value above and below the stated value by a variance of 20 percent up or down (higher or lower).
For purposes of the present disclosure, it is noted that spatially relative terms, such as “up,” “down,” “right,” “left,” “beneath,” “below,” “lower,” “above,” “upper” and the like, can be used herein for ease of description to describe one element or feature's relationship to another element(s) or feature(s) as illustrated in the figures. It will be understood that the spatially relative terms are intended to encompass different orientations of the device in use or operation in addition to the orientation depicted in the figures. For example, if the device in the figures is turned over or rotated, elements described as “below” or “beneath” other elements or features would then be oriented “above” the other elements or features. Thus, the exemplary term “below” can encompass both an orientation of above and below. The device can be otherwise oriented (rotated 90 degrees or at other orientations) and the spatially relative descriptors used herein interpreted accordingly.
As used herein, the term “standard of care” can refer to a diagnostic and/or treatment process for which a clinician follows for a certain type of patient, illness, or clinical circumstance. For example, a standard of care can refer to the ordinary level of skill and care that a clinician is expected to observe in providing clinical care to a patient. In embodiments, the standard of care can vary depending on the patient, the illness, or clinical circumstance. As used herein, “standard care practices” can refer to practices which are standard of care.
As used herein, the term “clinician” can refer to a person qualified in the clinical practice of medicine, psychiatry, or psychology. As used herein, the terms “clinician” and “practitioner” can be used interchangeably. For example, “clinician” can refer to a physician, a surgeon, a veterinarian, a physician assistant, a nurse, or a person practicing under the supervision thereof.
As used herein, the term “board-certified” can refer to a professional whose qualifications have been approved by an official group or governing body. For example, the person is a physician who has graduated from medical school, completed residency, trained under supervision in a specialty, and passed a qualifying exam given by a medical specialty board.
The terms “subject” and “patient” as used herein include all members of the animal kingdom including, but not limited to, mammals, animals (e.g., cats, dogs, horses, swine, etc.) and humans.
The term “cancer surgery” can include a surgical procedure for diagnosis or treatment of a cancer. The term can refer to a procedure for biopsy of potentially cancerous tissue, a procedure for resection or removal of cancerous tissue, or a combination thereof.
Disclosed herein is a sheath-probe device configured such that a Raman spectroscopy system can be deployed during an endoscopic or laparoscopic procedure. In embodiments, the device enables the user to perform an optical biopsy. As used herein, the term “optical biopsy” can refer to the utilization of properties of light to determine a diagnosis of a specimen, target tissue, or tissue of interest. For example, the specimen, target tissue, or tissue of interest can be an organ, a tissue, or a cell. For example, the organ can comprise a heart, an appendix, a larynx, a lymph node, an ovary, a kidney, a liver, a lung, a brain, a pancreas, a bladder, a stomach, an intestinal organ, cheek, gums, a tongue, skin, a tissue thereof, or a cell thereof. As used herein, the term “specimen” can refer to any organ, tissue, cell, or biological sample. As used herein, the terms “specimen,” “sample,” “target tissue,” or “anatomical tissue of interest” can be used interchangeably.
As used herein, “diagnosis” can refer to a characterization or classification of a disease, disorder, or lack thereof from a sign, symptom, or marker. For example, a specimen can be classified as non-malignant, pre-malignant, or malignant. As used herein, the terms “normal”, “non-cancer”, “benign”, and “non-malignant” can be used interchangeably. In embodiments, “pre-malignant” can refer to a state or classification which a specimen has that has a tendency or inclination to become malignant but is not currently classified as malignant based upon conventional standards. For example, the pre-malignant specimen can be keratosis with dysplasia. In embodiments, “malignant” can refer to a state or classification to a specimen has having characteristics of uncontrollable cell growth, infiltration, destruction, metastases, or a combination thereof. For example, a malignant comprises carcinoma. For example, the carcinoma comprises carcinoma in situ or invasive carcinoma. As used herein, the terms “malignant”, “cancer”, and “cancerous” can be used interchangeably.
In embodiments, the device described herein can be used to establish appropriate surgical margins. As used herein, “surgical margins” can refer to the edge of a resected sample. The terms “appropriate” or “adequate” in regard to surgical margins can refer to margins which are not positive for the cells targeted for removal. As used herein, the terms “superficial” in regard to surgical margins can refer to those which are positive. A “positive” margin can refer to a resection that contains the cells targeted for removal on the edge of the resected sample. For example, the cells targeted for removal are cancer cells. In embodiments, positive surgical margins can be indicative of an unsuccessful resection. As used herein, the term “negative” in reference surgical margins can refer to a resection that does not contain cells targeted for removal in the surgical margins. In embodiments, a negative surgical margin can be indicative of a successful resection.
In embodiments, the device described here can be configured to help inform a clinician or practitioner of a disease prognosis or clinical stage. As used herein, a clinician or practitioner can refer to a physician, surgeon, veterinarian, physician assistant, nurse, or student thereof.
In embodiments, the device described herein can be used to grade a specimen or to inform the clinician or practitioner of a grade of a specimen. As used herein, the term “grade” or “grading” can refer to the measure or description of specimen based upon whether the specimen comprises characteristics that are consistent with normal or abnormal specimens. “Grade” can further include a description of how quickly an abnormal specimen is expected to grow or spread.
The device herein can use Raman spectroscopy as a non-destructive tool for biological sample imaging and analysis. As used herein, the term “non-destructive” can refer to the ability to not permanently alter a specimen or the surrounding tissue after being subjected to a condition. When monochromic light from a laser source strikes a sample, the photons are absorbed by its surface and reemitted. The majority of the reemitted light occurs at the same frequency as the monochromic source (such reemitted light can be referred to as “elastic scattering”). If the sample is Raman-active, a portion of the reemitted light will radiate at frequencies above and below the incident frequency (such reemitted light can be referred to as “inelastic scattering”). As used herein, the term “Raman active” can refer to any molecule that exhibits the Raman effect. For example, a molecule that exhibits the Raman effect experiences a change in polarizability as a result of molecular vibration. This inelastic scattering of the photons by the component(s) in the sample can be used to generate a Raman spectrum, from which a structural fingerprint can be identified. As used herein, a “structural fingerprint” can refer to a spectral characteristic that is indicative of a molecular feature or structure. For example, the signal can comprise Raman shift (cm−1). For example, the molecular feature can be indicative of a functional group or other structural characteristic. For example, peaks and frequency of peaks at specific wavenumbers can be related to specific biological components. For example, the biological components are protein contents. For example, the protein content can be collagen. For example, the biological components can comprise lipids, nucleic acids, or a combination thereof. In embodiments, biological components in a specimen can be indicative of a cancerous or non-cancerous specimen.
In embodiments, the Raman probe comprises a hand-held probe configured for use during a medical procedure.
In embodiments, the Raman spectroscopy systems disclosed herein can be used in conjunction with an endoscope, such as those disclosed in
In embodiments the sheath 600, the Raman probe shaft 450, or both are reusable. The sheath 600, the Raman probe shaft 450, or both can be sterilizable. In certain embodiments, the sheath 600, the Raman probe shaft 450, or both are disposable.
In reusable embodiments, the sheath 600, the Raman probe shaft 450, or both can be sterilized or can be sterile when provided or obtained. By way the example, the sheath 600, the Raman probe shaft 450, or both can be configured to withstand autoclave sterilization, chemical sterilization, x-ray sterilization, or a combination thereof. In alternate embodiments, the sheath 600, the Raman probe shaft 450, or both can be configured for a single use.
The sheath 600, the Raman probe shaft 450, or both can be comprised of any material currently known by those of skill in the art or later developed that is suitable for use in medical or surgical procedures. In embodiments, the sheath 600, the Raman probe shaft 450, or both are comprised of a medical-grade material. The sheath 600, the Raman probe shaft 450, or both can be comprised of a medical grade polymer, metal, or a combination thereof. In certain embodiments, the sheath 600, the Raman probe shaft 450, or both are comprised of surgical metal. The sheath 600, the Raman probe shaft 450, or both can be comprised of stainless steel, titanium, tantalum, gold, platinum, palladium, or any other metal or combination of metals suitable for surgical use.
The inset of
In certain embodiment, the switch 570, 870 is configured to permit activation or engagement of the excitation laser and signal collection. The switch 570, 870 can be configured to turn off a light source. The switch 570 can be configured to turn off a light source and active the excitation laser. In certain embodiments, the switch 570, 870 is configured to simultaneously turn off the light source and turn on the excitation laser. The Raman probe 500, 800 can comprise control cables configured to control activation of the excitation laser, deactivation of the excitation laser, activation of the light source, deactivation of the light source, activation of the Raman spectrometer for signal collection, positioning of the Raman probe tip, positioning of the incident laser angle, or a combination thereof.
In embodiments, the Raman probe, the sheath 600 or a combination thereof comprise an outer diameter that is sufficient to fit within the nasal cavity or oral cavity of a patient. In embodiments, the outer diameter of the Raman probe, the sheath, or a combination thereof comprises an outer diameter of up to about 20 mm. In embodiments, the outer diameter of the Raman probe, the sheath, or a combination thereof comprises an outer diameter of up to about 10 mm. The outer diameter of the Raman probe, the sheath, or a combination thereof can comprise an outer diameter of about 1 mm, about 2 mm, about 3 mm, about 4, mm, about 5 mm, about 6 mm, about 7 mm, about 8 mm, about 9 mm, or about 10 mm. In certain embodiments, the outer diameter of the Raman probe, the sheath, or a combination thereof comprises a diameter of about 0.25 mm, about 0.5 mm, about 0.75 mm, about 1 mm, about 1.25 mm, about 1.5 mm, about 1.75 mm, about 2 mm, about 2.25 mm, about 2.5 mm, about 2.75 mm, about 3 mm, about 3.25 mm, about 3.5 mm, or about 4 mm.
In certain embodiments, the various embodiments described herein, comprises a sheath 600, a Raman probe system 200, 300, 400, an endoscope 100, 200, 1200, or a combination thereof. The Raman probe system described herein can comprise a Raman probe 250, 350, 450, 500, 800, a laser source 215, 315, an excitation signal filter 222, a collection filter 212, a charge couple device detector, a signal collection system, a housing unit 530, 830, a computer 312, a display, or a combination thereof. In embodiments, the endoscope can comprise an arthroscope, a bronchoscope, a colonoscope, a hysteroscope, a laparoscope, a laryngoscope, a mediastinoscope, sigmoidoscope, thoracoscope, ureteroscope, or an endoscope for use in operative endoscopy. As used herein, the term “endoscopy” can refer to any means for examining inside the body for medical purposes. As used herein, the term “endoscope” can refer to any instrument used to examine the interior of a body.
In embodiments, the endoscope comprises a rigid endoscope 150 or a flexible endoscope 110. For example, the flexible endoscope comprises a fiberoptic endoscope. In embodiments, the Raman probe comprises a flexible endoscope. In further embodiments, the Raman probe is a fiberoptic Raman probe. As used herein, the term “fiberoptic” can refer to the medium or technology with transmission of information as light pulses along a glass or plastic strand or fiber. In embodiments, the fiberoptic laryngoscope can be used in nasolaryngoscopy.
In embodiments, the laryngoscopy can comprise direct laryngoscopy and indirect laryngoscopy. As used herein, the term “direct laryngoscopy” can refer to a laryngoscopy wherein the clinician can visualize the anatomical area of interest by direct line of sight. As used herein, the term “indirect laryngoscopy” can refer to a laryngoscopy wherein the clinician visualizes the anatomical area of interest by means other than direct line of sight. For example, indirect laryngoscopy can comprise mirror or video-based visualizations. In certain embodiments, a camera can be utilized to obtain images of the anatomical area, which are transmitted the clinician for review. In embodiments, a flexible endoscope is used for indirect laryngoscopy.
As described herein, the term “sheath” can refer to a structure that envelopes the other. In embodiments, the device described herein comprises a sheath which envelopes a Raman probe shaft, an endoscopic probe shaft, or a combination thereof. The sheath dimensions of the device described herein can be varied to accommodate various flexible and rigid endoscopes. In embodiments, the Raman probe and endoscopic probe are deployed simultaneously via the sheath. In embodiments, the Raman probe can be deployed prior to the endoscopic probe. In embodiments, the endoscopic probe can be deployed prior to the Raman probe.
The device described herein can be configured to reduce or eliminate ambient or interfering light. In embodiments, the device can electronically turn off the endoscopic light during Raman spectrum acquisition. For example, an electronic control system can engage the Raman analysis while simultaneously switching off the endoscopic illumination. In embodiments, an electronic switch 570, 870 connects the probe fiber to the Raman spectrometer and a light emitting diode (LED) aiming beam. The housing unit 530, 830 can comprise a switch 570, 870 and circuitry to activate the Raman spectrometer and toggle between the LED source (e.g. the endoscopic light) and the incident laser signal. In certain embodiments, the probe instrument tip design can prevent introduction of ambient light due to a fiber tip offset (see the inset of
The probe instrument tip design can further be configured to control of the incident laser angle. In certain embodiments, the clinician can control the incident laser angle by hand, such as via rotation of the Raman probe until the desired incident laser angle is achieved.
In various embodiments, the presently disclosed device is configured to allow for signal detection in real-time. As used herein, the phrase “real time” indicates that data detection and collection becomes available to internal or external users as soon as the data is available or generated or within seconds or milliseconds of such data availability or generation. In certain embodiments, tissue analysis (e.g. cancerous versus non cancer, diseased versus normal, etc.) is transmitted in “real time.” In embodiments, “real time” can mean that less than about one minute. “Real time” can be less than about one second. In embodiments, “real time” can be less than about 100 milliseconds. “Real time” can be about 100 milliseconds, about 200 milliseconds, about 300 milliseconds, about 400 milliseconds, about 500 milliseconds, about 600 milliseconds, about 800 milliseconds, or about 900 milliseconds.
In embodiments, the laser source of the Raman system comprises a diode laser. The diode laser can comprise wavelengths in the visible, near infrared, infrared, near ultraviolet, or ultraviolet spectrum. In embodiments, the diode laser excitation wavelengths comprise about 532 nm, about 638 nm, about 785 nm, about 1064 nm.
In embodiments, the “optimal distance” can refer to the distance between the distal end of the Raman laser fiber and the specimen. The term “incident laser angle” can refer to the angle at which the incident laser (also referred to herein as “excitation laser” contacts the specimen or the angle between the sample and the Raman probe. In certain embodiments, the optimal distance, the incident laser angle, or both can depend upon the beam size. The optimal distance can vary with the incident angle. The incident angle can vary with the optimal distance. In embodiments, the optimal laser distance comprises about 0 mm to about 20 mm. the optimal distance can be about 1 mm, about 2 mm, about 3 mm, about 4 mm, about 5 mm, about 6 mm, about 7 mm, about 8 mm, about 9 mm, about 10 mm, about 11 mm, about 12 mm, about 13 mm, about 14 mm, or about 15 mm. In certain embodiments, the optimal distance is greater than about 20 mm. The optimal distance can be less than about 1 mm.
In embodiments, the systems, devices, and methods disclosed herein are configured to permit collection of reliable Raman data without damaging the target tissue. Embodiments are designed to enable tissue laser exposure times of less than about 30 seconds, less than about 20 seconds, less than about 10 seconds, less than about 5 seconds, less than about 2.5 seconds, or less than about 1 second. In embodiments, the maximum energy exposure of tissue subjected to the incident laser is less than that required to cause histological damage to skin. The maximum energy exposure can be less than that required to reveal histological damage to a mucosal layer. In one embodiment, the maximum energy exposure of the tissue of interest is less than about 6 joules, less than about 5 joules, less than about 4 joules, less than about 3 joules, less than about 2 joules, or less than about 1 joule. In one embodiment, the maximum energy exposure is less than about 5 joules/cm2, less than about 4 joules/cm2, less than about 3 joules/cm2, less that about 2 joules/cm2, or less than about 1 joule/cm2, In an embodiment, the maximum energy exposure to tissue subjected to the incident laser is about 2.2 joules/cm2, about 2.18 joules/cm2, about 2.16 joules/cm2, about 2.14 joules/cm2, about 2.12 joules/cm2, about 2.10 joules/cm2, about 2.08 joules/cm2, about 2.06 joules/cm2, about 2.04 joules/cm2, about 2.02 joules/cm2, about 2.0 joules/cm2, about 1.8 joules/cm2, about 1.6 joules/cm2. about 1.4 joules/cm2, about 1.2 joules/cm2, about 1.0 joules/cm2, about 0.8 joules/cm2, about 0.6 joules/cm2, about 0.4 joules/cm2, or about 0.2 joules/cm2.
Embodiments of the device comprise machine learning assisted analysis of Raman signals to differentiate between normal, pre-malignant, and malignant tissue samples. As used herein, the phrase “machine learning” can refer artificial intelligence or software algorithms used to implement supervised learning, unsupervised learning, reinforcement learning, or any combination thereof. For example, the algorithm can learn from and make predictions about data. As used herein, the phrase “deep learning” can refer to a form of machine learning that utilizes multiple interconnected neural network layers along with feedback mechanisms or other methods to improve the performance of the underlying neural network.
Spectral analysis methods described herein comprise principal component analysis (PCA), random forest (RF), and one-dimensional convolutional neural network (CNN) methods.
In embodiments, the devices and methods disclosed herein can be communicatively coupled with computer networks, computing devices, mobile devices, or combinations thereof. Under certain embodiments, the systems and methods disclosed herein can utilize the communicative coupling to relay data collected from the collection system. Such data can include, for example, the Raman spectral data.
The communicative coupling can be accomplished through one or more wireless communications protocols. The communicative coupling can comprise a wireless local area network (WLAN). A WLAN connection can implement WiFi™ communications protocols. Alternatively, the communicative coupling can comprises a wireless personal area network WPAN. A WPAN connection can implement Bluetooth™ communications protocols.
Embodiments can comprise a data port for relaying data to the mobile device or other computing device. The data port can comprise a USB connection or any other type of data port. The data port allows for a wired communication between the collection system and separate computing devices. The data port can be used alone or in combination with the wireless communications protocols of the systems and device described above.
Computer networks suitable for use with the embodiments described herein include local area networks (LAN), wide area networks (WAN), Internet, or other connection services and network variations such as the world wide web, the public internet, a private internet, a private computer network, a public network, a mobile network, a cellular network, a value-added network, and the like. Computing devices coupled or connected to the network can be any microprocessor controlled device that permits access to the network, including terminal devices, such as personal computers, workstations, servers, mini computers, main-frame computers, laptop computers, mobile computers, palm top computers, hand held computers, mobile phones, TV set-top boxes, or combinations thereof. The computer network can include one of more LANs, WANs, Internets, and computers. The computers can serve as servers, clients, or a combination thereof.
One or more components of the systems and methods described herein and/or a corresponding interface, system or application to which the systems and methods described herein are coupled or connected includes and/or runs under and/or in association with a processing system. The processing system includes any collection of processor-based devices or computing devices operating together, or components of processing systems or devices, as is known in the art. For example, the processing system can include one or more of a portable computer(s), portable communication device operating in a communication network, a network server, or a combination thereof. The portable computer can be any of a number and/or combination of devices selected from among personal computers, personal digital assistants, portable computing devices, and portable communication devices, but is not so limited. The processing system can include components within a larger computer system.
The processing system of an embodiment includes at least one processor. The term “processor” as generally used herein refers to any logic processing unit, such as one or more central processing units (CPUs), digital signal processors (DSPs), application-specific integrated circuits (ASIC), etc. The processor can be disposed within or upon a single chip.
The processing system can further include at least one memory device or subsystem. The processing system can also include or be coupled to at least one database. The processor and memory can be monolithically integrated onto a single chip, distributed among a number of chips or components, and/or provided by some combination of algorithms. The systems and methods described herein can be implemented in one or more of software algorithm(s), programs, firmware, hardware, components, circuitry, in any combination.
The components of any system that include the systems and methods described herein can be located together or in separate locations. Communication paths couple the components and include any medium for communicating or transferring files among the components. The communication paths include wireless connections, wired connections, and hybrid wireless/wired connections. The communication paths also include couplings or connections to networks including local area networks (LANs), metropolitan area networks (MANs), wide area networks (WANs), wireless personal area networks (WPANs), proprietary networks, interoffice or backend networks, and the Internet. Furthermore, the communication paths include removable fixed mediums like floppy disks, hard disk drives, and CD-ROM disks, as well as flash RAM, Universal Serial Bus (USB) connections, RS-232 connections, telephone lines, buses, and electronic mail messages.
Aspects of the systems and methods described herein can be implemented as functionality programmed into any of a variety of circuitry, including programmable logic devices (PLDs), such as field programmable gate arrays (FPGAs), programmable array logic (PAL) devices, electrically programmable logic and memory devices and standard cell-based devices, as well as application specific integrated circuits (ASICs). Some other possibilities for implementing aspects of the systems and methods described herein include: microcontrollers with memory (such as electronically erasable programmable read only memory (EEPROM)) or without memory, embedded microprocessors, firmware, software, etc. Furthermore, aspects of the systems and methods of surgical simulation described herein can be embodied in microprocessors having software-based circuit emulation, discrete logic (sequential and combinatorial), custom devices, fuzzy (neural) logic, quantum devices, and hybrids of any of the above device types. Of course the underlying device technologies can be provided in a variety of component types, e.g., metal-oxide semiconductor field-effect transistor (MOSFET) technologies like complementary metal-oxide semiconductor (CMOS), bipolar technologies like emitter-coupled logic (ECL), polymer technologies (e.g., silicon-conjugated polymer and metal-conjugated polymer-metal structures), mixed analog and digital, etc.
It is noted that any system, method, and/or other components disclosed herein can be described using computer aided design tools and expressed (or represented), as data and/or instructions embodied in various computer-readable media, in terms of their behavioral, register transfer, logic component, transistor, layout geometries, and/or other characteristics. Computer-readable media in which such formatted data and/or instructions can be embodied include, but are not limited to, non-volatile storage media in various forms (e.g., optical, magnetic or semiconductor storage media) and carrier waves that can be used to transfer such formatted data and/or instructions through wireless, optical, or wired signaling media or any combination thereof. Examples of transfers of such formatted data and/or instructions by carrier waves include, but are not limited to, transfers (uploads, downloads, e-mail, etc.) over the Internet and/or other computer networks via one or more data transfer protocols (e.g., HTTP, FTP, SMTP, etc.). When received within a computer system via one or more computer-readable media, such data and/or instruction-based expressions of the above described components can be processed by a processing entity (e.g., one or more processors) within the computer system in conjunction with execution of one or more other computer programs.
Described herein are non-limiting, exemplary embodiments for clinical use of the device(s) described herein. As standard of care is determined by the type of patient, illness, and clinical situation, the embodiments described herein are non-limiting and exemplary in nature and can be adapted for standard of care practice by one of ordinary skill in the art. For example, one of ordinary skill comprises a board-certified physician, board certified surgeon, or a person practicing under the supervision thereof.
POSITIONING THE PATIENT—The patient can be positioned according to standard of care. For example, if a patient is to be subjected to direct laryngoscopy, the patient can be placed in a “sniffing” position. For example, the sniffing position can comprise placing the patient in a supine position, elevating the head about 15°, and extending the neck about 35°. In embodiments, the patient's mouth can then be opened by flexing the thumb and middle finger past each other such that the thumb is pressing on the mandibular dentition and the middle finger is pressing on the maxillary dentition.
For example, if the patient is to be subjected to an upper GI endoscopy procedure, the patient can be placed in a left lateral decubitus position with the chin tucked against the chest. A bite guard can then be placed between the patient's teeth or fitted over the device sheath.
For example, if the patient is to be subjected to a lower GI endoscopy, the patient can be placed in a left lateral decubitus position.
For example, if the patient is to be subjected to a laparoscopic procedure, the patient can be placed in a supine position.
DEPLOYING THE DEVICE—The device can be deployed according to the standard of care for deploying an endoscopic or laparoscopic procedure. For example, if a patient is to be subjected to direct laryngoscopy, the laryngoscope can be inserted in the right side of the mouth and the Raman probe can be deployed similarly to that of a suction catheter. In embodiments, the Raman probe shaft and the laryngoscope can be deployed simultaneous via a sheath-based delivery system as more particularly described herein (see, for example
For example, if a patient is to be subjected to an upper GI endoscopy, and a bite guard has been placed in the patient's mouth, the presently disclosed device can be slid into the patient's mouth through the bite guard. If the bite guard has been fitted over the presently disclosed device sheath rather than directly in the patient's mouth, the bite guard will not be placed until the presently disclosed device has entered the upper esophagus of the patient.
For example, if a patient is to be subjected to a lower GI endoscopy, the clinician can lubricate the device for insertion into the anus and advance the device to the desired location.
For example, if a patient is to be subjected to a laparoscopic procedure, the clinician can make an incision in the patient's skin to create a port. For example, the incision can be made on the abdomen. For example, the incision size can be about 0.5 to about 1.0 cm in diameter. The incision size can be less than about 0.5 cm. In embodiments, the incisions size is greater than about 1.0 cm. The incision size can be about 0. 1 cm, about 0.2 cm, about 0.3 cm, about 0.4 cm, about 0.5 cm, about 0.6 cm, about 0.7 cm, about 0.8 cm, about 0.9 cm, about 1 cm, about 1.1 cm, about 1.2 cm, about 1.3 cm, about 1.4 cm, or about 1.5 cm. In embodiments, a trochar can be inserted into the port and the device can be deployed via the trochar.
IDENTIFYING ANATOMICAL TISSUE—The identification of anatomical tissue of interest and surgical margins can be determined according to standard of care for a particular anatomical tissue and clinical scenario. For example, identification of anatomical tissue can be performed through visualization with the naked eye, a microscope, an endoscope, a laparoscope, a camera/video device, or a mirror. In embodiments, one of ordinary skill in the art (e.g., a surgeon) can visualize a tissue identify anatomical tissue as a tissue of interest. For example, the tissue of interest is a cancer tissue.
In embodiments, the clinician can mark the surgical margins as about 0 mm, about 0.25 mm, about 0.5 mm, about 1.0 mm, about 1.5 mm, about 2.0 mm, about 2.5 mm, about 5 mm, about 10 mm, or about 15 mm from the center of the tissue of interest.
For example, if a patient is to be subjected to direct laryngoscopy, the clinician can move the device to the vocal cord tissue for visualization, identify an area that can comprise cancer, and identify surgical margins. For example, the surgical margin can be about 5 mm.
In an embodiment where a patient is to be subjected to an upper GI endoscopy, the clinician can move the device to an anatomical tissue of interest in the esophagus, stomach, or small intestine.
In embodiments wherein a patient is to be subjected to a lower GI endoscopy, the clinician can move the device to an anatomical tissue of interest in the large intestine.
In embodiments where a patient is to be subjected to a laparoscopic procedure, the clinician can move the device to an anatomical tissue of interest inside the patient's body cavity.
CLASSIFYING ANATOMICAL TISSUE AND SURGICAL MARGINS—The Raman probe device as evident from any of the various embodiments disclosed herein can be utilized to classify anatomical tissue and determine adequate surgical margins. In embodiments, when an anatomical tissue of interest is identified and surgical margins (as determined by standard of care) have been marked, the clinician can utilize the any of the various presently disclosed embodiments to acquire a Raman spectrum and assist with classification of the tissue. In some embodiments, the clinician can mitigate any intervening light sources via a switch or by design of the probe tip. Once a Raman spectrum has been acquired, computer analysis can inform the clinician of the classification of the tissue. For example, the method of informing the clinician can comprise an auditory signal, such as a tone, pitch, or statement, or a visual signal such as light flashes. For example, the pitch or the light flash can indicate the presence of cancer or the absence of cancer. The clinician can then use the computer output to inform their decisions regarding adequate surgical margins during tissue resection.
In embodiments the ambient or interfering light can be reduced via a switch within the device. In embodiments, the device can electronically turn off the endoscopic light during Raman spectrum acquisition. In one embodiment, the probe instrument tip design can permit contact with the tissue site. In embodiments, the probe tip design can control the incident laser angle.
In certain embodiments, the clinician can control the incident laser angle by hand, such as via rotation of the Raman probe until the desired incident laser angle is achieved.
An electronic control system can be utilized to engage the Raman analysis while simultaneously switching off any illumination, such as that that can be introduced via a fiberoptic light associated with the endoscope. In embodiments, an electronic switch connects the probe fiber to the Raman spectrometer and a light emitting diode (LED) aiming beam. The housing unit can be configured to contain a momentary switch and circuitry to activate the Raman spectrometer and toggle between the LED source and the incident laser signal. In embodiments, the Raman probe tip is configured such that the tip blocks any ambient or intervening light sources. For example, when the Raman probe tip contacts a specimen, the tip surrounding the Raman probe can block ambient or interfering light (e.g. see
Examples are provided below to facilitate a more complete understanding of the invention. The following examples illustrate the exemplary modes of making and practicing the invention. However, the scope of the invention is not limited to specific embodiments disclosed in these Examples, which are for purposes of illustration only, since alternative methods can be utilized to obtain similar results.
We disclose herein a Raman spectroscopy system that incorporates a tissue probe into a thin sheath. The design can be adapted to any flexible or rigid laryngoscope. The fiberoptic probe with lens has an outer diameter of approximately 2 mm with a length up to 1.5 m depending on the laryngoscope. The probe will permit laser light delivery and signal collection in real time during endoscopy.
In addition to the optical probe, the spectrometer includes a 785 nm diode laser light source, a dual wave number (800-1800 cm−1 and 2800-3600 cm−1) signal collection system and a computer for signal analysis and display. In embodiments, the signal collection system can collect spectra on a range of about 200 cm−1 to about 4000 cm−1. Photographs of the scopes and designs for the spectrometer and sheath-probe are shown in
In embodiments, the sheath-based Raman spectroscopy system can be commercially manufactured such that clinicians can routinely use the system in the clinic or the operating room during laryngoscopy.
In an ambulatory setting, clinicians can use the Raman spectroscopy system during flexible laryngoscopy to screen laryngeal lesions for premalignant (keratosis with dysplasia) and malignant (carcinoma in situ, invasive carcinoma) changes. The visual features of these lesions are not diagnostic. Multiple biopsies are often necessary for diagnosis. Raman spectroscopy will simplify post-operative management of laryngeal malignancy during serial follow up laryngoscopy.
Adequate resection is the most significant determinant of success after larynx sparing surgery for carcinoma. Margin detection is especially difficult during endoscopic surgery for malignancy. Deployed on a rigid c, Raman spectroscopy will permit precise margin control during endoscopic laryngeal surgery.
Similar to the laryngeal cancer surgery, the major challenge of the current pancreatic cancer surgery is the inability to remove all existing tumor. With a flexible operative endoscope, Raman spectroscopy will allow the precise intraoperative cancer detection, which aids the complete resection of pancreatic cancer. Raman spectroscopy will be used to detect primary tumor, satellite tumor, tumor margin and surgical bed.
A design that adapts the Raman probe to both flexible and rigid scopes will allow deployment in the clinic for early detection of premalignant and malignant vocal fold lesions and in the operating room for precise intraoperative margin control during endoscopic resection. There are no existing sheath-based designs for flexible laryngoscopy and there are no Raman probe adaptations for rigid operative endoscopy.
The Raman signal is weak and the device has to be carefully designed with enough sensitivity and calibrated with ex vivo specimens. This is a miniaturized probe design for sheath deployment. There are additional challenges with device miniaturization while maintaining signal detection sensitivity and short acquisition times.
We have developed a miniaturized spectroscopy system that can work with endoscopes including sigmoidoscopes, angioscopes, and dental imaging devices using near infrared spectroscopy. The existing technology can be adapted to a Raman spectroscopy system.
See, for example, Li Z, et al. Endoscopic near-infrared dental imaging with indocyanine green: a pilot study. Ann. N.Y. Acad. Sci. 2018; 1421:88-96, which is incorporated herein by reference in its entirety.
Disclosed herein is a Raman spectroscopy system that can be deployed in an endoscope sheath during laryngoscopy. The device will enable optical diagnosis of laryngeal lesions including carcinoma and accurate margin detection during endoscopic laryngeal tumor resection. The dimensions of the sheath can vary to accommodate different fiberoptic and rigid endoscopes. There are commercially produced sheaths used as microbial barriers during endoscopy and a similar design should accommodate the Raman fiberoptics. There are no prior reports of an endoscopy sheath-probe device with Raman fiberoptics and there are no reports of spectroscopy performed with a rigid endoscope.
Early detection of laryngeal cancer significantly increases survival rates and permits more conservative, larynx sparing treatments. It also reduces the length of hospital stay and health care costs1. The past 20 years have seen a trend towards more conservative, larynx sparing surgery for stage I and II laryngeal carcinoma performed through a trans-oral endoscopic approach2. Achieving adequate margins of resection can be challenging. A non-invasive, optical form of biopsy for laryngeal carcinoma will improve early detection, permit more accurate surveillance for recurrence and improve intraoperative margin control.
Raman spectroscopy measures the difference between incident and scattered monochromatic light that is a function of the molecular composition of tissue. Previous studies have demonstrated a spectral difference between normal tissue, benign lesions and carcinoma in ex vivo laryngeal specimens3,4.
Lin, et al. published the only report of in vivo Raman spectroscopy for laryngeal lesions in 20165. They documented spectral discrimination between normal and cancerous tissue during laryngoscopy. Their system used simultaneous 800-1800 cm−1 and 2800-3600 cm−1 wave number analysis. The Raman spectroscopy system included a near infrared diode laser source (785 nm) and a thermo-electric cooled CCD camera. They achieved a diagnostic accuracy of 91.1% (sensitivity—93.3%, specificity—90.1%) for laryngeal cancer. Acquisition times were on the order of milliseconds. They used a 1.8 mm (O.D.) fiberoptic probe with a 1 mm Sapphire ball lens at the distal tip. The Raman probe used during the Lin study was deployed intraoperatively through the operating channel of a flexible laryngoscope. The type and dimensions of the laryngoscope were not specified in their report.
An embodiment described herein is a sheath delivery system that incorporates the Raman probe. The dimensions of the sheath can vary to accommodate different fiberoptic and rigid endoscopes. Embodiments further comprise a disposable sheath design. There are no prior reports of an endoscopy sheath-probe device with Raman fiberoptics and there are no reports of spectroscopy performed with a rigid endoscope.
Specialists inspect the vocal folds and other parts of the larynx with specialized endoscopes. Typical indications for laryngoscopy include voice changes, difficulty breathing, dysphagia and hemoptysis. The scopes are available in two basic designs. Flexible scopes allow simple diagnostic evaluation in an un-anesthetized patient. Rigid scopes afford greater magnification and resolution than flexible scopes but are used during operative procedures under general anesthesia performed especially for biopsy and surgical resection of laryngeal lesions.
Current video endoscopy systems rely on visual evaluation by the specialist for interpretation. Spectrographic and other optical assessment techniques during laryngoscopy is under-utilized because of cost and deployment difficulties.
When monochromic light from a laser source strikes a substance, photons are absorbed by its surface and reemitted. Most of the reemitted light occurs at the same frequency as the monochromic source (elastic scattering). Depending on the sample, a small portion of the reemitted light will radiate at frequencies above and below the incident frequency (inelastic scattering). Inelastic scattering is dependent upon the molecular structure of the specimen and is termed the Raman effect. One can determine the molecular structure of certain substances using their Raman spectra.
A typical Raman spectroscopy system comprises a laser light source, an optical filter to define the incident frequency and a charge couple device (CCD camera) to detect Raman scattering. Raman spectroscopy developers have incorporated a number of modifications to improve frequency resolution and acquisition times.
Raman spectroscopy is useful in any field where there is a need for a non-destructive chemical analysis or imaging. Engineers use it to detect chemical impurities and manufacturing defects. It also has applications in the environmental science, geology, pharmaceutical and semi-conductor industries.
Advantages include distinct visible spectra for numerous substances, small-required sample size and nondestructive analysis. Not all molecules exhibit Raman effect. For example, Raman spectroscopy cannot be used to analyze metals and alloys. Historically, the most significant disadvantage of Raman analysis has been its long signal acquisition time.
The Raman spectra of tissue specimens represent the weighted some of their macro molecular species and are highly tissue specific. Researchers have identified cellular differentiation in several epithelial types using Raman spectroscopy. In situ applications have been reported in brain6, bladder7, breast8, and skin9 neoplasms.
The American Cancer Society estimates that there are 13,000 new cases of laryngeal cancer diagnosed each year10. Approximately 3700 patients with laryngeal cancer die annually. Early diagnosis and precise control of disease margins are essential for the surgical management of laryngeal carcinoma. Important visual features related to malignancy can be missed on routine laryngoscopy and multiple biopsies are often necessary for diagnosis. The five-year relative survival rate for stage I glottic carcinoma is 90%. This drops to 74%, 56% and 44% for stages II, Ill and IV, respectively. Lack of locoregional control is the most significant cause of surgical failure. Stage I and II lesions are often treatable with larynx sparing endoscopic surgery as long as adequate margins of resection are achieved. Persistent positive margins can require total laryngectomy and/or radiation therapy that might otherwise have been avoided with complete initial resection.
Early detection of malignant and premalignant lesions of the larynx will significantly improve larynx preservation and survival rates after surgical treatment. Disclosed herein is a Raman spectroscopy optical biopsy system that can be used routinely in the clinic or operating room. A design that adapts the Raman probe to both flexible and rigid scopes will allow deployment in the clinic for early detection of premalignant and malignant vocal fold lesions and in the operating room for precise intraoperative margin control during endoscopic resection.
Annual direct healthcare expenditures for laryngeal disease are estimated between 178 and 294 million dollars with 50-70% spent on surgical management11. In the United States there are approximately 10 million medical encounters for laryngeal related symptoms annually. The exact number of laryngoscopies performed is difficult to estimate but it is one of the most commonly performed in-office procedures and most visits to an otolaryngologist with a complaint of vocal dysfunction will be evaluated endoscopically. With sufficient sensitivity and efficient deployment, Raman spectroscopy could become standard practice in adult airway endoscopy.
Our approach to deploying the fiberoptic probe in a sheath allows the system to adapt to any flexible or rigid telescopic laryngoscope. Deployed in-line with existing endoscopes, the device can be used routinely in the clinic or the operating room. There are no similar systems for laryngoscopy in existence.
A commercially manufactured, disposable fiberoptic spectroscopy sheath can be made from materials currently used in medical endoscopy systems. A disposable sheath could be deployed inexpensively. A one-use design avoids infection control, storage and sterilization issues and lends itself to unit pricing in the clinic and operating room. Similar systems such as fiberoptic surgical laser probes have proven attractive to hospital and clinic administrators where the vendor provides the base system free of charge and the user pays only for the disposables kept in inventory.
To our knowledge, there are no similar fiberoptic spectroscopy probe sheaths in development or production. The design to incorporate the probe into a disposable sheath with different dimensions for existing flexible and rigid endoscopes should provide a stable barrier to market competition.
Laryngeal cancer is diagnosed histologically. Many lesions spread in a submucosal plane and multifocal occurrences (“field cancerization”) are common. Identifying the best sites for biopsy can be difficult and many patients undergo multiple procedures for biopsy before a diagnosis is confirmed. Any method that enhances the detection of submucosal tumor will improve biopsy site selection and diagnostic accuracy.
Optical coherence tomography (OCT)12 is a near-infrared interferometry technique. It has excellent resolution and can image on scales ranging from 10 μm to approximately 10 mm. It is used widely in ophthalmology for retinal imaging. There are also reports of OCT imaging of the larynx and upper airway to measure airway patency. The main disadvantage is cost (approximately $75,000 for a retinal imaging system). OCT imaging is deployed with fiberoptic technology and the same endoscopic sheath delivery system disclosed for use with Raman spectroscopy herein can be used for OCT.
Narrow-band imaging (NBI)13 uses blue (415 nm) and green (540 nm) light filtering to enhance the visualization of hemoglobin. It is useful in identifying lesions with enhanced microvascular patterns including neoplasms. Findings are nonspecific but the technique has shown some promise in characterizing neo-angiogenesis in precancerous and cancerous aerodigestive lesions. It is inexpensive and easily deployed in any endoscope. It has proven cost effective in screening endoscopy for colon polyps but success in identifying upper airway lesions has been limited.
Embodiments described herein comprise a 785 nm laser light source, a high-resolution CCD camera, a spectrometer, two filters and a specifically designed fiber with lens. The miniaturized probe will be adapted to a sheath for deployment over the shaft of a flexible or rigid laryngoscope. Various materials for the sheath design can be useful. Constraints on the probe size should be less severe than those encountered in other designs where the probe had to be deployed through an operating channel of the scope. During operation, the camera will image the target tissue, the spectrometer will identify the Raman features and the results will be analyzed and displayed in real time on a computer system.
To determine if there are any histologic effects of the incident laser light on tissue, we performed Raman spectrographic analysis on fresh tonsil tissue. With IRB approval we studied the tonsils of 8 patients undergoing tonsillectomy for obstructive sleep apnea with tonsillar enlargement. A total of 61 specimens were subjected to a range of laser (785 nm) powers and exposure times. Total energy exposure of the tissue samples ranged from 0 (controls) to 4.6 Joules (230 milliwatts×20 seconds exposure). The pathologist, who was blinded to the exposure settings, graded each specimen histologically for thermal change. Findings were graded from 0 (no fields with histologic change) to 5 (100% of fields with thermal change). Using logistic regression, we determined that there was no significant tissue alteration compared to controls at any total laser energy level (p=1.6×10-6). The results demonstrate that adequate energy can be delivered to detect Raman scattering in tissue samples without sample degradation.
We have obtained IRB approval to begin an ex-vivo study of human laryngeal carcinomas that are resected endoscopically. The study will be completed at Our Lady of the Lake Hospital in Baton Rouge. After establishing spectral comparisons between documented malignant and non-malignant margins, we will complete a double-blind study comparing Raman spectrometry predictions to the histologically determined margin status on ex-vivo specimens. Without wishing to be bound by theory, the device disclosed herein can be utilized for intraoperative margin detection during endoscopic resection of upper aerodigestive cancers.
We completed our first Raman examination of a laryngeal resection specimen. Superficial (grossly positive) and deep (grossly negative) margins were evaluated in a patient with a history of laryngeal carcinoma-in-situ.
We have designed and prototyped an inexpensive sheath-based delivery system for single fiber Raman spectrometry suitable for upper airway endoscopy. We are also designing a handheld probe for fiber deployment during suspension laryngoscopy that will improve resolution by eliminating signal degradation along the fiber.
We are working to design a system that can deliver the incident laser light and detect the Raman system in the absence of interfering ambient light. For an in-vivo system we can accomplish this by using the delivery probe to electronically turn off the endoscopic light during the brief period of spectrum measurement. For the ex-vivo carcinoma margins study we are using an enclosure to eliminate ambient light.
Endoscopic resections will be performed and the specimen orientation will be tracked (
In the operating room, the specimen will be placed on a histology cassette with the orientation noted (
The cassette will be taken to a workspace just outside of the OR where the engineering team will be waiting to complete the study on the specimen. This is a large room with plenty of space for equipment and personnel. The engineering team can setup in this area without observing OR protocol; this is a non-sterile area without patient contact. The orientation of the specimen must be maintained whenever the cassette is opened. The specimen will have a unique patient identifier.
During the Raman study, the clinical group will explain the specimen orientation and margins. We should perform Raman spectroscopy on the four peripheral margins, the deep margin and the central tumor. The spectra should be labeled with the patient identifier appended with the name of the margin on the cassette for peripheral margins or with “deep” or “tumor” depending on the specimen location. After Ramen spectroscopy analysis, the specimen cassette will be closed, transferred to the specimen processing room, and processed according to hospital/pathology protocol.
Each case takes approximately 1-2 hours to complete the resection. The Raman processing will take approximately 15 min.
Laryngeal carcinoma is the second most common cancer of the respiratory tract with an overall 5-year survival rate of 60%. Clinical stage at the time of diagnosis and margin status during surgical resection are the two most important factors correlated with disease prognosis. Five-year survival drops from approximately 90% for tumors confined to the larynx, to 60% and 27% for regional and distant disease, respectively. Early diagnosis depends upon careful assessment of suspicious laryngeal lesions noted on diagnostic laryngoscopy. Confirmation requires biopsy and histologic evaluation. Laryngeal tissue sampling is complicated by the need for endoscopic instrument placement in the airway and the heterogeneous appearance of premalignant and malignant lesions. Margin detection during surgical resection of laryngeal carcinoma also requires histologic assessment that is highly dependent on surgeon selected sample sites and the tendency for submucosal tumor extension. Nationally, the incidence of positive margins following endoscopic resection of laryngeal carcinoma is 22%.
An accurate method of optical biopsy during laryngoscopy would enable real-time classification of laryngeal lesions. Disclosed herein is a Raman spectroscopy imaging system that will mitigate the problem of delayed diagnosis and high incidence of positive margins following tumor resection. We will build a Raman spectrometer optical system and probe designed for laryngoscopy. Data from our laboratory and clinical studies show that Raman spectroscopic imaging of laryngeal pathology is safe and can distinguish between normal and cancerous tissue. This project will increase our understanding of the spectral characteristics of laryngeal cancer, extend lesion differentiation to include mucosal dysplasia and carcinoma in situ, and facilitate rapid cancer detection using an endoscopic imaging approach. This research will reduce morbidity and improved survival rates in laryngeal cancer. Principles and methods derived from the study will also be applicable to other forms of head and neck cancer.
Laryngeal carcinoma is one of the most devastating forms of cancer, even in patients who are ultimately cured of the disease. Failure to efficiently detect laryngeal carcinoma leads to diagnostic delays, incomplete tumor resection during surgical management and poor outcomes. This project will solve problems related to delayed diagnosis and inadequate surgical removal of laryngeal carcinoma using an innovative optical biopsy technique based on spectroscopic imaging.
Early diagnosis and adequate resection with clear tumor margins are crucial for local control and disease-free survival following the treatment of laryngeal cancer. Visual estimation of surgical margins is subject to errors related to heterogeneous tumor appearance and submucosal extension. Frozen section analysis only determines the margin status at discrete points along the lesion. There is a need for early diagnosis and improved intraoperative margin detection for carcinoma of the larynx.
To improve early diagnosis and tumor margin determination in laryngeal carcinoma, disclosed herein is a Raman spectroscopy optical system for laryngoscopy. This approach is based on published literature and results from our laboratory that show Raman spectral differences between tumor and non-cancerous tissue. Without wishing to be bound by theory, real-time detection of normal tissue, dysplasia, carcinoma in situ and invasive carcinoma will substantially lower the incidence of delayed diagnosis and incomplete tumor resection. The benefit will be lower rates of local and regional recurrence, reduced need for postoperative radiation therapy and fewer organ sacrificing procedures for advanced disease.
Develop spectroscopy hardware optimized for the detection of laryngeal carcinoma.
We will construct a Raman system for laryngeal cancer detection by optimizing its optical configuration. Using current designs, we will also construct a laryngoscopy probe to detect tumor tissue and map surgical margins. The outcome will be a system capable of rapidly capturing Raman inelastic scattering across frequencies useful for in vivo use during laryngoscopy.
Develop machine-learning-assisted signal processing algorithms for Raman spectroscopic cancer detection.
Machine learning signal processing software can be utilized to classify Raman spectral features. This includes new multi-modality neural networks and a murine-human hybrid, interspecies transfer learning model. The outcome will be Raman specific algorithms to classify normal tissue, dysplasia, carcinoma in situ and invasive carcinoma in the human larynx.
Perform initial system optimization and train a baseline neural network for transfer learning using mice.
We will validate the system on a murine cancer model by analyzing spectral differences between normal tissues and murine head and neck cancer xenografts. We will fine tune system parameters for in vivo testing in humans and accumulate data for neural network training in a transfer learning model.
Evaluate the Raman imaging system in patients with laryngeal carcinoma undergoing laryngoscopy.
We will evaluate the system in patients diagnosed with stage I/II laryngeal carcinoma. In this clinical study, we will compare tumor classifications performed by our Raman spectroscopic imaging system to their histologic findings. We will prepare the system for clinical trials and commercialization.
The study will establish innovative new technology to overcome the current limitations of tumor identification and margin control in head and neck cancer. It will also provide new insights into current approaches to the diagnosis and surgical management of laryngeal carcinoma. Without wishing to be bound by theory, the impact will be decreased morbidity and improved survival rates in laryngeal cancer.
Tumor identification and surgical margin control in endoscopic laryngeal carcinoma resection.
The past 20 years have seen a trend towards more conservative, larynx sparing surgery for stage I and II laryngeal carcinoma performed through a trans-oral endoscopic approach1,2. For optimal management, the surgeon must accurately distinguish between normal mucosa, dysplasia, carcinoma in situ and invasive carcinoma. Surgical margin status is the most important prognostic factor in laryngeal cancer managed with primary surgery3.
In 2019, a national cancer database survey of early laryngeal carcinoma treated with trans-oral laser resection showed an alarming 22% incidence of positive margins4. In 2017 a review of outcomes related to positive margins in endoscopic laser resection of early laryngeal cancer showed that positive or close margins reduced 5-year survival in laryngeal cancer from 85.1% to 61.3%5. There was a significant increase in the recurrence rate and need for postoperative radiation therapy and revision surgery. The system disclosed herein will allow the surgeon to distinguish invasive carcinoma from other mucosal changes and map accurate tumor margins in real time during endoscopic laryngeal carcinoma resection.
Raman spectroscopy (RS) detects the small fraction of inelastic scattering of photons from an incident monochromatic light source6. A RS system integrates a laser light source for sample excitation, a charge-coupled device sensor, an optical analysis module and a fiberoptic transmission channel7. It provides a highly specific, structural fingerprint of chemical samples.
A number of chemical species of biomedical interest produce Raman scattering, including proteins and nucleic acids. The Raman spectra of tissue specimens represent the weighted sum of their macromolecular species and are highly tissue specific. Researchers have identified cellular differentiation in several epithelial types using Raman spectroscopy. Prior analyses of laryngeal lesions have been ex vivo studies8,9,10,11. Tissues studied include normal laryngeal mucosa, dysplasia, carcinoma and squamous papilloma. When correlated with histologic results, reported lesion detection sensitivities for carcinoma ranged from 69 to 92%. Specificities ranged from 90 to 94%. Other ex vivo studies have been reported in brain12, bladder13, breast14 and skin15 neoplasms. Due to the technical constraints of current RS systems, in vivo studies are limited. Preliminary reports of in vivo RS have documented its ability to detect cancer in gastrointestinal, breast, brain, skin and cervical tumors16.
There are significant challenges to in vivo RS implementation16. Most RS systems are deployed in the laboratory and, until recently, these devices have occupied large, non-portable footprints. The quantum yield of RS scattering is weak, representing approximately 1 in 107 incident photons. Biomolecules are prone to fluorescence upon monochromatic illumination with a photon yield in 1-10% range17,18, resulting in significant signal-to-noise ratio reduction. RS system designers often compensate for fluorescent spectral interference with increased laser energy, longer exposure times, mechanical sample stabilizers and thermoelectric cooling systems19. These signal enhancement methods will require careful engineering before they can be applied in an in vivo setting. Streamlined designs for clinical application will need to maintain laser light source power, optical alignment, an intact transmission pathway and instrument sensitivity.
Microscopic evaluation of hematoxylin-eosin stained biopsy samples remains the standard for tumor diagnosis and margin detection in head and neck cancer. However, histopathologic analysis relies on a subjective assessment of tissue morphology and is subject to sampling error. Many lesions spread in a submucosal plane and multifocal occurrences are common. Identifying the best sites for biopsy can be difficult and many patients undergo multiple biopsy procedures before a diagnosis is confirmed.
Our goal is to design and implement a RS system capable of rapidly distinguishing cancerous from non-cancerous tissue during airway endoscopy. Ultimately, Raman spectroscopic characterization of normal, precancerous and cancerous tissue at a molecular level can be more relevant than routine histopathology.
Current impediments to routine clinical use of Raman spectroscopy include limited anatomical access, the design constraints listed above, and the need to control the lighting environment in the region of interest. Unlike other approaches to biomedical Raman spectroscopy, we are designing a system to specifically address these issues. Upon successful completion of our specific aims, we will be prepared to begin clinical trials of a Raman spectroscopy system that can perform non-invasive optical biopsy of suspicious laryngeal lesions and map surgical margins during endoscopic resection of laryngeal carcinoma.
Raman spectroscopy can be useful to identify cancer by detecting tissue changes at the molecular level. However, there are limited reports of RS used to detect cancer and many of the existing studies cannot be replicated8,9,20,21,22. This can be attributed to the complicated implementations of spectrometer hardware, delivery systems that cannot be adapted to clinical use and a limited number of analysis algorithms16,23,24. This project will advance the use of RS for the detection of laryngeal carcinoma. Knowledge and techniques gained from this disclosure are also applicable in other clinical applications of RS.
In vivo spectrometer deployment will require portability and efficient power consumption while maintaining instrument sensitivity. Also, the total laser energy is limited by potential tissue alterations resulting in damage to normal structures and interference with histologic evaluation of tissue samples25.
We are solving the in vivo spectrometer deployment issues with innovative optimizations of the optical parameters and sensor design. We have successfully used a portable Raman spectrometer prototype that can be moved to the operating room. Initial studies of ex vivo laryngeal cancer resection specimens have shown differences in the spectra of normal tissue and carcinoma using incident laser wavelengths and power settings suitable for in vivo use (see data described herein).
Standard probes used for Raman spectroscopy in the laboratory for chemical analysis are not suitable for use in human tissue studies16,23,24. Important parameters include designs for endoscopic placement, sterilization, and precise probe positioning in the field of study. RS studies performed in the laboratory eliminate the effects of ambient light on recordings by darkening the environment. The incident laser angle is precisely controlled by mechanical fixation of the sample and probe. While effective for laboratory use, these designs cannot be applied in a clinical setting.
We have designed an instrument specifically for in vivo use of RS during laryngoscopy. The probe includes an innovative design of the instrument tip to allow precise contact with the tissue site and control of the incident laser angle. It also eliminates ambient lighting effects. An electronic control system triggers the Raman analysis while simultaneously switching off the fiberoptic illumination. The design is intended for the real-time detection of laryngeal mucosal tissue changes and surgical margin mapping during the endoscopic excision of laryngeal carcinoma.
Machine learning assisted analysis of Raman signals to differentiate normal mucosa, dysplasia, carcinoma in situ and carcinoma of the larynx.
Investigators analyze Raman spectra using standard signal processing techniques based on one or two spectral peaks. Although there are studies showing single peak differences in signals from normal and cancerous tissue, there does not appear to be sufficient sensitivity for clinical application.
Our group discloses Raman specific machine learning algorithms26,27 to detect multiple spectral features and differentiate normal from cancerous/precancerous tissue. Inspection of our ex vivo laryngectomy data indicate that there is sufficient information to train and deploy a convolutional neural network for autonomous classification of these lesions.
We are exploring deep learning technology using multimodality networks that combine spectral data with endoscopic imaging. We have also designed an innovative interspecies transfer learning model where the base network is trained on a large murine Raman spectrum dataset and fine tuned to a limited set of human spectra.
Our overall strategy starts with our existing experience with clinical Raman spectroscopy. We will build and optimize the in vivo Raman system using the knowledge gained during the build of our current device. Data acquired during the initial phases of the project will enable us to create advanced machine learning algorithms capable of real time classification during laryngoscopy. After validating the overall system design we can proceed with clinical testing without trials to compare in vivo Raman spectroscopy tissue classification with histologic findings in patients undergoing endoscopic surgery for laryngeal carcinoma.
Prior to using the 785 nm Raman laser to analyze laryngeal specimens, we completed a controlled study of its histologic effects on fresh tonsillar tissue over a range of incident laser power (0-230 Mw) and exposure times (1-20 sec). The presence or absence of thermal tissue damage was assessed on histologic sections. Logistic regression analysis confirmed no histologic changes related to Raman laser irradiation at energies used to analyze human tissues.
In an IRB approved study of ex vivo laryngeal carcinoma resection specimens, we evaluated tumor and non-cancerous margins using a 785 nm Raman imaging system. A single fiber system was deployed in an enclosed stage for specimen stabilization and ambient light exclusion. The Raman spectroscopy findings were compared to the reported pathology. Differences in Raman shift between cancerous and non-cancerous sites were most pronounced in the 939 and 1438 nm peaks. These correspond to increases in protein C—C and lipid CH2-CH3 molecular bonds, respectively 24.
We will improve the current design of our Raman spectrometer by optimizing the optical parameters and components. Optimizations will improve tissue detection sensitivity and enable in vivo use.
All reported Raman systems for cancer detection use incident lasers in the near-infrared region I (NIR I, 700-1000 nm) to reduce the tissue auto-fluorescence prevalent in the 300-726 nm region28,29,30. However, Raman signal intensity is proportional to 2-4 and the spatial resolution is proportional to 2. The Raman signal intensity excited by a 785 nm laser is only 21.1% of that excited by a 532 nm laser at the same power. We will evaluate designs with different wavelengths, from UV (244 nm, 257 nm) to visual (532 nm, 633 nm) to NIR II (1064 nm)30,31. UV Raman spectroscopy can provide better Raman signal intensity and auto-fluorescence suppression18. NIR II Raman spectroscopy provides better auto-fluorescence suppression than NIR I32, but needs a indium gallium arsenide (InGaAs) spectrometer. InGaAs technology is available in our laboratory and our group has experience in NIR I and II imaging33,34.
To explore the laser wavelengths, we will use chemical systems. These include a large Renishaw in Via Reflex Raman confocal microscope operating with laser wavelengths of 532, 633 and 785 nm. We will use biological waste tissues for examination under various excitation wavelengths to compare the imaging performance, including the Raman signal-to-background ratio (SBR). Although the equipment is not suitable for in vivo tissue diagnosis, the test results will enable us to screen wavelengths that are useful for our portable system.
Appropriate filters can be important in separating the Raman signals from the Rayleigh signals (elastic scattering). We will use a high-optical-density (OD) band-pass filter for the excitation signal and a high-OD long-pass filter to prevent laser excitation light from reaching the sensor. Lasers with different wavelengths will be coupled with different optical filters to improve the detection sensitivity. A silicon-based spectrometer will be used for UV, visible and NIR-I Raman spectroscopic imaging; an InGaAs based spectrometer will be used for NIR II studies. The laser power will be tailored to achieve optimal Raman signals without thermal tissue damage.
A visible spectrum illumination beam will help to determine the optimal distance and angle for the incident laser. We will calibrate the relationship of beam size and shape to the distance and angle between the sample and Raman probe tip. The calibration data will enable us to calculate the optimal laser incidence distance (ranging in 0-10 mm) to achieve the best Raman SBR for the tissue detection. For clinical applications, we will need to optimize system design to enable exposure times less than 5 seconds.
Using our CAD models for Raman fiber deployment, we will refine and construct an instrument that providers can use in the larynx to detect tumor and map surgical margins. The design allows for the delivery of the Raman probe including connectors for the spectrometer; switchable fiberoptic lighting of the surgical field; and an actuator to turn off the visible light source, activate the incident Raman laser and record the spectrum. Our objective is to create a prototype suitable for clinical testing.
An exemplary surgical probe for direct laryngoscopy and carcinoma margin detection is shown in
The probe assembly and controller housing will be custom manufactured. The stainless-steel probe tubing, control unit electronics, fiberoptic cable and LED components are commercially available. Design parameters, including the fiber cable diameter, housing materials, and tip offset distance, can be manipulated based on the standard of care, the needs of the patient, or the preference of the clinician. Embodiments can be produced using 3D printed models, injection molding manufacturing, or a combination thereof.
Machine-learning-assisted signal processing software can be utilized to classify the Raman spectral features and differentiate cancerous, precancerous and normal laryngeal mucosal tissues. A range of machine learning (ML) algorithms can be employed, including deep neural networks (DNNs).
Deep neural networks have become the focus of recent machine ML research and have shown improved accuracy across a range of classification tasks. This research will develop convolutional neural networks (CNNs) to classify the tissues. Different CNN architectures will be evaluated for optimal classification. As is often the case in medical applications, limited training data will pose the main challenge in the development of our Raman spectral tissue classifier35. Without being bound by theory, to alleviate the need for data, three strategies can be employed to enhance training, reduce overfitting and improve generalization—regularization, transfer learning and multimodality networks.
Regularization has been shown to improve ML model performance36. For deep neural networks, in addition to parameter regularizations, we can regularize a neural network through pre-training or co-training. One of our research group members has developed co-trained deep neural networks using auxiliary generative networks. These are pre-trained or concurrently trained, multitasked systems that are combined with the target network. The two neural networks share common convolutional layers. By regularizing the target classifier with the generative auxiliary network, we have improved classification accuracy with reduced overfitting. In previous work, we have used this strategy for cell type classification using one-dimensional DNA sequence information and for mammogram image classification with two-dimensional data and a generative adversarial network (GAN)37,38. We will extend these designs to the spectral-based tissue classification and explore different auxiliary networks and tasks to reduce overfitting and enhance network accuracy.
Transfer learning is a common practice in DNNs designed for medical image analysis39. A neural network that was previously trained on non-specialized images is fine-tuned to the specific application domain. For the one-dimensional spectral data, it can be difficult to find an existing, pre-trained model. Our research will focus on a different transfer learning paradigm. We will train a neural network model using a large mice dataset and then fine-tune the model using human data. Without wishing to be bound by theory, this cross-species transfer learning model to outperform models trained directly from human tissue spectra and transfer learning models based on common environmental images (e.g. ImageNet).
A multi-modality DNN integrates inputs of different types and classifies based on the joint information. For our classification task, in addition to the spectral signal, digital photographic images are also available. Without being bound by theory, neural networks can be developed that process the two different types of information jointly and perform classification based on the combined features. Members of our research group have developed multi-modality neural networks for recommender systems40 and CNN classification systems for endoscopic images41.
Software for data processing and neural network training can be developed. Without being bound by theory, the software can be implemented in Python and use TensorFlow or other deep learning frameworks for neural network computation. Once a ML model with satisfactory sensitivity and specificity is trained, the model will be deployed for real time processing of Raman spectral data. Without wishing to be bound by theory, a computational system capable of distinguishing normal, dysplasia, carcinoma in situ and invasive carcinoma using Raman spectra obtained during laryngoscopy will be developed.
We can study the Raman imaging system disclosed herein using a head and neck cancer mouse model that has been developed by our group. Fifty-five athymic mice (Nu/J) will be implanted with human squamous cell carcinoma cells (FaDu)42. The tumors and surrounding normal tissues will be imaged. During this study we will perform initial optimization of the system optical parameters and collect data for the neural network classifier. The full description, including protocol, is included in the Vertebrate Animals section.
Evaluate the Raman Imaging System in Patients with Laryngeal Carcinoma Undergoing Laryngoscopy.
After development of the in vivo Raman spectroscopy system for laryngoscopy, we can pursue an IRB approved, in vivo study of the system without clinical trials. A series of 30 patients with stage I or II laryngeal carcinoma will be enrolled with consent. We will evaluate the diagnostic and surgical margin detection accuracy during laryngoscopy. Diagnosis and margin status determined spectroscopically will be compared to the histologic findings. This will be an internally controlled, single cohort study with no influences on patient diagnosis or surgical margin selection. The full protocol is described in the Human Subjects section.
Multiclass observations from the diagnostic study (normal, dysplasia, carcinoma in situ, carcinoma) and binary classification (negative or positive) in the case of the surgical margin study will be summarized in their respective confusion matrices. Observations will be reported in terms of specificity, sensitivity, precision and overall performance (accuracy and F score).
Our long-term goal is a clinical trial of a Raman spectroscopy system tuned for head and neck cancer evaluation. Following successful clinical testing of the laryngeal system, we plan to design additional instruments and algorithms for cancer detection. This includes designs for the early detection of oral cavity, pharyngeal and laryngeal cancers in the clinic.
Early detection of laryngeal cancer significantly increases survival rates, permits more conservative larynx sparing treatments, and reduces healthcare costs. Laryngoscopy is the current standard for the initial detection and surveillance of laryngeal cancer but relies on visual evaluation for interpretation. A non-invasive optical form of biopsy for laryngeal carcinoma can improve early detection, allow more accurate surveillance for recurrence, and improve intraoperative margin control.
We have designed and evaluated a Raman spectroscopy system for rapid intraoperative detection of human laryngeal carcinoma. Our spectral analysis methods include principal component analysis (PCA), random forest (RF), and one-dimensional convolutional neural network (CNN) methods.
In an IRB approved protocol, we measured the Raman spectra from 207 normal and 500 tumor sites collected from 10 laryngeal cancer surgical specimens (2 total laryngectomies and 8 endoscopic laryngeal cancer resections). RF analysis averaged an overall accuracy of 90.5%, a sensitivity of 88.2%, and a specificity of 92.8% in 10 trials. The 1D CNN can be performed with a 96.1% average accuracy, 95.2% sensitivity, and 96.9% specificity in 50 trials. In predicting the first three principal components (PCs) of normal and tumor data, both RF and CNN can perform well except for the tumor PC2.
This is the first study using CNN-assisted Raman spectroscopy to identify human laryngeal cancer tissues. While convolutional neural networks can be successful in various clinical applications, human cancer classification has been challenging. Our Raman spectroscopy feature extraction approach has not been reported for human cancer diagnosis. Raman spectroscopy, assisted by machine learning (ML) methods, can have application as an intraoperative, non-invasive tool for the rapid diagnosis of laryngeal cancer, including margin detection.
The American Cancer Society estimates that there are 13,000 new cases of laryngeal cancer diagnosed each year [1], [21]. Approximately 3700 patients with laryngeal cancer die annually. Early diagnosis and precise control of disease margins are essential for the surgical management of laryngeal carcinoma. important visual features related to malignancy can be missed on routine laryngoscopy and multiple biopsies can be necessary for diagnosis.
Raman spectroscopy exploits the inelastic scattering of incident monochromatic light exhibited by many substances. Complex biological molecules, including proteins, nucleic acids and lipids have distinct Raman spectral signatures that have been well characterized in the laboratory [3]. Recent developments in Raman spectroscopy technology have enabled investigators to detect variations in the structure and concentration of biomolecules in tissues, including biochemical markers associated with neoplasia [4].
Principal component analysis (PCA) is a dimension reduction algorithm for data processing. PCA is a useful technique for increasing the interpretability of large datasets while minimizing information loss. Previously, PCA for Raman spectra analysis was used to reduce complex, multipeak spectra to 2 or 3 principal components. Supervised machine learning algorithms, with or without PCA, can be used for automating Raman spectra interpretation.
Random forests are a type of supervised machine learning based on decision tree generation for solving classification or regression problems. The algorithm trains multiple decision trees to predict labeled outcomes. During classification, the model traverses all the decision trees and outputs the class reached by most trees. In 2009 Teh, et al. evaluated RF analysis of laryngeal tissue Raman spectra and reported a diagnostic sensitivity of 88.0% and specificity of 91.4% for laryngeal malignancy identification [5].
Neural networks are a form of AI comprising multiple units (neurons) arranged in layers that calculate the relationship between their inputs and outputs based on a set of parameters. Neural networks are a focus of research on machine learning and artificial intelligence [6]. Convolutional neural networks extend the neural network concept with the addition of specialized layers for pattern recognition [7, 8]. Two-dimensional CNNs have been studied in computer vision and image processing research. Similarly, 1D CNNs can be implemented to interpret spectral data. In 2019 Dong, et al. used 1D CNN analysis of Raman spectra to discriminate human from animal blood [9]. There are no reports of CNN enabled Raman spectroscopy used to classify tissue specimens.
We combined PCA with RF and CNN to automate the interpretation of Raman spectra of ex-vivo laryngeal cancer resection specimens. We evaluated this approach to distinguishing laryngeal carcinoma from cancer free margins.
We designed a Raman spectroscopy system comprising a Raman probe (RPS785, InPhotonics, Inc) connected to a 2 mm optical fiber, a 785 nm diode laser source (Turnkey Raman Lasers-785 Series, Ocean Optics, Inc), and a Raman spectrometer with CCD detector (QE Pro; Ocean Optics, Inc). The system was interfaced with a laptop computer for data collection and assembled on a cart for portability. An enclosure was used to shield the specimen and probe during the ex-vivo tissue study. For the tissue evaluations following laryngeal cancer resection, the system was deployed in a non-sterile workspace adjacent to the operating room where the procedures were performed.
All patients consented to the study and all the procedures followed human ethical guidelines.
We collected Raman spectroscopic data from specimens taken from patients undergoing resections for histologically confirmed laryngeal carcinoma. Based on evaluations of the system on resected tissue, we determined the ideal incident laser parameters. The laser intensity was set to 160 mW and spectra were sampled with 3-second exposures. For each sample site, we recorded 7-10 spectra.
Ten laryngeal cancer specimens were studied, including two total laryngectomy (
For each specimen, the engineering team recorded multiple Raman spectra from the labeled margins and central tumor bed. Findings were compared to the final histopathology results. In all cases, the consulting pathologist was blinded to the spectroscopy findings.
The recorded Raman data were preprocessed prior to analysis [10], [11]. The signal processing protocol was coded using MATLAB (R2018a; MathWorks Inc, Natick, Mass) and included 1) autofluorescence removal with baseline subtraction with asymmetric least squares smoothing [12], 2) Savitzky-Golay filter smoothing and 3) signal normalization.
The mean Raman spectra were calculated from the labeled, preprocessed data for cancer and non-cancer tissue. Principal component analysis was used for dimensional reduction and coded in Python and sci-kit-learn [14]. The sample size was set to the number of components with the singular value decomposition (SVT) solver set to auto. The first three components were calculated (PC1, PC2, PC3) with specific peak marking. The RF and CNN models were also trained and tested with the PCA data to determine the predictive value of machine learning with the Raman PCA datasets.
Our random forest model was developed with Python and sci-kit-learn. The RF model architecture is shown in
For both models, we identified the Raman spectral features corresponding to cancer and non-cancer tissues by examining the trained model weights for individual peaks (RN) and wavenumber ranges (CNN) The features were ranked in order of decreasing magnitude.
A total of 500 Raman spectra were recorded from histologically confirmed cancer positive biopsy sites and 207 confirmed negative margins. Discrete spectra were recorded from 509 cm−1 to 3978 cm−1 across 440 wave numbers. For classifier training, we randomly selected 207 of the cancer spectra to create a balanced dataset totaling 414 recordings.
RF classification was assessed using 10-fold cross validation with a 90% training and 10% testing split of the combined dataset. The average accuracy was 90.5%. Sensitivity and specificity were 88.2% and 92.8% respectively. The RF-assisted receiver operating characteristic (ROC) curve is shown in
The averaged weights calculated for branches in the RF model decision trees corresponding to individual Raman peaks are shown in Table 1. The 24 highest weights are shown.
For CNN training, a 90%/10% random data allocation was used for 50 epochs/run. Convergence was noted at 30 epochs (
Overall accuracy was 96.1%. Sensitivity and specificity were, 95.2%, and 96.9%, respectively. The CNN-assisted ROC curve is shown in
Table 2 shows the top 26 features by weight in the CNN model, corresponding to spectral wavenumber ranges most important for the binary cancer classifier. The weights were extracted and summed from 32 mappings in the linear layer. The most significant range was between 3744 cm−1-3779 cm−1, consistent with the 3756 cm−1 and 3761 cm−1 peak weights in the RF model. The ranges of 921 cm−1-963 cm−1 and 1461 cm−1-1498 cm−1 in the fingerprint region, both had regression weights approximately equal to 1, consistent with existing empirical data [16], [17].
Single wavenumber importance was also calculated by averaging the range weight followed by normalization. For example, the average weight corresponding to the first five wavenumbers (w1, w2, w3, w4, and w5), averaged across the 32 convolution mappings, was 0.119, and the average weight corresponding to the second five wavenumbers (w3, w4, w5, w6, and w7, stride=2) was 0.038. Then the weight for the wavenumber w3 could be calculated as the mean of the two, (0.119+0.038)/2=0.0785. The averaged weights were normalized by dividing by the sum of the weights. Table 3 illustrates the 36 largest single wavenumber weights for the CNN model.
The PCA analysis is summarized in
Tissue classification with the RF model and PCA data showed 100% sensitivity when predicting normal tissue with all three principal components. Tumor prediction sensitivity was only 50% with PC1 and no tumor spectra were correctly predicted with PC2. Tumor prediction sensitivity was 100% using PC3.
Using the same dataset and the CNN model, prediction accuracy for normal tissue was 100%, 100% and 86% for PC1, PC2 and PC3, respectively. For tumor tissue accuracy was 100%, 0% and 100% for PC1, PC2 and PC3, respectively. PCA-ML sensitivity results are summarized in Table 4.
Early detection of laryngeal cancer can increase survival rates and permit more conservative, larynx sparing treatments. It also can also reduce the related length of hospital stay and health care costs [18]. The five-year relative survival rate for stage I glottic carcinoma is 90%. This drops to 74%, 56% and 44% for stages II, Ill and IV, respectively. Lack of locoregional control is the most significant cause of surgical failure.
The past 20 years have seen a trend towards more conservative, larynx sparing surgery for stage I and II laryngeal carcinoma performed through a trans-oral endoscopic approach [19]. Early-stage laryngeal carcinomas can be treatable with larynx sparing endoscopic surgery if adequate margins of resection are achieved. Persistent positive margins can require total laryngectomy and/or radiation therapy that can have otherwise been avoided with complete initial resection[20], [21]. Achieving adequate margins of resection can be challenging [22].
Laryngeal cancer is diagnosed histologically. Frozen section analysis is the current method for intraoperative diagnosis and margin assessment[23], [24]. Lesions can spread in a submucosal plane and multifocal occurrences (“field cancerization”). Identifying the sites for biopsy can be difficult and patients can undergo multiple procedures for biopsy before a diagnosis is confirmed [25]. Without wishing to be bound by theory, a non-invasive, optical form of biopsy for laryngeal carcinoma will improve early detection, permit more accurate surveillance for recurrence, and improve intraoperative margin control.
Optical coherence tomography is a near-infrared interferometry technique. It has resolution and can image on scales ranging from about 10 μm to about 10 mm. It can be used in ophthalmology for retinal imaging. There are also reports of OCT imaging of the larynx and upper airway to measure airway patency. A disadvantage is cost (approximately $75,000 for a retinal imaging system). OCT imaging is deployed with fiberoptic technology that can adapt to existing endoscopes.
NIR optical fluorescence spectroscopy incorporates a fluorescent dye, for example, indocyanine green (ICG) to visualize neoplastic vascularization[27]. In 2017, investigators reported real time NIR-ICG identification of head and neck mucosal lesions[28]. Positive findings correlated with histological malignance with a 89% overall accuracy.
Narrow-band imaging [29], uses blue (415 nm) and green (540 nm) light filtering to enhance the visualization of hemoglobin. It can be useful in identifying lesions with enhanced microvascular patterns including neoplasms. Findings are nonspecific but the technique has shown it can be useful in characterizing neo-angiogenesis in precancerous and cancerous aerodigestive lesions. It is inexpensive and can be deployed in any endoscope. It has proven cost effective in screening endoscopy for colon polyps but success in identifying head and neck lesions has been limited. Other less frequently use modalities for laryngeal tumor imaging include NIR visual imaging with tumor photosensitizing[31] and in-vivo microscopy [32].
When monochromic light from a laser source strikes a substance, photons are absorbed by its surface and reemitted. Most of the reemitted light occurs at the same frequency as the monochromatic source (elastic scattering). Depending on the sample, a portion of the reemitted light will radiate at frequencies above and below the incident frequency (inelastic scattering). Inelastic scattering is dependent upon the molecular structure of the specimen and is termed the Raman effect. One can determine the molecular structure of certain substances using their Raman spectra. Raman signals can be assigned to specific molecular chemical groups and chemical bond vibrational modes. Biomolecules with distinct Raman signals include proteins, nucleic acids, and lipids. Molecular interactions can also display distinctive spectral features[33].
A Raman spectroscopy system can comprise a laser light source, an optical filter to define the incident frequency and a charge couple device (CCD camera) to detect Raman scattering. Raman spectroscopy developers have incorporated several modifications to improve frequency resolution and acquisition times. For example, modular, portable systems have been developed for deployment in the field[34].
The Raman spectra of tissue specimens represent the weighted sum of their macro molecular species and can be tissue specific. Researchers have identified cellular differentiation in several epithelial types using Raman spectroscopy. In situ applications have been reported in brain [35], bladder [36], breast [37], colon [38], and skin neoplasms. Previous studies have also demonstrated a spectral difference between normal tissue, benign lesions, and carcinoma in ex-vivo laryngeal specimens[40], [41].
Lin, et al published a report of in vivo Raman spectroscopy for laryngeal lesions in 2016[42]. They documented spectral discrimination between normal and cancerous tissue during laryngoscopy. Their system used simultaneous 800-1800 cm−1 and 2800-3600 cm−1 wave number analysis. The Raman spectroscopy system included a near infrared diode laser source (785 nm) and a thermo-electric cooled CCD camera. They achieved a diagnostic accuracy of 91.1% (sensitivity—93.3%, specificity—90.1%) for laryngeal cancer. Acquisition times were on the order of milliseconds. They used a 1.8 mm (O.D.) fiberoptic probe with a 1 mm Sapphire ball lens at the distal tip. The Raman probe used during the study was deployed intraoperatively through the operating channel of a flexible laryngoscope. The type and dimensions of the laryngoscope were not specified in their report.
The advantages Raman spectroscopy over other potential tissue imaging modalities include distinct visible spectra for numerous substances, small sample size and nondestructive analysis. Disadvantages of Raman analysis have been its long signal acquisition time and signal-to-noise (SN) reduction due to tissue fluorescence[43]. Recent developments in Raman spectroscopy technology in the near infrared (NIR) spectrum have mitigated fluorescent interference, shortened signal acquisition times, and enabled biomedical applications.
Different components or chemical bonds can result in various peak intensities and wavenumbers of Raman signal. Our results show prominent laryngeal cancer peaks at 727 cm−1 (DNA based Adenine, bending) [44], and 1553 cm−1 (DNA based Guanine, bending). Non-cancer laryngeal tissue shows stronger Raman signals at 1125 cm−1 (proteins, C—N stretching), and 1655 cm−1 (amide I, C═O stretching). The findings are consistent with the known biochemical and cellular transformations in cancerous tissues with increased nucleic acids to protein/lipid ratios [33], [41].
Investigators have documented the importance of DNA glutathione (GSH) as a marker in human head and neck cancer [45], [46]. Glutathione (GSH) is identified in Raman spectra with wavenumbers which can be at ˜1365 cm−1, 1536 cm−1, and ˜1638 cm−1. These identifiers can play a role in cancer vs non-cancer tissue classification. Additional GSH related peaks are identified in the CNN model, including ˜921 cm−1, 932 cm−1, 953 cm−1, ˜963 cm−1, ˜1077 cm−1, ˜1413 cm−1, ˜1461 cm−1, and 1536 cm−1 [47]. The 1536 cm−1 peak is present in both the RF and CNN models.
Raman Spectral Feature Extraction and Classification with Machine Learning Models
Identifying Raman features for tissue classification can be challenging. Studies have designated specific peaks for binary cancer tissue classifiers [48], [49]; however, these designations cannot capture all the feature information available for classification. Diagnostic accuracy can be improved using machine learning trained classifiers capable of capturing a broad range of features. By examining the machine learning feature weights generated during model training[50], we can assign wavenumbers in the Raman spectrum to specific tissues of interest[51].
We evaluated the ability of RF and CNN model to classify cancer vs non-cancer tissue when trained on the original spectra and the first three principal components of the preprocessed data. The accuracy of principal component analysis-linear discriminant analysis has been documented with an overall accuracy of 84.3% when detecting colorectal cancer in ex-vivo samples [52]. Our data indicate improved accuracy using machine learning algorithms, even with a small dataset, compared to empirical peak assignments. In the case of binary classification of cancer vs non-cancer of the larynx, the CNN algorithm outperforms the RF in terms of specificity, sensitivity, and overall accuracy.
Early detection of malignant and premalignant lesions of the larynx can significantly improve larynx preservation and survival rates after surgical treatment. Without wishing to be bound by theory, we can develop a Raman spectroscopy optical biopsy system that can be used in the clinic or operating room. Without wishing to be bound by theory, a design that adapts the Raman probe to both flexible and rigid scopes will allow deployment in the clinic for early detection of premalignant and malignant vocal fold lesions and in the operating room for precise intraoperative margin control during resection. Herein, we describe the non-limiting conceptual design of an endoscopically deployed system and research for detecting laryngeal carcinoma using Raman spectrometry.
Rapid spectrographic classification of cancer vs non-cancer tissue during surgery for laryngeal carcinoma resection is possible. Raman spectroscopy integrated with machine learning can enable system designers to automate the classification process; the convolutional neural network model can be a ML approach. Principal component analysis provides insight into features in machine learning algorithms used to classify laryngeal cancer resection specimens. Without wishing to be bound by theory, we can rapidly diagnose and define the margins of laryngeal carcinoma.
Pancreatic cancer is the deadliest cancer type with a five-year survival rate of less than 9%. Detection of tumor margins plays a role in the success of surgical resection. However, histopathological assessment is time-consuming, expensive, and labor-intensive. We constructed a lab-designed, hand-held Raman spectroscopic system that can enable intraoperative tissue diagnosis using convolutional neural network (CNN) models to distinguish between cancerous and normal pancreatic tissue. To our best knowledge, this is the first reported effort to diagnose pancreatic cancer by CNN-aided spontaneous Raman scattering with a lab-developed system designed for intraoperative applications. Classification based on the original one-dimensional (1D) Raman, two-dimensional (2D) Raman images, and the first principal component (PC1) from the principal component analysis on the 2D image, can all achieve high performance: the testing sensitivity, specificity, and accuracy were over 95%, and the area under the curve approached 0.99. Although CNN models can show success in classification, it can be challenging to visualize the CNN features in these models, which has never been achieved in the Raman spectroscopy application in cancer diagnosis. By studying individual Raman regions and by extracting and visualizing CNN features from max-pooling layers, we identified Raman peaks that can aid in the classification of cancerous and noncancerous tissues. 2D Raman PC1 yielded more peaks for pancreatic cancer identification than that of 1D Raman, as the Raman intensity was amplified by 2D Raman PC1. To our best knowledge, the feature visualization was achieved for the first time in the field of CNN-aided spontaneous Raman spectroscopy for cancer diagnosis. Based on these CNN feature peaks and their frequency at specific wavenumbers, pancreatic cancerous tissue was found to contain more biochemical components related to the protein contents (particularly collagen), whereas normal pancreatic tissue was found to contain more lipids and nucleic acid (particularly deoxyribonucleic acid/ribonucleic acid). Overall, the CNN model in combination with Raman spectroscopy can serve as a useful tool for the extraction of features that can help differentiate pancreatic cancer from a normal pancreas.
Pancreatic cancer is the fourth most prominent cause of cancer deaths in the United States (US). Its five-year survival rate is ˜9% for all stages combined, and only 3% for patients in the advanced stage (Pandya, et al., 2008; Society, 2020). According to recent statistics published by the American Cancer Association, there were 57,600 new cases of pancreatic cancer diagnosed in 2020 alone and 47,050 deaths. Pancreatic cancer has the highest ratio of new death/new cases at 81.68% (Society, 2020). In clinical practice, surgical resection is the primary treatment approach for the removal of pancreatic cancer at an early stage. If surgery fails to resect cancer completely, the resection site is subject to local relapse. Rapid intraoperative differentiation of cancer and normal tissue plays an essential role in achieving complete resection of pancreatic cancer (Yang, et al., 2014).
To date, several approaches have been used for the detection of pancreatic cancer, such as computerized tomography, magnetic resonance imaging, and positron emission tomography. However, these techniques can be time-consuming, expensive, require bulky equipment, and are unsuitable for intraoperative tissue diagnosis with sufficient sensitivity and specificity. A method of tissue diagnosis is the postsurgical histological examination of tumor specimens (Yang, et al., 2014). However, this procedure can take several hours to days to acquire the final diagnostic report, and the accuracy of such an analysis can rely on the sample quality, experience of the pathologist, and medical procedures (Pandya, et al., 2008; Society, 2020).
Raman spectroscopy is a real-time diagnostic tool for analyzing chemical components with advantages including nondestructive examination, no sample preparation, and specificity to the chemical components (Boiret, Rutledge, Gorretta, Ginot, & Roger, 2014; Kourkoumelis, et al., 2015; Pence & Mahadevan-Jansen, 2016). Raman spectroscopy has been used in many fields, including chemistry, food, environmental science, and medicine (Boiret, et al., 2014; He, et al., 2018; Notingher, et al., 2004; Sato-Berru, et al., 2007; Xu, Gao, Han, & Zhao, 2017). Moreover, it has been used to diagnose cancers, such as oral, skin, and breast cancers (Gebrekidan, et al., 2018; Ghosh, et al., 2019; Kourkoumelis, et al., 2015; Manoharan, et al., 1998). To explore the features from the spectral data, some methods for the classification of Raman signals are principal component analysis (PCA), linear discriminant analysis, and support vector machines (SVMs) (Sohn, Lee, & Kim, 2020).
However, the identification of critical Raman features using the above-mentioned methods can be time-consuming and labor-intensive. For example, Raman spectra are processed by PCA in groups. This can result in low efficiency in data usage, and many features cannot be identified manually from other PCA components or scatter plots (Boyaci, et al., 2014; Pandya, et al., 2008; N. Stone, Kendall, Shepherd, Crow, & Barr, 2002; Uy & O'Neill, 2005). In comparison with the conventional methods (e.g., PCA), deep learning does not require manual intervention to extract Raman features, and has now been applied in many medical fields, such as electrocardiogramalysis and tumor segmentation in medical images (Al Rahhal, et al., 2016; Havaei, et al., 2017; Zhao, et al., 2018). The convolutional neural network (CNN) includes a deep learning model, and exhibits several advantages, including the requirement of little prior knowledge or design of explicit features, and a capability to capture inner structures (Fan, Ming, Zeng, Zhang, & Lu, 2019). However CNN models can be difficult to visualize Raman features (peak information) from CNN representations, even most of them can achieve performance in distinguishing different tissue types (Fukuhara, Fujiwara, Maruyama, & Itoh, 2019). The role of recognizing spectral features (particularly peak information) from CNN features is critical in the identification of biomolecular components (Fukuhara, et al., 2019). Our study represents the first effort in this field to extract critical CNN Raman features that aid in cancer diagnosis. In addition, to the best of our knowledge, this is also the first study that adopts CNN-aided spontaneous Raman spectroscopy for pancreatic cancer diagnosis.
Thus far, 1D Raman has been used directly as a dataset for CNN classification (Carvalho, et al., 2017; Lee, Lenferink, Otto, & Offerhaus, 2019; Shao, et al., 2020). However, it can be difficult to perform prefeature extraction from individual Raman data with the use of statistical algorithms such as PCA. Herein, we not only evaluate 1D Raman but also explore the efficiency of 2D Raman images obtained from the dot products of 1D Raman. We also extract and visualize the hidden CNN features from max-pooling layers.
In this study, the human CFPAC-1 cell line (ATCC® CRL-1918™, pancreatic ductal adenocarcinoma) was used. Before the injection of the cells in an animal host, tumor cells were grown in Iscove's Modified Dulbecco's Medium (ATCC® 30-2005™) with 10% fetal bovine serum (Neuromics, Edina, Minnesota) at 37° C. and 5% CO2 in a humidified environment. For the animal model, we used 6-8-week-old female immunocompetent athymic nude Nu/J mice (stock #002019, Jackson Laboratories, Bar Harbor, Maine, USA).
After the CFPAC-1 cells were incubated in media, approximately 2×106 cells were transplanted to the dorsa of the mice by subcutaneous injection. When the size of the tumor was approximately 1 cm, the mice were euthanized, and the entire tumor and normal pancreas were dissected. This study was approved by the Institutional Animal Care and Use Committee of Louisiana State University, and all operations followed the guidelines on animal research.
The Raman spectroscopy system comprised a 785 nm laser source (laser diode, Turnkey Raman Lasers-785 Series, Ocean Optics Inc., Dunedin, Flurida, United States), QE Pro spectrometer (Ocean Optics, Inc), and a Raman probe (RPS785, InPhotonics Inc., Norwood, Massachusetts, United States). When the Raman spectra were acquired, the Raman probe was fixed behind the specimen at a distance of approximately 5 mm. In this study, 20 mice were used. The entire tumor and normal pancreas were extracted from each mouse. Subsequently, 1,305 Raman spectra were collected from the tumor, and 1,224 Raman spectra were collected from the pancreas.
Raman Measurements, Data Processing, and Data Analysis with CNN Models
The measured raw Raman signals can include noise (e.g. from tissue autofluorescence), and the real Raman signals require preprocessing before feature extractions (Pandya, et al., 2008). The procedures of Raman spectral processing can be (1) autofluorescence removal by asymmetric truncated quadratic processing (Mazet, Carteret, Brie, Idier, & Humbert, 2005), (2) background removal, (3) noise removal with the Savitzky-Golay filter (Cordero, et al., 2017), and (4) normalization.
In addition to the original 1D Raman signal, 2D Raman images were also obtained from the dot product of each normalized Raman spectrum and its transpose, as depicted in
We explored two types of CNN models for the classification task: a 1D CNN model, which was used for 1D Raman signals and the 2D Raman PC1, and a 2D CNN model used for the 2D Raman images. The overall structures of these two types of CNN models were the same and were composed of four convolution layers (Conv), a dropout layer, a max-pooling layer, a full connection layer, and a softmax layer, as illustrated in
In the 2D Raman CNN model, the filter sizes were changed to [10, 10] for the Conv1 layers, and [5, 5] for the other three convolution layers. The configurations of the other layers were maintained similar to those of the 1D CNN model. All analyses were conducted with MATLAB (version R2019b, MathWorks Inc, Natick, MA, USA).
The typical 1D Raman spectrum obtained from the cancerous pancreatic tissue (
Evaluation of a CNN Model for Detection of Pancreatic Cancer with 1D and 2D Raman Images
To evaluate the efficiency of the CNN models in identifying cancerous and normal pancreatic tissues, 80% of the Raman data were used for training, 10% of the data were used for validation, and 10% of the data were used for testing. To train the CNN models based on 1D Raman, 2D Raman image, and 2D Raman PC1 (
We also gathered statistical measures of the CNN model performance (
Visualization of the CNN Hidden Features from the Max-Pooling Layer
Although CNN was successful in differentiating species by Raman signals, it can be challenging to identify the Raman features that help with this differentiation. A study was conducted to visualize the Raman features from max-pooling layers to differentiate between pharmaceutical compounds and numerically mixed amino acids. (Fukuhara, et al., 2019). Herein, we explored the feasibility of visualization of hidden Raman features in differentiating between cancerous and normal tissues from max-pooling layers.
The mean Raman spectra of the 1D Raman and 2D Raman PC1 of the cancerous and normal pancreatic tissues were loaded in their corresponding trained CNN classifiers; the strongest activation channels were then extracted from the max-pooling layers. Given that the sizes of the feature maps in each convolution layer were the same as those of the input features, the visualized CNN features were plotted with the Raman shifts, and some CNN Raman features were visualized from the max-pooling layers. In the 1D Raman CNN model (
The mean 2D Raman images of the cancerous and normal pancreatic tissues were also loaded in the trained 2D CNN classifier. The CNN features were then extracted from the max-pooling layer and reshaped to the original size of the input 2D Raman images. On the reconstructed 2D images, the R1 region was relatively brighter in pancreatic cancer; however, this area was darker for the normal pancreas (
From the plot of the normalized diagonal pixel values, it can be seen that the normal pancreas had a lower intensity than the cancerous pancreas when the Raman shift was lower than 1800 cm−1. This tendency was reversed when the Raman shift was beyond this value. In the wavenumber range of 600-1800 cm−1, the peaks at 1128, 1449, and 1720 cm−1 could be found on the diagonal pixel curve for the cancerous pancreas (red line: Tum_diagnol), and a 1010 cm−1 peak was found on that for the normal pancreas (black line: Pan_diagnol) (
Similar to
We also studied the frequency of Raman peaks of visualized CNN features that appeared across all the tested spectra. In 1D Raman, 623, 727, 821, 855, 1449, 1583, 1620, and 1640 cm−1 can appear for pancreatic cancer, whereas 720, 1100, 1258, 1482, and 1744 cm−1 can be found for the normal pancreas (
To examine the contribution of each wavelength region to the classification, we divided the full wavenumber range (600-3970 cm−1) into 215 individual subregions, each being 22 cm−1 wide. The subregions were then loaded individually into the 1D CNN model to differentiate between a pancreatic tumor and a normal pancreas.
Pancreatic cancer deaths are ranked fourth among all cancer deaths in the US with a low five-year survival rate (<9%). Thus far, there has been no reliable method to rapidly differentiate between cancerous and normal tissues (e.g., <1 min) (Pandya, et al., 2008; Society, 2020). Herein, we designed effective CNN models to differentiate between cancerous and normal pancreatic tissues based on Raman spectroscopy. To further utilize the hidden Raman signal features for the CNN models, we constructed 2D Raman images from the original 1D Raman spectra by the dot products of the Raman spectra and their transposes. Additionally, each 2D Raman image was processed by PCA to generate 2D Raman PC1 data. Compared with the original 1D Raman spectrum, 2D Raman PC1 can enlarge the difference between the maximum and minimum values of the signal. 1D Raman, 2D Raman images, and 2D Raman PC1 were then loaded into 1D or 2D CNN models to classify cancerous and normal pancreatic tissues.
All three methods can be used to classify cancerous and normal pancreatic tissues, where the training accuracies were over 98.8% and the training losses were smaller than 0.05. Compared with 1D Raman and 2D Raman PC1, the model that used 2D Raman images can acquire a higher training accuracy (close to 100%) and a lower training loss (less than 0.005). From the test dataset, it can be seen that all three methods yielded high-testing accuracy (>96%), sensitivity (>95%), specificity (>95%), and precision (>95%) in the identification of cancerous and noncancerous Raman spectra. In the three cases, AUC could reach up to 0.99. Thus, the CNN model with four convolutional layers has high efficiency in classifying cancerous and normal pancreatic tissues.
Currently, gross examination, intraoperative frozen section analysis (IFSA), and postsurgical histopathological examinations are the most common approaches used for the evaluation of pancreatic cancer (Ghosh, et al., 2019; Handgraaf, et al., 2014; Yang, et al., 2014; Zhou, et al., 2012). However, these conventional methods have drawbacks and are incompatible with the accurate intraoperative diagnosis because they are (1) time-consuming (e.g. postsurgical histopathological examinations>20-30 min), (2) subjective to histopathological translation, (3) dependent on tissue preparation (particularly in IFSA), (4) are associated with a limited number of biopsy points, and (5) associated with sampling bias and tissue loss owing to biopsies (Jaafar, 2006; Pandya, et al., 2008; Society, 2020; Vahini, Ramakrishna, Kaza, & Murthy, 2017). Compared with these traditional approaches, Raman signals can be dependent on the changes of chemical compositions (e.g., proteins or nucleic acids) in the biological samples. Raman peak positions and amplitudes contain biochemical information on tissue compositions (Sohn, et al., 2020). Therefore, identifying the Raman peaks or regions that help differentiate cancerous from noncancerous tissues can be crucial in the understanding of the chemical composition changes among various tissues. Many studies have reported that using a CNN model in combination with Raman spectra can serve as a rapid and nondestructive approach to classify chemical species (Fan, et al., 2019; Mazet, et al., 2005; Ralbovsky & Lednev, 2020).
However, visualization and interpretation of the features from the CNN model remains a challenge of this approach. In the field of CNN-aided Raman spectroscopy, there was only one recent report that used simple spectra, generated by the Lorentzian function and white noise, to aid in the feature visualization of amino acids and pharmaceutical compounds mixed with known ratios (Fukuhara, et al., 2019). Herein is the first report in this field to identify the Raman features that can assist in the classification of cancerous and noncancerous tissues. We extracted the maximum activation channels from max-pooling layers to visualize the critical features that are most relevant to the classification by CNN models. In addition, to the best of our knowledge, our effort is also the first CNN-aided spontaneous Raman spectroscopy study to provide intraoperative pancreatic cancer diagnosis.
Based on the accuracy of the screening regions (
In addition, the CNN features from the entire test dataset provided more feature visualization. 2D Raman PC1 yielded larger peak magnitudes in CNN features than those in 1D Raman. In the CNN features of the test dataset and their high-frequency peaks, the Raman peaks at 623, 727, 821, 855, 1128, 1177, 1449, 1555, 1583, 1620, 1640, and 1720 cm−1 were found in pancreatic cancer, whereas the peaks at 720, 1010, 1100, 1258, 1482, 1575, and 1744 cm−1 were found in the normal pancreas.
Given that Raman peaks are related to chemical components, the visualized CNN features listed above and the high-frequency peaks indicative of pancreatic cancer can be linked with the peaks of 623, 645, 727, 821, 855, 1128, 1177, 1243, 1449, 1555, 1583, 1620, 1640, and 1720 cm−1 that represent protein components, for example collagen contents (Chan, et al., 2006; Cheng, Liu, Liu, & Lin, 2005; Dawson, Rueda, Aparicio, & Caldas, 2013; Frank, McCreery, & Redd, 1995; Frank, Redd, Gansler, & McCreery, 1994; Lau, et al., 2003; Pandya, et al., 2008; Schulz & Baranska, 2007; Talari, 2015). The normal pancreas yields distinct peaks at 671, 720, 1010, 1100, 1258, 1482, 1575, and 1744 cm−1, indicating the components of lipids and nucleic acid (particularly DNA/RNA) (Notingher, et al., 2004; Pandya, et al., 2008; Nicholas Stone, Kendall, Smith, Crow, & Barr, 2004; Talari, 2015). Thus, this method can aid in the analysis of biomolecular tissue components through chemical analysis, and pancreatic cancerous tissue can be rapidly identified by its Raman features.
Herein, we designed CNN models that could classify cancerous and normal pancreatic tissue based on spontaneous Raman spectra, which was the first such effort in the application of pancreatic cancer diagnosis. The original 1D Raman and the two 2D Raman methods can achieve high performance: testing sensitivity, specificity, and accuracy were higher than 95%, and the AUC could be up to 0.99. Through the screening of increasing regions and individual subregions and the visualization of CNN features, the fingerprint regions (600-1800 cm−1) can be involved in the recognition of pancreatic cancer. With the CNN features extracted from max-pooling layers, we located Raman peaks that were involved in classification of cancerous and normal tissue. From these peaks, we can identify that the cancerous pancreatic tissue contained increased protein content, particularly collagen, whereas the normal pancreas contained more lipids and nucleic acid (particularly DNA/RNA). This was the first effort that visualized features in the field of CNN-aided Raman spectroscopy for cancer diagnosis. Overall, the CNN model, in combination with spontaneous Raman spectroscopy, can serve as a useful tool for the extraction of key features that can aid in the differentiation of pancreatic cancerous tissue from a normal pancreas.
Our work has demonstrated the feasibility of pancreatic cancer detection by CNN-assisted spontaneous Raman scattering and the possibility of important Raman feature visualization from CNN models for the first time in this field. The findings will be evaluated further in conjunction with human pancreatic cancer studies. Moreover, our CNN-aided spontaneous Raman spectroscopy, with our lab-designed Raman system, can become a rapid (<30 s), accurate tool for intraoperative tissue diagnosis in pancreatic cancer surgery.
To find the most significant regions that contribute to the accuracy of CNN models, 1D Raman and 2D Raman PC1 datasets were screened in forward (from 600 to 3970 cm−1, ˜16 cm−1/step) and backward (from 3970 to 600 cm−1, ˜ 16 cm−1/step) directions. If one region's range makes more contribution to the classification, the accuracy would increase, for example, at the beginning of the screening.
1D Raman had a rising region from 600 to 1800 cm−1 in the forward screening (
Those skilled in the art will recognize, or be able to ascertain, using no more than routine experimentation, numerous equivalents to the specific substances and procedures described herein. Such equivalents are considered to be within the scope of this invention, and are covered by the following claims.
This application claims priority from U.S. Provisional Application No. 63/089,322 filed on Oct. 8, 2020, and U.S. Provisional Application No. 63/234,062 filed on Aug. 17, 2021, the contents of each of which are incorporated herein by reference in their entireties.
| Filing Document | Filing Date | Country | Kind |
|---|---|---|---|
| PCT/US2021/054262 | 10/8/2021 | WO |
| Number | Date | Country | |
|---|---|---|---|
| 63089322 | Oct 2020 | US | |
| 63234062 | Aug 2021 | US |
