Patent Application
20160150963
1. Field of the Invention
The invention relates to an apparatus and method for functional imaging of tissue.
2. Related Art
Over the past few decades, our understanding of the properties of individual neurons and their role in brain computations has advanced significantly. However, we are still very far from understanding how large ensembles of neurons in the brain interact to process information. For monitoring neuronal activity, extracellular electrical recording provides unparalleled temporal resolution. It is not possible, though, to record electrically from specific cell types, and up-scaling recording density to track the activity of every neuron in an extended brain region appears infeasible. Functional imaging by free-space two-photon microscopy enables single-cell resolution of large neuron ensembles at anatomical densities and provides cell-type specificity of activity via genetically encoded fluorescent reporters. But it works ideally only with thin and transparent specimens. More generally, light scattering and absorption in tissue impose significant fundamental limits: in mammalian brains, accessible depths in vivo are restricted to superficial cortical regions, <1 mm. Endoscopic methods developed to circumvent such restrictions impart significant damage to tissue given the large probe diameter (0.3 to >1 mm).
More than a century ago, Ramon y Cajal speculated that the brain's varied and complex functions arise from two fundamental properties of neurons: their individual morphologies, and their connections to each other. A modern revision of these precepts underlies the current perspective on the cerebral cortex: first, different regions of the brain contain distinct, genetically specified neuronal cell types—and these cell types possess distinct and characteristic electrophysiological morphological properties (i.e. dendritic inputs, axonal outputs); and second, this variety of cell types seem to be arranged in stereotypical microcircuits that enable each brain area's local functions. As have neuroscientists since Cajal, it is presently believed that the key to understanding how the brain works is first to attain an understanding of how different neuron classes interact functionally, in vivo. This detailed knowledge is expected to elucidate how functional, i.e. microcircuit, interactions break down in disease. At present, the requisite tools to monitor complex brain circuits do not exist, however, and this has posed a universal and long-standing obstacle to such pursuits.
In some aspects, a new approach of integrated neurophotonics is provided. Integrated neurophotonics is a novel paradigm for functional optical imaging that surmounts the limits of present methods. It permits functional imaging with cellular resolution in highly scattering brain tissue, can offer complete coverage of all neurons within target volumes, and has eventual prospects for human applications. This approach is based on distributing a dense 3-D lattice of emitter and detector pixels within the brain itself, spaced by distances on the order or less than the optical attenuation length. These pixel arrays are embedded onto neurophotonic probes, realized as implantable, ultra narrow shanks that leverage recent advances in nanoprobe-based electrophysiology and integrated nanophotonics. Used with functional optical reporters, one 25-shank probe module is capable of recording activity from all neurons within a 1-mm3 volume (˜100,000 neurons). Further, this methodology is scalable; multiple modules can be tiled to densely cover extended regions deep within the brain. Accordingly, it can permit simultaneous recording from millions of neurons, at arbitrary positions and depths in the brain, to unveil dynamics of complete neural networks—with single-cell resolution and cell-type specificity. Ultra narrow neurophotonic probes can perturb brain tissue minimally, and can impose negligible tissue displacement and only minute local power dissipation. Importantly, the neurophotonic probes can be produced by existing methods of large-scale integration via wafer-scale foundry (factory) based technology. The probes will transform studies of circuit-level mechanisms of brain computation and neurological disorders, and accelerate drug discovery by high throughput screening in vivo.
In one aspect, a method for detecting functional cellular activity within a volume of a tissue is provided. The method includes, a) inserting a three-dimensional array of optical emitters and optical detectors into a volume of a tissue, the tissue volume including one or more cells labeled with an optical reporter of cellular activity, b) illuminating the one or more cells with photons from the optical emitters of the three-dimensional array to generate optical signals from the optical reporter that labels the one or more cells, and c) detecting the optical signals using the optical detectors of the three-dimensional array, wherein the illumination includes one-photon excitation of the optical reporter.
In embodiments of the method, a) the optical signals are fluorescent optical signals, b) the tissue is nervous tissue or living brain tissue, and each cell is labeled with an optical reporter of neural activity, c) the optical reporter is a genetically encoded fluorescent protein, a chemical fluorescent reporter, or a fluorescent nanoparticle reporter, or a combination thereof, d) the array includes elongated microsized shanks including the optical emitters and the optical detectors, each shank being 100 μm or less in width, e) each shank includes optical emitters and optical detectors, f) the shanks extend to any arbitrary location in the tissue, g) the optical emitters are time-gated, h) the optical emitters include optical elements for spatial profile control of the illumination, i) the optical detectors include optical filters, focusing elements, planar optical elements, or metamaterial-based optical elements, or any combination thereof, j) the detection elements includes both continuous or time-gated collection of the optical signals, k) the detection includes optical intensity sensing of the optical signals, or optical signal detection by avalanche current amplification of individual photon absorption events, or any combination thereof, l) the method further includes optical spike sorting of the detected optical signals, or m) any combination of a)-l).
In another aspect, a device for detecting functional cellular activity is provided. The device includes, elongated microsized shanks, each shank including one or more optical emitters and one or more optical detectors, wherein the shanks are sized in width and thickness to fit between adjacent neuronal cell bodies in a neural tissue, and the shanks are arranged to form a three-dimensional array of the optical emitters and the optical detectors.
In embodiments of the device, a) the shanks are about 100 μm or less in width, b) the shanks are about 1 mm or more in length, c) the array has a pitch that is on the order of, or less than, one optical attenuation length of a predetermined wavelength of light to be emitted from the optical emitters, d) the optical detectors include optical filters, or focusing elements, or any combination thereof, e) the emitters are time-gated, f) the optical detectors include optical intensity sensors or avalanche current amplification sensors, g) the device further includes a time-to-digital converter connected to the optical detectors
For a more complete understanding of the present invention, reference is now made to the following descriptions taken in conjunction with the accompanying drawings, in which:
The following are incorporated by reference herein: U.S. Provisional Patent Application Nos. 61/900,216, filed on Nov. 5, 2013, and 62/054,893, filed on Sep. 24, 2014, and U.S. patent application Ser. No. 13/627,755, filed on Sep. 26, 2012.
In a particular aspect, a method for detecting functional cellular activity within a volume of a tissue is provided. In the method, a three-dimensional array of optical emitters and optical detectors is inserted into the tissue volume. The array can be realized as a probe having elongated microsized shanks of an arbitrary length to reach any region within the tissue, with the shanks comprising optical emitters and/or optical detectors. In some embodiments, a shank can have a length of about 1 mm or more, about 2 mm or more, about 3 mm, about 4 mm or more, or about 5 mm or more. A shank can be sized to minimize damage to the tissue. For example, a shank can be sufficiently narrow so as to circumvent immune responses, scarring and gliosis after implantation into brain or other nervous tissue. In some embodiments, the width of the shank can be about 500 μm or less, about 400 μm or less, about 300 μm or less, about 200 μm or less, about 100 μm or less, about 50 μm or less, or about 25 μm or less, and the thickness of the shank can be about 100 μm or less, about 75 μm or less, about 50 μm or less, about 25 μm or less, or about 15 μm or less. Different shanks in an array can have different lengths, widths and/or thicknesses than other shanks in the array, and some or all of the shanks in an array can be similarly sized.
To form the array, the microsized shanks can be arranged to form a three-dimensional array of shanks. In some embodiments, the shanks can be ultra-thin and ultra-narrow shanks, giving the array of shanks a small total cross-sectional area (transverse to the length of the shanks) that minimizes the displacement of, and perturbation to, the tissue.
Examples of optical emitters include, but are not limited to, waveguide terminals, micro-ring resonators, photonic crystal resonators, microfabricated diffraction gratings, nano-fabricated pillars, or zone-plates, or any combination thereof.
Examples of time-gated semiconductor-based optical detectors include, but are not limited to, photon-counting detectors, such as single-photon avalanche photo-detectors (SPADs); or integrating detectors; with internal gain, such as avalanche photodiodes, or without, such as PIN photodiodes; or any combination thereof.
The pitch of the three-dimensional array of optical emitters and photo-detectors can be adjusted by changing either or both the shank-to-shank spacing, and the spacing of the optical emitter and detector elements upon each shank. The pitch can be chosen to scale with the optical attenuation length of the neural tissue at the wavelength(s) employed by the functional optical reporters. The choice of pitch permits the system's optical emitters and detectors to operate near or within the regime of ballistic photon propagation.
In some embodiments, the photodetector elements can include optical filters, focusing elements, planar optical elements, or metamaterial-based optical elements, or any combination thereof. For example, the photodetector elements can include spectral filters that enable optical signals from the functional optical reporters to be separated from other undesired sources of illumination (which can include endogenous tissue fluorescence by pigments absorption). Examples of spectral filtering components include, but are not limited to, resonant cavities, gratings, nanopillars or other nanostructures, or plasmonic absorption elements, or combinations thereof. In addition, the photodetector elements can include focusing elements such as microlens elements, which can enhance the collection of illumination emanating from functional optical reporters.
Cells in the tissue volume can be labeled with an optical reporter of cellular activity, including an optical reporter of functional neural activity. Examples of optical reporters, include but are not limited to: a) genetically encoded fluorescent proteins that report neural activity, including voltage indicators, chemical indicators such as calcium, pH, neuromodulator or neurotransmitter indicators, indicators sensitive to local forces, etc.; b) exogenous fluorescent activity reporters, for example, chemically-sensitive fluorescent molecules or nanoparticles, such as calcium-sensitive reporters like GCaMP; c) fluorescent voltage-sensitive dyes or nanoparticles, or other reporters of neural activity, including voltage indicators, chemical indicators such as calcium, pH, neuromodulator or neurotransmitter indicators, or indicators sensitive to local forces; d) or any combination thereof (see, for example, Molecular Probes-Production Information MP03010, Long-Wavelengh Calcium Indicators (2005); C. Grienberger, A. Konnerth, Imaging calcium in neurons. Neuron 73, 862-885 (2012); J. Akerboom, et al, Optimization of a GCaMP calcium indicator for neural activity imaging. Journal of Neuroscience 32, 13819-13840 (2012); all incorporated by reference herein).
In some embodiments, the optical emitter arrays on the shanks can deliver programmed, sub-nanosecond pulsed-excitation light within the tissue, which can be neural tissue, with repetitive or asynchronous rates engineered to permit optimal signal extraction defined by the properties of the functional optical reporters. In these and other embodiments, the optical detector arrays on the shanks can permit programmed signal integration (including either intensity integration or photon counting) of the time-varying illumination that is impingent upon them. Programmed operation can include time-gated collection that permits rejection of the “feed-through” illumination (which results from, and occurs during, excitation pulses), allowing it to be separated from the desired, functionally-induced optical signals emanating from the optical reporters.
Integrated neurophotonics is a novel technology that enables unprecedentedly dense, simultaneous, and cell-type specific monitoring of neurons and their interactions, in vivo, in real time. As an example, elucidating the neural circuitry of the neocortex is among the new classes of studies possible—for this, recording neural activity with cellular resolution and cell-type specificity in all six cortical layers will be required. Cortical architecture appears to be organized in columns; in the mouse brain these contain ˜100,000 neurons in a ˜1-mm3 volume. The fact that ˜90% of the column's connections are local[1,2] suggests detailed investigation of these as candidate microcircuits. Here, to clarify description of the technology and to provide concrete methods for its embodiment, the specific target of recording densely from a single cortical column will be used as an example.
One embodiment includes the recording of all the activity from the ˜100,000 neurons within one cortical column of a mouse. The system can be modular and scalable; this permits tiling multiple nanophotonic modules to cover neural circuits spanning extended brain regions. Engaging in large-scale production of integrated neurophotonic modules can make it feasible to enable recording from millions of neurons with single-neuron resolution. For example, assembly of ten of the prototype modules described herein would enable recording from all ˜1 million neurons in the mouse visual cortex.
The research enabled by these powerful tools will provide unprecedented and massive data sets that will, in turn, enable a mechanistic understanding of how the cortical circuits functions normally and how they fail in neuropsychiatric disorders. Recording from all neurons in a local circuit will revolutionize understanding of information processing in the brain. For example, it would enable testing of the long-standing idea that the neocortex is built from repeating computational circuit modules. By contrast, present in vivo 3D-imaging technologies are many orders magnitudes away from being able to achieve such a result.
Integrated neurophotonics will ultimately also transform the study of neuropsychiatric disorders such as depression and post-traumatic stress disorder (PTSD). These devastating illnesses are believed by many to be brain circuit dysfunctions resulting from subtle alterations of circuit interactions between specific neuronal subtypes [3,4]. Consistent with this view, new findings in the field of human genetics that has revealed hundreds of gene mutations in the past decade that correlate with such neuropsychiatric disorders; and many of these disease-related genes are linked to synapse formation and function. The expression of these disease-associated genes has recently begun to be systematically mapped to specific brain regions and neuronal cell types [5]. However, it is still not known which properties of these cell classes are affected, and how their functional roles might be altered in dysfunctional brain circuits. Understanding the mechanisms of specific circuit interactions that play a role in animal models of psychiatric disorders can facilitate development of drugs specifically targeting aberrant circuit elements.
Currently, functional imaging of neuronal activity in the rodent cortex is achieved using free-space two-photon laser-scanning microscopy [6] together with fluorescent calcium reporters [7]—and this combination provides cellular resolution of activity. Calcium reporters, introduced within the soma, are now widely employed as a robust proxy for electrophysiological measurements. Among such reporters are exogenous synthetic molecules, providing no cellular specificity (e.g. Oregon Green BAPTA-1); or genetically encoded proteins, such as the GCaMP family [8], which can provide cellular specificity through promoter activation and repression [9]. These reporters operate by sensing the intracellular calcium influx following an action potential; this modulates the calcium binding to the reporter and thereby alters its optical cross-section. This stereotypical fluorescent transient is interrogated optically to provide a “report” on calcium influx after the neuron fires.
To excite these optical reporters, a serial scanning optical method based on two-photon microscopy is often employed. This involves the simultaneous absorption of two photons by nonlinear processes to induce excitation of the reporter; its subsequent decay to the ground state results in fluorescence emission. Often, near infrared excitation wavelengths are used for biological microscopy; the resulting fluorescence is in the visible spectrum. Because very high photon density is required to induce two-photon absorption, the technique requires a single, tightly spatially- and temporally-focused beam of light generated by pulsed, femtosecond-scale, laser light. Accordingly, to achieve volumetric sampling in three dimensions, a serial, point-scanning methodology or holographic spatial light modulation becomes necessary. In the first methodology the two-photon interrogation voxel, which is typically ˜0.5×0.5×4 m3, is scanned in 3-D, one-location-at-a-time, to map the activity-dependent fluorescence of reporters in individual neurons. Today's state-of-the-art practice utilizes random-access acousto-optic deflectors (AODs), providing ˜10 μs point-access time. Currently, this permits routine mapping of ˜400 neurons in a 3-D volume of 200×200×100 μm3 with the requisite SNR to track spiking activity via the modulated somatic calcium signals (
The aforementioned approach has two fundamental limitations that preclude scaling it up to enable functional imaging of large neuronal ensembles spanning extended brain regions: (i) serial optical interrogation, and (ii) signal-to-noise ratio (SNR) degradation with depth.
While the aforementioned serial point-scanning optical techniques can provide sub-cellular resolution, they have the significant disadvantage that the total number of scanned voxels is limited, in practice, by scanner speed. This speed limitation also affects current technology for spatial light modulators. This sampling-speed limitation is further exacerbated by the photometric requirement that excitation illumination must dwell at each voxel long enough to achieve requisite SNR. Parallelization of scanned two-photon microscopy in a fixed plane has been demonstrated in brain tissue by using a multiplicity of excitation beams simultaneously followed by conventional wide-field detection. Each beam is encoded with specific binary amplitude modulation to guarantee the unequivocal localization of the fluorescence generated [13]. Such “depth multiplexing”, using four pulsed laser beams with sequential pulses, simultaneously focused at different depths and interrogated with gated detection, has been used to map cortical activity in four optical planes at four different depths [14]. However, it is clear that only a limited number of beams that can be implemented with such a technique; the maximum level of multiplexing that can be achieved is ultimately determined by the laser repetition rate and the reporter fluorescence decay time. To scale this upward requires facing challenging, practical questions concerning the provision of sufficient power in each beam to permit deep imaging in highly scattering neural tissue.
Scattering and absorption limit the ability to deliver ballistic (i.e., unscattered) light with sufficient intensity to achieve tightly focused two-photon excitation deep within the brain. Ultimately, water absorption (
An alternative approach involves using longer excitation wavelength NIR excitation around 1.6 μm [16]. This is possible by harnessing three-photon absorption processes, but their far smaller cross-sections for existing protein-based reporters imposes as a serious limit on the utility of this approach.
Another proposed approach is to employ adaptive optical corrections to rectify wavefront aberrations that are induced by optical scattering and absorption in brain tissue [17]. In principle, this could restore optical resolution in the two-photon modality, and thereby improve deep-imaging capability. However, the approach is contingent upon measuring, and employing, the precise aberration matrix for a large volume of very heterogeneous media. This is a difficult prospect in highly scattering mammalian brain tissue; further, once obtained it is unclear whether its values would remain sufficiently stationary over typical measurement intervals.
Scattering also acts to dramatically suppress the fluorescence signal accessible via free-space optics. After two-photon excitation, the fluorescent photons originating deep within brain tissue suffer multiple scattering during their propagation. Hence capturing them efficiently after they emerge from the brain's surface requires free-space collection optics with large angular acceptance, i.e. a large field of view, and low magnification [18]. Optics that provide sufficiently large numerical apertures for excitation, and large angular acceptance for light collection, become physically immense. Ultimately, their benefits are limited.
These aforementioned complications in the delivery of excitation light from free space into neural tissue, and the subsequent collection of emitted light after it emerges from tissue back into free-space—to and from regions deep within the brain—have motivated the development of microendoscopy. This method involves implanting a rather large and rigid cannula containing an optical fiber into targeted regions of the brain. After implantation it is then employed for local, functional calcium imaging at the fiber's distal end via one-photon fluorescence excitation of reporters. Although microendoscopy resolves the issue of light delivery and recovery from remote and deep regions of neural tissue, it has very significant limitations. Prominent among these are: i) imaging occurs only within one optical plane near the tip of the endoscope; ii) tissue along the path of the large (typically 0.3-1 mm) implanted cannula/fiber is completely and irreversibly destroyed; and, hence, iii) the approach does not permit studies of vertical structures simultaneously (e.g. cortical layers). Accordingly, the approach is feasible only for acute measurements at the fiber's tip, using direct CCD-imaging [19] or probe-based confocal laser microendoscopy [20]. The goal of integrated neurophotonics is to achieve functional imaging of extended brain regions at arbitrary depths with minimal perturbation of neural tissue. This is not achievable with this method, nor it is compatible while preserving the brain's integrity—given the endoscopic cannula's size.
Integrated neurophotonics is an entirely new paradigm for functional imaging [21,22,23]. It harnesses recent advances in integrated nanophotonics and functional optical reporters. This new technological approach will surmount the limitations of existing methodologies outlined above. It will enable:
a) Electrophysiological recording and stimulation, with cellular resolution, in highly scattering (mammalian) brain tissue,
b) Access to all regions of the brain, no matter how deep,
c) Complete coverage of all neurons within targeted volumes,
d) Cell-specific interrogation (via protein-based reporters) and complex, finely tuned neurological control (via optogenetics and precisely controlled fields of excitation light),
e) Up-scaling to complex systems that permit simultaneous interrogation and control of millions of neurons, and
f) Mass-production of complete measurement systems using existing microelectronics foundries for ultimate dissemination to the neuroscience and neuromedical communities.
The novel methodology of integrated neurophotonics is based on distributing a dense, 3-D lattice of thousands of emitter pixels (E-pixels) and detector pixels (D-pixels) within the brain on an architecture of neurophotonic probes (
In the some embodiments, tracking of somatic calcium transients that arise from neural spiking can be read out, in parallel from a multiplicity of labeled neurons, to acquire the ensemble of their individual fluorescence time-series. This information can be retrieved in the time domain by gated integration of illumination or by nanosecond optical interrogation and time-correlated photon counting. This is enabled by the arrays of integrated-nanophotonics-based emitters (E-pixels) that operate in concert with detector arrays (D-pixels). Among possible embodiments for the individual D-pixels are gated CMOS photodetectors or single-photon avalanche photodiodes (SPAD). The D- and E-pixel elements can be specially designed for embedding as large arrays onto ultranarrow shanks for acutely or chronically implanting into neural tissue. The data acquired from such an integrated neurophotonic system can yield fluorescence time records for the entire ensemble of active neurons within the volume probed. New protocols, which are termed “optical spike sorting”, enable such data extraction from the raw data provided by the D-pixel arrays. These protocols employ model-based clustering algorithms, similar to spike sorting protocols used in multi-site electrical recording.
The integrated neurophotonics technology images neuronal activity from inside the brain by distributing thousands of local light sources and detectors within large volumes arbitrarily deep in the brain. This can be achieved by distributing light emitters and detectors throughout the brain on, for example, ultra fine silicon shanks (on the order of ˜25 μm wide and 15 μm thick), and then controlling them with integrated nanophotonic and nanoelectronic chips (
The integrated neurophotonic systems described herein merge three key technologies. First, they leverage current developments in fabrication of advanced silicon-based nanoprobe arrays for deep and massively multiplexed electrophysiological recording in brain tissue (
To provide a specific embodiment of one possible embodiment, an integrated neurophotonic system is envisioned that coalesces NE=2050 E-pixels and ND=2050 D-pixels within a volume of 1×1×1 mm3 of brain tissue, using an array of 25 ultra fine shanks. The detection pixels are realized as photon-counting detectors, specifically, as single-photon avalanche photodiodes. The stimulation pixels are realized as E-pixels located at the termini of integrated nanophotonic waveguides running along the probes shanks; these termini are spatially distributed along the shank in configurations determined by experiments and computations as providing the most ideal raw data for “de-mixing”. At the top of the integrated photonics probe, these waveguides efficiently interface with the sources via a separate, active photonics-source chip at the layer head.
Referring to
Referring to
Fabrication of photonic probes involves a variety of standard lithographically-based micro- and nano-fabrication steps, concatenated in a unique sequence to realize this novel technology.
The shanks can be patterned from layered materials, such as silicon-on-insulator (SOI) substrates, using techniques of surface micromachining. The topmost silicon layer of the SOI substrate in this instantiation provides the structural material for the probe shanks themselves. Alternatively, bulk micromachining techniques common to microelectromechanical systems (MEMS) processing can be employed to realize probe shanks through processes such as selective chemical etching.
Integrated photonic waveguide elements are fabricated using standard methods and in standard requisite geometries by selectively patterning layered materials possessing different indices of refractions. For visible wavelength instantiations, for example, silicon nitride and silicon oxide layer can provide the requisite light confinement.
Emitter elements are fabricated at the termini of each integrated photonic waveguide on the probe chip. These may be as simple as a blunt end of a waveguide, lithographically etched to provide a surface perpendicular to the axis of the waveguide. More advanced E-pixel termini—as mentioned, involving conventional lens elements, planar lens elements, or metamaterials-based elements—can be employed to engineer the spatial profile of the light beam emerging from each terminus.
Detector elements are described below. These can take the form of integrating elements providing grey scale values for the illumination collected at each D-pixel, or photon-counting elements that quantify the precise numbers of photon each D-pixel collects during a specified time interval.
A multi-channel optical interface between the head of the integrated neurophotonics probe and the separate, active photonics-source chip is present in the embodiment. The coupling elements used to form such an interface—which are present on both the integrated neurophotonics probe and the separate, active photonics-source chip—shall be termed chip-to-chip coupler arrays. A variety of methodologies for providing such coupler arrays are possible. For example, opposing arrays of planar grating couplers can be used on both chips and then registered spatially as the chips are brought to proximal face-to-face alignment. Efficient transfer of photonic signals is then mediated in the free-space region across the gap between the aligned chips. Importantly, such a stand-off between the integrated neurophotonics probe and the separate, active photonics chip can provide thermal isolation from system elements operating with high power dissipation—this can very effectively circumvent very critical issues of heating in delicate neural tissues.
The aforementioned grating-coupler-mediated chip-to-chip photonic interfaces can be augmented or supplanted by faceted mirrors, which may be fabricated by various standard lithographic and etching methods. These can be further augmented, or supplanted, by lens elements that can provide enhanced efficiency. Among possible lens elements are conventional structures comprising three-dimensional shapes patterned from materials with different indices of refraction, or those comprised of planar lithographically-defined focusing elements (such of zone plates, etc.), or alternatively those comprised of metamaterials-based elements.
Arrays of miniature light sources are employed to drive the E-pixel arrays. These photonics-source arrays, which are located on the separate, active photonic-source chip, deliver light through the chip-to-chip coupler array to the waveguide arrays that run along the integrated photonics probe and ultimately terminate at the array of E-pixel elements.
Two instantiations exemplify distinct realizations of such photonics-source arrays.
First, a single source of fast, pulsatory illumination may be employed. Among possible single-source elements can be pulsed laser sources, supercontinuum laser sources, etc. In this instantiation the single incoming light beam is divided up in the N daughter beams by an optical power splitter, then each daughter beam is modulated by integrated photonic elements to temporally encode the requisite time profile that is ultimately desired to realize specific patterns of illumination emerging from the E-pixel array. This is further described below.
Second, an array of microscale light sources can be employed. (Several possible instantiations are described below.) Each photonics-source element in these arrays provide excitation light at the requisite wavelength(s) for the optical reporters or optical effectors, and also permit fast modulation of the light to permit formation of complex illumination—patterned both spatiotemporally, and also spectrally (if desired)—from the E-pixel arrays. Given that typical florescence lifetimes can be of nanosecond duration, modulation rates exceeding 1 GHz are desirable.
To drive the E-pixels, optical sources are required. For pulsatory or time-gated implementations, these must produce light pulses with shorter temporal duration than the lifetime of the optical reporters (for recording local activity) or effectors (for stimulating local activity) that are employed. Typical fluorescent lifetimes are in the several nanosecond range, hence sub-nanosecond pulses at the desired wavelength will be required. Many measurements are desired during the time course of a typical activity event. For example, there will be a time window of up to tens of milliseconds during which detection of neuronal action potentials via intracellular Calcium reporters is possible. In this case, measurement repetition rates on the order of one hundred MHz will provide optimal sampling of fluorescence excitation without saturation. Such measurements can be implemented in two possible ways: (i) A free-space mode-locked laser can be coupled to an integrated optical splitter on the photonic chip which will produce NE=2050 individual light sources. Each optical channel of the splitter comprises an on/off switch (using all-optical, acousto-optical or electro-mechanical modulators) for creating a vast multiplicity of illumination patterns from the emitter pixel array. The 2050-optical channels are carried by waveguides running on the fine shanks, which terminate at desired locations along the shanks. At these termini, various optical elements can be employed to control the angular profile of the termini-emitted light. (ii) In a second instantiation, NE=2050 individual integrated light sources (laser diodes, LED, resonant-LED) can be coupled—for example, via grating couplers, mirrors, planar, nanostructured, or metamaterial-based optical elements, and other implementations—to integrated optical-modulators or optical gates in order to generate sub-nanosecond light-pulses with rates on the order of 100 MHz and sub-nanosecond duration. The patterning of light emission from the E-pixel array termini can be achieved by on/off or continuous modulation of the individual sources driving each E-pixel element; this can be achieved either by direct electrical control of the sources, or by a downstream array of optical modulators. In the latter case, the light modulator outputs would couple to on-chip waveguides running along the fine shanks to the aforementioned termini (with any included optical elements to control the spatial profile of emission).
For detector pixel realizations based on single-photon detectors (such as SPADs), signal output is in the form of electrical pulses corresponding to the detection of individual fluorescence photons. These pulses, characterized by their duration and amplitude, can be be sent through fast electrical lines, e.g. coplanar waveguides, that run along the fine shanks up to a time-to-digital-converter circuit located on the probe head. This readout circuit then generates a digital quantification of the fluorescence photon counts detected by each pixel during the relevant integration time dictated by to the reporter kinetic. Electrical commands, i.e. time-gates and bias, to the detector pixels will be provided by the same circuit comprising the time-to-digital circuit, and can be routed to the detector pixels by electrical connections running on shanks as well. The readout detector circuitry subsequently outputs digital signals to computer data acquisition interface.
Referring to
The shank-array spacing and the pixel pitch together delineate a unit volume that is interrogated by adjacent nanophotonic emitter/detector pixels (
The optical attenuation length of the neural tissue at the wavelength of interest can be experimentally evaluated by measuring the loss of optical power from a plane-wave like illumination going trough a tissue slice of known thickness.
At 480 nm the optical attenuation length is typically deduced to be on the order of 200 μm [6]. Together, the emitter pixel array makes possible illumination of each unit volume's contents with at least to 218˜262,000 different patterns. For each illuminated unit volume, readout of the induced fluorescence will be possible from 18 independent positions for each of the chosen illumination patterns. Together, this provides >4 million combinations of measurements for every unit volume. The >1 B combinations available with the 25-shank module yield sufficiently dense coverage of all ˜100,000 neurons within the 1 mm3 volume of brain tissue to permit their unique and individual sampling.
The total optical power of one emitter pixel can be in the range of ˜10 to ˜100 μW at visible wavelengths (λE=480±10 nm). Output powers in this range and below preclude induction of tissue damage near the E-pixel termini. Excitation and collection pixels can be oriented orthogonally on the shanks to minimize direct spectral feedthrough, although the most effective method for suppression of direct E- to D-pixel optical coupling (feedthrough) is through use of time gating. This enables staggering the excitation and detection time windows; during the latter the illumination can be completely turned off allowing fluorescence from optical reporters to be sensed with minimal background. The requisite readout bandwidth can be dictated, at the low end, by reporter kinetics. This is in the range of ˜100 Hz when using slow calcium reporters such as genetically encoded GCaMP proteins or exogenous BAPTA-based molecules like Oregon-Green®488BAPTA-1 (Life-Technologies). At the high end, bandwidths of order 1 kHz will become achievable with use of fast voltage sensitive reporters such as genetically encoded Arch (Archaerhodopsin)-based fluorescent voltage sensor or dyes like ANEP (aminonaphthylethenylpyridinium) and variants. This will enable implementation of rapid changes in patterned illumination to realize optimal spike sorting protocols. A total detector-integration time of 10 ms per readout is envisioned. This is compatible with the targeted mean irradiance of 1016 photon/s/cm2 within the unit volume, and with the kinetics of the current GCaMP reporters.
CMOS-compatible sensors integrated on the shank could take two forms. In traditional CMOS image sensors (such as those employed in the cameras of many light microscopes), photocurrent is integrated onto the reverse-biased photodiode on which it is generated, producing a voltage signal that is directly proportional to the light intensity. The sensor itself is “low-gain”; that is, it produces fewer electrons that incident photons. An alternative sensor is a “high-gain” one that produces many electrons from a single incident photon. The photomultiplier tube is an example of such a photon-counting sensor. In the solid-state world, detectors providing single-photon sensitivity take the form of single-photon avalanche diodes (SPADs), which are photodiodes biased beyond their avalanche breakdown voltage. When a photon is incident, it creates an electron-hole pair with a probability known as the photon detection probability (PDP), which triggers carrier avalanche within the diode (
These high-gain single-photon detectors allow one to accurately measure the arrival time of individual incoming photons, in a measurement technique known as time-correlated single-photon counting (TCSPC). For each SPAD, this requires a time-to-digital converter (TDC), which accurately captures the arrival time of the photon in digital form. The combination of a SPAD and pulsed excitation light, to which one can synchronize the measurement, one can easily time-gate the fluorescence measurement and eliminate the feedthrough of interrogation light to the detector.
SPADs are used in a time-correlated single-photon counting mode in which arrival time histograms are recorded through time-to-digital conversion of photon-activated pulses from the detectors. SPAD detection limits are determined by noise in the form of the device's dark count rate (DCR). DCR is dominated by avalanche events that are triggered by the thermal generation of carriers from recombination-generation (RG) centers within a diffusion length of the multiplication region of the SPAD. The shallow trench isolation (STI) that is used to separate devices in modern CMOS processes creates a relatively defect-rich interface and a significant source of RG traps. The inventors have figured out approaches to eliminate these STI interfaces from the multiplication region of the SPAD structure, allowing the formation of SPADs in a rather advanced 0.13-μm CMOS node[30]. These SPADs have an octagonal photosensitive area with a diagonal extent of ˜5 μm (
In existing devices [30] relatively shallow junction depths (300 nm), which result from using the p+ mask for a PFET source and drain implant, cause the photon detection probability (PDP) to peak at ˜425 nm. Pixel implants can be optimized (using lighter doping) to achieve higher PDP at longer wavelengths.
To reduce deadtime, active quenching circuitry is often utilized. In the inventors' previous SPAD design [31,32], the pixel circuitry shown in
A time-to-digital converter (TDC) is used to measure the arrival time of the first photon detected during each measurement window. In previous designs [31,32], the design targets for this TDC were for a 62.5 ps resolution with a 64 ns range (
If a 10-bit time value is output for every pixel after each excitation event, with a laser pulse rate of 20 MHz, an off-chip data rate of 1.8 Tbps would be required for the array. This data rate, however, does not reflect the sparseness of the data. In particular, TCSPC experiments typically record a photon hit for only 1-2% of laser repetitions. Through the use of an event-driven readout approach, sparseness is exploited in our design to reduce the average data rate to approximately 18 Gbps. To achieve this, the time data for each pixel are appended with a valid bit that indicates whether a pixel event has occurred. This valid bit is used to control the flow of data out of the array such that only data associated with pixel events are allowed to pass.
Current power dissipation is about 80 μW for the SPAD and 30 μW for each TDC channel. For a total power budget of 5 mW down each shank, which creates less than 0.5 degree of heating at the tissue, about 40 SPAD detectors per shank can be supported. The D-pixel arrays can be increased if power consumption is further reduced. The current data path design consumes in excess of 4 mW/channel through inefficient use of the data sparseness, consuming a significant amount of power “clocking zeros” through the design. This can be expected to be reduced to less than 500 μW/channel in a new design
The data acquired from an integrated neurophotonics functional imaging system will be inherently complex; algorithms for “de-mixing”—that is, transforming the acquired photon counts at each D-pixel in the detector array into time records of activity from individual neurons—will need to be developed. Each E-pixel illuminates a local volume within the brain, which contains multiple neurons and the surrounding neuropil. The resulting fluorescent emission is then measured from many perspectives, via multiple detectors within roughly one or two attenuation lengths, LA, from the emitting neuron. The overarching goal of the prototype described here is to enable recording all of the activity of a 1 mm3 volume of the mouse cortex, containing ˜100,000 neurons, with single cell resolution. This is feasible using one 25-shank array of photonic probes. Using 2050 E-pixels that are separately activatable to create complex patterns of local illumination, and simultaneously monitoring the evoked fluorescence with 2050 D-pixels, provides access to a space of almost 5 million measurement configurations. More complex patterns of illumination, beyond simple on/off modulation, can further increase the richness, i.e. the complexity, of this measurement space. Reconstructing the sources of activity, i.e. identifying the activity of the neurons in this volume, which shall be termed “optical spike sorting” is an important signal decomposition problem, and development of efficient algorithms for this can be produced.
Specific design considerations help to make this problem more tractable. Initial use of the GCaMP6 activity reporter is envisioned; this reporter has different optical properties that depend upon whether it is in the calcium-bound or—unbound state—the former corresponds to conditions of high local Calcium concentration. Conversely, this reporter has a very low fluorescence level under conditions of low calcium concentration. Accordingly, the amount of background noise from the inactive neurons within an illuminated volume can be very low. Second, by placing detectors and emitters much closer to the neurons than in traditional microscopy, the efficiency of illumination and detection increases dramatically, improving the signal amplitude. Third, because of the local configuration of the illumination, a single emitter excites only a small subset of the unit volume. This optical “sectioning” further facilitates decomposing the datasets into individual neurons. Specifically, a particular configuration of activated E-pixels can only excite a subset of the neurons that are active (i.e. “spiking”), and a particular configuration of D-pixels will only collect from another subset of the neurons. This reduces the entire decomposition problem into a number of overlapping smaller problems.
The signal decomposition problem can be formulated as follows:
where d(t) are the observations over time, measured from the detectors with each combination, i.e. “pattern”, of excitation. Note that the d(t) are samples in ˜5 million dimensional space for the prototype example (i.e. 2050 D-pixels and 2050 E-pixels), n(t) is the noise (which may be partially correlated across channels), δt(t) is 1 when neuron fires an action potential and zero otherwise, and ri(τ) is the kernel that describes the fluorescence activity. The kernel can be largely predicted based on the geometry between a neuron and the emitter and detector arrays—as well as the known, typical time course of the reporter's fluorescence signal (however, it will likely benefit from some degree of fitting). Finding the latent variables δi(t) allows solving for the set of spike times that would mostly likely result in the given dataset. Latent variable problems in neuroscience such as this are typically solved using probabilistic techniques such as particle filters [35]. However these methods can be optimized to deal efficiently with the high-dimensional data space and the multiplicity of neuronal sources involved in the present approach. Alternative methodologies such as detecting events, clustering them, and “unpeeling” them to recover the underlying activity hold additional promise for such analyses [36].
Although the system has been described mainly in connection with neural tissue, the methods, systems, devices and other embodiments are also applicable to other tissues, such as muscle. Examples of cells in that can be investigated and appropriately labeled include, but are not limited to, neurons, glial cells and muscle cells. The tissue can be in an organism, or can be explanted tissue.
In some embodiments, the tissue can be prepared by optogenetic methods. In optogenetics, photoactivatable proteins, receptors or channels can be incorporated into tissues, making the tissues photo-responsive (Yizhar, O., et al., Optogenetics in Neural Systems, Neuron 71, 2011; Zhang, F., et al., Channelrhodopsin-2 and optical control of excitable cells, Nature Methods 3(10), 2006; Boyden, E., et al., Millisecond-timescale, genetically targeted optical control of neural activity, Nat. Neurosci 8(9), 2005).
Although various components of the probe device have been described separately, it should be understood that any embodiment of one component is contemplated to be combined with any embodiment of another component. Thus, for example, any combination of optical emitters, optical detectors, fiber shanks and optical sources is envisioned. Similarly, although various features of the methods have been described separately, it should be understood that any embodiment of one feature is contemplated to be combined with any embodiment of another feature.
The present invention may be better understood by referring to the accompanying examples, which are intended for illustration purposes only and should not in any sense be construed as limiting the scope of the invention.
A simulation based on the prototype design for the neurophotonic probe arrays has been executed (
The following publications are incorporated by reference herein:
Although the present invention has been described in connection with the preferred embodiments, it is to be understood that modifications and variations may be utilized without departing from the principles and scope of the invention, as those skilled in the art will readily understand. Accordingly, such modifications may be practiced within the scope of the invention and the following claims.
| Number | Date | Country | |
|---|---|---|---|
| 62054893 | Sep 2014 | US | |
| 61900216 | Nov 2013 | US | |
| 61539133 | Sep 2011 | US | |
| 61568331 | Dec 2011 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | 13627755 | Sep 2012 | US |
| Child | 14534163 | US |
