The present technology relates generally to restoring and augmenting brain function. In particular, some embodiments of the present technology include systems and methods for inducing neuroplasticity in the nervous system of a human patient or for enhancing the capabilities of the nervous system of an able-bodied individual.
The human brain is one of the world's most powerful autonomous computers, but nevertheless remains a fragile organ that can be difficult to train and repair, and that is limited by the sensory inputs and processing capacity provided by the human body. The parallel functions of and similarities between the brain and typical silicon computers have prompted considerable interest in the field of brain-computer interfaces. Brain-computer interfaces may be able to address some of the limitations of the human brain as well as bolster our understanding of an organ with many functions and operations that remain to be understood. More specifically, the computational capabilities of biological neural networks and silicon computers are complementary. For example, human brains commonly transfer information bidirectionally with computers through normal sensory and motor channels. However, transferring information through direct recording of neural activity and electrical stimulation of brain sites is much more challenging. Nevertheless, recent advances in interface technologies, computing systems, and our understanding of the human brain have sparked new investigations into the potential of brain-computer interfaces that directly record and/or stimulate the brain.
Aspects of the present disclosure are directed generally toward systems for restoring or enhancing neural function by inducing new neural connections in a nervous system (e.g., in the brain or spinal cord of a human subject) and associated methods. In several of the embodiments described below, a system for inducing new neural connections in a nervous system of a human subject includes at least one sensor implanted in the subject for detecting and recording neural signals from a first region of the nervous system of the subject. In other embodiments, in addition to or instead of sensors implanted in the human body, the sensor may be positioned external to the body of the subject and configured to detect and record external signals. The sensors can be communicatively coupled to a computing device that receives the signals recorded by the sensors and generates a stimulation pattern based on (a) the received signals and (b) an adaptive artificial neural model. The system can further include a stimulating component implanted in and/or worn externally by the subject and communicatively coupled to the computing device for receiving the stimulation pattern. The stimulating component can stimulate a second region of the nervous system of the subject based on the stimulation pattern to generate a measurable response (e.g., one or more neural patterns or signals internal to the nervous system, motor behavior of the subject, sensory behavior of the subject, etc.). An error signal can be determined by comparing the measurable response to a desired response (e.g., from rehabilitation procedures), and the artificial neural model can be adjusted based on the error signal and/or based on data, information, or signals other than the error signal.
In the following detailed description, specific details are set forth to provide an understanding of the present technology. However, the present technology may be practiced without some of these specific details. In some instances, well-known structures and techniques have not been shown in detail so as not to obscure the present technology. The terminology used in the description presented below is intended to be interpreted in its broadest reasonable manner, even though it is being used in conjunction with a detailed description of certain specific embodiments of the disclosure. Certain terms may even be emphasized below; however, any terminology intended to be interpreted in any restricted manner will be overtly and specifically defined as such in this Detailed Description section.
The present technology can include a computing device (a “co-processor”) that is effectively arranged “in parallel” with a portion of the nervous system of the subject (e.g., the brain) and that operates synergistically with the brain to jointly optimize a task or behavior. The systems and methods described herein can be used to augment the natural computational abilities of the brain and promote neural plasticity for the creation of new natural pathways between different areas of the brain, nervous system, or body organs. These new neural pathways can function to replace lost neural function (e.g., resulting from injury) or to augment existing neural function in the brain or other parts of the nervous system.
The following discussion provides a general description of a suitable environment in which the present technology may be implemented. Although not required, aspects of the technology are described in the general context of computer-executable instructions, such as routines executed by a general-purpose computer. Aspects of the technology can be embodied in a special purpose computer or data processor that is specifically programmed, configured, or constructed to perform one or more of the computer-executable instructions explained in detail herein. Aspects of the technology can also be practiced in distributed computing environments where tasks or modules are performed by remote processing devices, which are linked through a communication network (e.g., a wireless communication network, a wired communication network, a cellular communication network, the Internet, and/or a short-range radio network (e.g., via Bluetooth)). In a distributed computing environment, program modules may be located in both local and remote memory storage devices.
Computer-implemented instructions, data structures, screen displays, and other data under aspects of the technology may be stored or distributed on computer-readable storage media, including magnetically or optically readable computer disks, as microcode on semiconductor memory, nanotechnology memory, organic or optical memory, or other portable and/or non-transitory data storage media. In some embodiments, aspects of the technology may be distributed over the Internet or over other networks (e.g. a Bluetooth network) on a propagated signal on a propagation medium (e.g., an electromagnetic wave(s) or a sound wave) over a period of time, or may be provided on any analog or digital network (e.g., packet switched, circuit switched, or other scheme).
The sensor module 120 can be any type of sensor such as, for example, a sensor configured to detect neural signals in the nervous system of the subject (e.g., for detecting neuroelectrical activity, neurochemical activity, etc.), an ultrasonic sensor, an infrared sensor, or another type of sensor. Accordingly, the sensor module 120 can be configured to detect signals that are external (e.g., infrared signals from an environment around the subject) and/or internal (e.g., neural signals from the nervous system of the subject) relative to the subject. In some embodiments, the sensor module 120 is implanted proximate a region of interest 124 in the subject. The region of interest 124 can be, for example, a region of the subject's brain or another portion of the subject's nervous system. In some embodiments, the sensor module 120 can be a multi-channel sensor that includes a plurality of individual sensor elements 122. Each sensor element 122 can be configured to detect one or more signals generated or detected at a different location in and/or on the region of interest 124. For example, where the sensor module 120 is implanted within and/or worn externally by the subject and the region of interest 124 is a region of the subject's brain, the sensor elements 122 can be configured to provide a spatial and/or spatiotemporal sampling of neural signals generated over the region of interest 124. In some embodiments, the number of individual sensor elements 122 (e.g., measurement channels) in the sensor module 120 can be selected to correspond to a characteristic of the region of interest 124 (e.g., the physical structure of the region, the degrees of freedom in a detectable signal at the region, the numbers of neurons in the region, etc.).
In some embodiments, the information source 125 is provided instead of or in addition to the sensor module 120. The information source 125 can comprise any source of information such as the Internet, a data structure containing data, a source of user input, another nervous system, etc., for providing input to the processing subsystem 110.
The stimulation module 130 is positioned at or proximate to a stimulation region 134 in the subject (e.g., implanted in and/or worn externally by the subject). The stimulation region 134 can be, for example, a region of the brain of the subject, another region of the nervous system of the subject (e.g., the spinal cord or different nerves), a region of muscle, and/or a region of an organ of the subject. The stimulation module 130 can include any invasive or non-invasive hardware for stimulating the stimulation region 134 of the subject. For example, the stimulation module 130 can be configured to stimulate the stimulation region 134 using one or more of: electrical activation, optical activation, magnetic activation, ultrasonic activation, and chemical activation. In some embodiments, the stimulation module 130 can be a multi-channel module that includes a plurality of individual stimulating elements 132. Each stimulating element 132 can be configured to stimulate the stimulation region 134 at a different location in and/or on the stimulation region 134. For example, the stimulation region 134 can be a region in the subject's brain (e.g., the primary motor cortex of the brain, a region of the brain implicated in a certain neural function, etc.) and the stimulating elements 132 can be configured to provide a spatially and/or spatiotemporally differing stimulation pattern along and/or through the stimulation region 134. In some embodiments, the number of individual stimulating elements 132 (e.g., stimulation channels) in the stimulation module 130 can be selected to correspond to a characteristic (e.g., the physical structure) of the stimulation region 134.
The processing subsystem 110 comprises several components including memory 111 (e.g., one or more computer readable storage modules, components, devices, etc.) and one or more processors 112. The memory 111 can be configured to store information (e.g., signal data, subject information or profiles, environmental data, data collected from one or more sensors, media files, etc.) and/or executable instructions that can be executed by the one or more processors 112. The memory 111 can include, for example, instructions for processing multi-channel sensor data from the sensor module 120 and/or information from the information source 125 using an adaptive method such as an artificial neural model, for suppressing noise and other artifacts from signals received from the sensor module 120, for generating stimulation patterns for output to the stimulation module 130 based on the adaptive method, and for adjusting a neural model or other adaptive method, as described in further detail below.
The processing subsystem 110 also includes communication components 113 (e.g., a wired communication link and/or a wireless communication link (e.g., Bluetooth, Wi-Fi, infrared, and/or another wireless radio transmission network)) and a database 114 configured to store data (e.g., signal data acquired from the region of interest, equations, filters, etc.) used in the techniques for co-adapting with and inducing new neural connections in the subject, as disclosed herein. One or more first sensors 115 can provide additional data for use in inducing new neural connections in the subject. The sensors can also provide other data pertinent to a condition and/or environment of the subject. For example, the one or more first sensors 115 may include one or more ECoG sensors, voltammetry sensors of neurochemicals, blood pressure monitors, galvanometers, accelerometers, thermometers, hygrometers, blood pressure sensors, altimeters, gyroscopes, magnetometers, proximity sensors, barometers, etc. The first sensors 115 can also be configured to provide information about the system 100 itself, such as an operating condition (e.g., power level, noise level, etc.) of any or all of the components included therein. One or more displays 116 can provide video output and/or graphical representations of data obtained by the system 100, and can include one or more devices (e.g., keyboards, mice, joysticks, number pads, etc.) for supplying a user input at the displays 116. A power supply 117 (e.g., a power cable, one or more batteries, and/or one more capacitors) can provide electrical power to components of the processing subsystem 110 and/or the system 100. In embodiments that include one or more batteries (e.g., where the system 100) is a portable system), the power supply 117 can be configured to recharge, for example, via a power cable, inductive charging, and/or another suitable recharging method. Furthermore, in some embodiments, the processing subsystem 110 may include one or more additional components 118 (e.g., one or more microphones, cameras, Global Positioning System (GPS) sensors. Near Field Communication (NFC) sensors, etc.).
In some embodiments, the processing subsystem 110 can include one or more components partially or wholly incorporated into the sensor module 120, information source 125, and/or stimulation module 130. In other embodiments, however, the processing subsystem 110 may include components remote from the sensor module 120, information source 125, and/or stimulation module 130, and connected thereto by a communication network (e.g., the Internet and/or another network or cloud computers). In some embodiments, for example, at least a portion of the processing subsystem 110 may reside on a mobile device (e.g., a mobile phone, a tablet, a personal digital assistant, etc.) and/or a computer (e.g., a desktop computer, a laptop, etc.) communicatively coupled to the sensor module 120, information source 125, and/or stimulation module 130. Moreover, the processing subsystem 110 can be configured to be worn by the subject (e.g., carried by their body) and/or implanted in their body (e.g., proximate a region of their nervous system).
In some embodiments, the system 100 can optionally include one or more additional second sensors 140 and/or an error signal generating system 150. The second sensors 140 can be communicatively coupled to the error signal generating system 150 and/or directly communicatively coupled to the processing subsystem 110. The second sensors 140 can be configured to detect and/or record a measurable output by the subject resulting from stimulation of the stimulation region 134 by the stimulation module 130. The measurable output may be internal (e.g., a neural activity pattern, a neurochemical response, etc.) or external (e.g., a motor response) relative to the subject. Accordingly, the second sensors 140 may be internal (e.g., implanted within) relative to the subject and/or external to the subject. The optional error signal generating system 150 can be communicatively coupled to the processing subsystem 110 and configured to generate an error signal, as described in further detail below. The error signal generating system 150 can have some components generally similar to those of the system 10X) (e.g., one or more processors, sensors, memory, etc.). In some embodiments, the error signal generating system 150 stores or generates desired outputs corresponding to the measurable outputs recorded by the second sensors 140, compares the desired outputs to the measurable outputs, and calculates an error signal.
In certain embodiments, the second sensors 140 and/or the error signal generating system 150 may be partially or wholly incorporated into the processing subsystem 110, the stimulation module 130, the information source 125, and/or the sensor module 120, or omitted entirely. For example, the function of the second sensors 140 may be performed by the first sensors 115 of the processing subsystem 110, or the second sensors 140 can be omitted and any information about the measurable output from the subject can be determined by other means (e.g., by an observing physician, researcher, user, etc.) and directly input via the processing subsystem 110 (e.g., at the display 116).
Beginning at block 202, the process 200 comprises receiving neural signals from a region of interest in a nervous system of a subject. For example, in the embodiment illustrated in
At block 204, the process generates a stimulation pattern based on the neural signals received at block 202, and based on a neural model. The memory 111 (
At block 206, the process outputs the generated stimulation pattern to the nervous system of the subject to produce a measureable output within or outside the nervous system of the subject. For example, as illustrated in the embodiment of
The stimulation pattern can, for example, comprise a plurality of stimulation signals s1-sm each configured to stimulate a different portion of the stimulation region 134 (e.g., via the individual stimulating elements 132 of the stimulation module 130 of
The measureable output produced by stimulation of the nervous system can be either external or internal to the subject. For example, external measureable outputs include behavioral or motor responses of the subject (e.g., a physical movement of the subject) and/or sensory responses of the subject (e.g., an ability to perceive a specific stimuli), while internal measureable outputs include neural responses of the subject (e.g., a specific neuroelectrical or neurochemical activity pattern detected in the brain). As described above, in some embodiments, the measurable output can be detected, recorded, and/or measured by one or more sensors (e.g., the second sensors 140) of the system 100. In other embodiments, the measurable output is detected, recorded, and/or measured by another system, device, or person apart from the system 100 (e.g., a physician or researcher working with the subject).
At block 208, the process receives one or more error signals based, at least in part, on the measurable output. The error signals may be either internal or external to the subject depending on the measurable output. For example, as illustrated in
In other embodiments, the error signal EI can be an internal reward-based signal (e.g., a neurochemical reward prediction error) measured in one or more of the reward centers of the brain of the subject. That is, stimulation of the brain of the subject can produce a measurable output in the form of a release of dopamine or other neurotransmitter corresponding to a “reward or reward prediction error.” In some embodiments, the error signal generating system 150 includes one or more sensors (e.g., the second sensors 140) for measuring such reward-based error signals EI in reward centers of the brain 300 of the subject. The detection of a reward-based signal (e.g., the release of dopamine) can indicate a desired output resulting from the stimulation, and can be used to reinforce the behavior that produced the reward-based signal.
At block 210, the process adjusts the neural model based, at least in part, on the error signal. For example, the external or internal error signal can be used to adapt parameters (e.g., to adjust one or more weights or values of one or more variables defining the neural model) of the neural model using an optimization method for minimizing error. In some embodiments, where the error signal is a reward-based error signal, the parameters of the neural model can be adapted to maximize the reward-based signal. The parameters of the neural model can be adapted offline, or adapted online in response to streaming input of the error signal (e.g., from the error signal generating system 150). In some embodiments, the parameters of the neural model can be adjusted based on user input, data, information, and/or signals other than the error signal, either continuously or at certain offline times. That is, in certain embodiments the process 200 is an adaptive process for restoring lost neural function in which the adjustments to the neural model are based on signals or information different than an error signal.
Lastly, the system returns to block 202 and the process 200 can be repeated. The neural model can therefore be optimized through iterative adjustments to minimize error between a resulting output and a desired output while, at the same time, inducing new neural connections between the region of interest 124 and the stimulation region 134 (represented by arrow A in
In contrast to conventional brain-computer interfaces that use neural activity in the brain to control external devices, the present technology is directed to a bidirectional brain-computer interface having recurrent connections that allow activity-dependent stimulation of the brain, spinal cord, or muscles. Such a brain-computer interface can be made sufficiently small to be worn externally and/or implanted in a subject and operate autonomously during hours of free behavior.
Beginning at block 402, the process 400 includes receiving augmentation signals related to a neural function to be augmented. For example, in the embodiment illustrated in
In yet other embodiments, the augmentation signals a1-an are not derived from external sensors but are the neural signals n1-nk themselves received from a region of interest (e.g., the region of interest 124) in the nervous system of the subject. For example, the augmentation signals a1-an can be neural signals detected and/or recorded at the prefrontal cortex of the brain 500 of the subject with the goal of augmenting motor skill-learning, short-term memory, or planning capabilities of the subject. In such embodiments where the augmentation signals a1-an are internal to the subject, the embodiment illustrated in
At block 404, the process generates a stimulation pattern based on (a) the augmentation signals received at block 402, (b) the recorded neural signals, and (c) an adaptive artificial neural model. In some embodiments, the stimulation pattern is based only on the recorded augmentation signals. The neural model can be generally similar to the neural model described above with reference to
As illustrated in the embodiment of
Stimulation of the implicated region of the brain produces a measureable output that is either internal or external to the brain, as described above with reference to
In certain embodiments for augmenting neural function, utilizing a reward-based error signal can be particularly advantageous since a subject's desired output may be difficult to know beforehand, and therefore cannot be used as a basis for comparison for generating the error signal. For example, where the augmentation signals a1-an are received from an ultrasonic sensor in a naturalistic setting, it may be difficult to determine a precise desired output for use in determining the error signal. In such a context, a reward-based error signal (e.g., derived from neurochemical signals such as dopamine inside the brain) provides a tool for adjusting and optimizing the neural model that does not require previous knowledge of or measurement of desired outputs for the subject.
At block 410, the process adjusts the neural model based, at least in part, on the error signal. For example, the external or internal error signal can be used to adapt parameters (e.g., one or more sets of weights or network activations, regression parameters, etc.) of the neural model using an optimization method for minimizing error. However, in other embodiments, the process 400 is an adaptive process for augmenting neural function in which the adjustments to the neural model are based on signals or information different than an error signal. The process 400 then returns to the block 402 and can be repeated again. The process 400 causes the system 100 to co-adapt and jointly optimize task performance with the nervous system to (a) incorporate the artificial system 100X) (i.e., a neural co-processor) within the nervous system's computational processing pathways, and (b) promote neuroplasticity, rewiring, and creation of new neural pathways between populations of neurons, allowing the combined nervous system and computing system to learn a new skill or new sensation, and augment the brain's computational capacities. For example, in the embodiment of
From the foregoing, it will be appreciated that specific embodiments of the present technology have been described herein for purposes of illustration, but that various modifications may be made without deviating from the scope of the present technology. For example, in particular embodiments, the parameters of the neural model can be adjusted based on user input, data, information, and/or signals other than the error signal, either continuously or at certain offline times. That is, receiving an error signal is not a requirement of the neural co-processor (e.g., the system 100) described above with reference to
Several aspects of the present technology are set forth in the following examples.
1. A method for augmenting or restoring neural function and inducing new neural connections in a nervous system of a human subject, the method comprising:
2. The method of example 1, further comprising receiving neural signals from a region of interest in the brain of a human patient, wherein an injured region of the nervous system is between the region of interest and the stimulation region, and wherein the method further comprises promoting neuroplasticity between the region of interest and the stimulation region to promote recovery and restoration of lost neural function.
3. The method of example 1 or 2, further comprising receiving both neural and external signals associated with a neural function to be augmented, wherein the stimulation region is a region of the brain of the subject that is implicated in the neural function to be augmented, and wherein the method further comprises co-adapting with and promoting neuroplasticity in the brain or another region of the nervous system of the individual.
4. The method of any one of examples 1-3 wherein the neural signals include at least a first neural signal recorded at a first location in the nervous system and a second neural signal recorded simultaneously at a second location in the nervous system different from the first location.
5. The method of any one of examples 1-4 wherein the measurable output is sensory or motor behavior of the subject, and wherein the error signal is based on a difference between a measured sensory or motor behavior of the subject and a desired sensory or motor behavior of the subject.
6. The method of any one of examples 1-5 wherein the measurable output is a neural activity pattern, and wherein the error signal is based on a difference between the neural activity pattern and a desired neural activity pattern.
7. The method of example 6 wherein the neural activity pattern is in a region of the brain associated with sensory, motor, or other behavior.
8. The method of any one of examples 1-7 wherein the measurable output is a reward signal measured at a reward center of the brain of the subject.
9. The method of any one of examples 1-8 wherein the neural model is at least one of an artificial neural network model, a generalized linear model, a logistic regression, a polynomial regression, a Gaussian process regression, or other machine learning method for function approximation, and wherein adjusting the at least one parameter of the neural model includes adjusting a weight or value of one or more variables defining the neural model.
10. The method of any one of examples 1-9, further comprising:
11. The method of any one of examples 1-10 wherein the neural signals are first neural signals, wherein the external signals are first external signals, wherein the stimulation pattern is a first stimulation pattern, wherein the measurable output is a first measurable output, wherein the error signal is a first error signal, and wherein the adjusted neural model is a first adjusted neural model, and wherein the method further comprises:
12. A system for inducing new neural connections and restoring lost neural function in a nervous system of a human patient, the system comprising:
13. The system of example 12 wherein the computing device is implanted in and/or worn by the human patient.
14. The system of example 12 or 13 wherein the injured region is an injured region of the spinal cord of the human patient, wherein the first region is a motor-intention forming region of the brain of the human patient, wherein the second region is a region of the spinal cord different from the injured region, and wherein the system is configured to promote neural plasticity between the first and second regions.
15. The system of any one of examples 12-14 wherein the injured region is an injured region of the brain of the human patient caused by a stroke, traumatic brain injury, or disease, wherein the first region and the second region are regions of the brain disconnected by the injured region, and wherein the system is configured to promote neural plasticity between the first and second regions.
16. The system of any one of examples 12-15 wherein the measureable patient output is external to the nervous system.
17. The system of any one of examples 12-16 wherein the measureable patient output is internal to the nervous system.
18. A system for augmenting a neural function of a nervous system of a human subject, the system comprising:
19. The system of example 18 wherein the information source is the Internet or another network, and wherein the computing device is configured for wireless communication with the information source.
20. The system of example 18 or 19 wherein the region of the nervous system related to the neural function to be augmented is the prefrontal cortex of the brain of the subject, wherein the augmentation region is a motor region of the brain, and wherein the computing device is configured to augment and/or accelerate the ability of the subject to store a short-term memory, plan, or learn a motor skill.
21. The system of any one of examples 18-20 wherein the first sensor, the second sensor, and the computing device are implanted in and/or worn by the human subject.
The above detailed descriptions of embodiments of the technology are not intended to be exhaustive or to limit the technology to the precise form disclosed above. Although specific embodiments of, and examples for, the technology are described above for illustrative purposes, various equivalent modifications are possible within the scope of the technology, as those skilled in the relevant art will recognize. For example, while steps are presented in a given order, alternative embodiments may perform steps in a different order. Moreover, the various embodiments described herein may also be combined to provide further embodiments (e.g., the disclosed system may include components for simultaneous augmentation and restoration of function in a nervous system of a subject).
Moreover, unless the word “or” is expressly limited to mean only a single item exclusive from the other items in reference to a list of two or more items, then the use of “or” in such a list is to be interpreted as including (a) any single item in the list, (b) all of the items in the list, or (c) any combination of the items in the list. Where the context permits, singular or plural terms may also include the plural or singular term, respectively. Additionally, the term “comprising” is used throughout to mean including at least the recited feature(s) such that any greater number of the same feature and/or additional types of other features are not precluded. It will also be appreciated that specific embodiments have been described herein for purposes of illustration, but that various modifications may be made without deviating from the technology. Further, while advantages associated with certain embodiments of the technology have been described in the context of those embodiments, other embodiments may also exhibit such advantages, and not all embodiments need necessarily exhibit such advantages to fall within the scope of the technology. Accordingly, the disclosure and associated technology can encompass other embodiments not expressly shown or described herein.
The present application claims priority to U.S. Provisional Application No. 62/366,951, filed Jul. 26, 2016, and titled NEURAL CO-PROCESSOR FOR RESTORATION AND AUGMENTATION OF BRAIN FUNCTION, which is incorporated herein by reference in its entirety.
This invention was made with government support under Grant No. EEC-1028725, awarded by the National Science Foundation, and Grant No. R01 NS012542, awarded by the National Institutes of Health. The government has certain rights in the invention.
Filing Document | Filing Date | Country | Kind |
---|---|---|---|
PCT/US17/44012 | 7/26/2017 | WO | 00 |
Number | Date | Country | |
---|---|---|---|
62366951 | Jul 2016 | US |