Patent Application
20220387748
The present invention generally relates to the field of neuromodulation, and more specifically to systems and methods for selectively replicating desired mental states in a human or an animal.
Mental State. A mental state is a state of mind that a subject is in. Some mental states are pure and unambiguous, while humans are capable of complex states that are a combination of mental representations, which may have in their pure state contradictory characteristics. There are several paradigmatic states of mind that a subject has: love, hate, pleasure, fear, and pain. Mental states can also include a waking state, a sleeping state, a flow (or being in the “zone”), and a mood (an emotional state). A mental state is a hypothetical state that corresponds to thinking and feeling, and consists of a conglomeration of mental representations. A mental state is related to an emotion, though it can also relate to cognitive processes. Because the mental state itself is complex and potentially possess inconsistent attributes, clear interpretation of mental state through external analysis (other than self-reporting) is difficult or impossible. However, a number of studies report that certain attributes of mental state or thought processes may in fact be determined through passive monitoring, such as EEG, with some degree of statistical reliability. In most studies, the characterization of mental state was an endpoint, and the raw signals, after statistically classification or semantic labelling, are superseded and the remaining signal energy treated as noise. Current technology does not permit an accurate or precise abstract encoding or characterization of the full range of mental states based on neural correlates of mental state.
Brain The brain is a key part of the central nervous system, enclosed in the skull. In humans, and mammals more generally, the brain controls both autonomic processes, as well as cognitive processes. The brain (and to a lesser extent, the spinal cord) controls all volitional functions of the body and interprets information from the outside world. Intelligence, memory, emotions, speech, thoughts, movements and creativity are controlled by the brain. The central nervous system also controls autonomic functions and many homeostatic and reflex actions, such as breathing, heart rate, etc.
The human brain consists of the cerebrum, cerebellum, and brainstem. The brainstem includes the midbrain, the pons, and the medulla oblongata. Sometimes the diencephalon, the caudal part of the forebrain, is included.
The brainstem provides the main motor and sensory innervation to the face and neck via the cranial nerves. Of the twelve pairs of cranial nerves, ten pairs come from the brainstem. This is an extremely important part of the brain, as the nerve connections of the motor and sensory systems from the main part of the brain to the rest of the body pass through the brainstem. This includes the corticospinal tract (motor), the posterior column-medial lemniscus pathway (fine touch, vibration sensation, and proprioception), and the spinothalamic tract (pain, temperature, itch, and crude touch). The brainstem also plays an important role in the regulation of cardiac and respiratory function. It also regulates the central nervous system and is pivotal in maintaining consciousness and regulating the sleep cycle. The brainstem has many basic functions including heart rate, breathing, sleeping, and eating.
The function of the skull is to protect delicate brain tissue from injury. The skull consists of eight fused bones: the frontal, 2 parietal, 2 temporal, sphenoid, occipital and ethmoid. The face is formed by 14 paired bones including the maxilla, zygoma, nasal, palatine, lacrimal, inferior nasal conchae, mandible, and vomer. The bony skull is separated from the brain by the dura, a membranous organ, which in turn contains cerebrospinal fluid. The cortical surface of the brain typically is not subject to localized pressure from the skull. The skull therefore imposes a barrier to electrical access to the brain functions, and in a healthy human, breaching the dura to access the brain is highly disfavored. The result is that electrical readings of brain activity are filtered by the dura, the cerebrospinal fluid, the skull, the scalp, skin appendages (e.g., hair), resulting in a loss of potential spatial resolution and amplitude of signals emanating from the brain. While magnetic fields resulting from brain electrical activity are accessible, the spatial resolution using feasible sensors is also limited.
The cerebrum is the largest part of the brain and is composed of right and left hemispheres. It performs higher functions, such as interpreting inputs from the senses, as well as speech, reasoning, emotions, learning, and fine control of movement. The surface of the cerebrum has a folded appearance called the cortex. The human cortex contains about 70% of the nerve cells (neurons) and gives an appearance of gray color (grey matter). Beneath the cortex are long connecting fibers between neurons, called axons, which make up the white matter.
The cerebellum is located behind the cerebrum and brainstem. It coordinates muscle movements, helps to maintain balance and posture. The cerebellum may also be involved in some cognitive functions such as attention and language, as well as in regulating fear and pleasure responses. There is considerable evidence that the cerebellum plays an essential role in some types of motor learning. The tasks where the cerebellum most clearly comes into play are those in which it is necessary to make fine adjustments to the way an action is performed. There is dispute about whether learning takes place within the cerebellum itself, or whether it merely serves to provide signals that promote learning in other brain structures.
The brain communicates with the body through the spinal cord and twelve pairs of cranial nerves. Ten of the twelve pairs of cranial nerves that control hearing, eye movement, facial sensations, taste, swallowing and movement of the face, neck, shoulder and tongue muscles originate in the brainstem. The cranial nerves for smell and vision originate in the cerebrum.
The right and left hemispheres of the brain are joined by a structure consisting of fibers called the corpus callosum. Each hemisphere controls the opposite side of the body. Not all functions of the hemispheres are shared.
The cerebral hemispheres have distinct structures, which divide the brain into lobes. Each hemisphere has four lobes: frontal, temporal, parietal, and occipital. There are very complex relationships between the lobes of the brain and between the right and left hemispheres: Frontal lobes control judgment, planning, problem-solving, behavior, emotions, personality, speech, self-awareness, concentration, intelligence, body movements; Temporal lobes control understanding of language, memory, organization and hearing; Parietal lobes control interpretation of language; input from vision, hearing, sensory, and motor; temperature, pain, tactile signals, memory, spatial and visual perception; and Occipital lobes interpret visual input (movement, light, color).
Neurons A neuron is a fundamental unit of the nervous system, which comprises the autonomic nervous system and the central nervous system.
Neurons are electrically excitable cells that receive, process, and transmit information, and based on that information sends a signal to other neurons, muscles, or glands through electrical and chemical signals. These signals between neurons occur via specialized connections called synapses. Neurons can connect to each other to form neural networks. The basic purpose of a neuron is to receive incoming information and, based upon that information send a signal to other neurons, muscles, or glands. Neurons are designed to rapidly send signals across physiologically long distances. They do this using electrical signals called nerve impulses or action potentials. When a nerve impulse reaches the end of a neuron, it triggers the release of a chemical, or neurotransmitter. The neurotransmitter travels rapidly across the short gap between cells (the synapse) and acts to signal the adjacent cell. See www.biologyreference.com/Mo-Nu/Neuron.html#ixzz5AVxCuM5a.
Neurons can receive thousands of inputs from other neurons through synapses. Synaptic integration is a mechanism whereby neurons integrate these inputs before the generation of a nerve impulse, or action potential. The ability of synaptic inputs to effect neuronal output is determined by a number of factors:
Size, shape and relative timing of electrical potentials generated by synaptic inputs;
the geometric structure of the target neuron;
the physical location of synaptic inputs within that structure; and
the expression of voltage-gated channels in different regions of the neuronal membrane.
Neurons within a neural network receive information from, and send information to, many other cells, at specialized junctions called synapses. Synaptic integration is the computational process by which an individual neuron processes its synaptic inputs and converts them into an output signal. Synaptic potentials occur when neurotransmitter binds to and opens ligand-operated channels in the dendritic membrane, allowing ions to move into or out of the cell according to their electrochemical gradient. Synaptic potentials can be either excitatory or inhibitory depending on the direction and charge of ion movement. Action potentials occur if the summed synaptic inputs to a neuron reach a threshold level of depolarization and trigger regenerative opening of voltage-gated ion channels. Synaptic potentials are often brief and of small amplitude, therefore summation of inputs in time (temporal summation) or from multiple synaptic inputs (spatial summation) is usually required to reach action potential firing threshold.
There are two types of synapses: electrical synapses and chemical synapses. Electrical synapses are a direct electrical coupling between two cells mediated by gap junctions, which are pores constructed of connexin proteins—essentially result in the passing of a gradient potential (may be depolarizing or hyperpolarizing) between two cells. Electrical synapses are very rapid (no synaptic delay). It is a passive process where signal can degrade with distance and may not produce a large enough depolarization to initiate an action potential in the postsynaptic cell. Electrical synapses are bidirectional, i.e., postsynaptic ell can actually send messages to the “presynaptic cell.
Chemical synapses are a coupling between two cells through neuro-transmitters, ligand or voltage gated channels, receptors. They are influenced by the concentration and types of ions on either side of the membrane. Among the neurotransmitters, Glutamate, sodium, potassium, and calcium are positively charged. GABA and chloride are negatively charged. Neurotransmitter junctions provide an opportunity for pharmacological intervention, and many different drugs, including illicit drugs, act at synapses.
An excitatory postsynaptic potential (EPSP) is a postsynaptic potential that makes the postsynaptic neuron more likely to fire an action potential. An electrical charge (hyperpolarization) in the membrane of a postsynaptic neuron is caused by the binding of an inhibitory neurotransmitter from a presynaptic ell to a postsynaptic receptor. It makes it more difficult for a postsynaptic neuron to generate an action potential. An electrical change (depolarization) in the membrane of a postsynaptic neuron caused by the binding of an excitatory neurotransmitter from a presynaptic ell to a postsynaptic receptor. It makes it more likely for a postsynaptic neuron to generate an action potential. In a neuronal synapse that uses glutamate as receptor, for example, receptors open ion channels that are non-selectively permeable to cations. When these glutamate receptors are activated, both Na+ and K+ flow across the postsynaptic membrane. The reversal potential (Erev) for the post-synaptic current is approximately 0 mV. The resting potential of neurons is approximately −60 mV. The resulting EPSP will depolarize the post synaptic membrane potential, bringing it toward 0 mV.
An inhibitory postsynaptic potential (IPSP) is a kind of synaptic potential that makes a postsynaptic neuron less likely to generate an action potential. An example of inhibitory post synaptic s action is a neuronal synapse that uses gamma-Aminobutyric acid (γ-Aminobutyric acid, or GABA) as its transmitter. At such synapses, the GABA receptors typically open channels that are selectively permeable to Cl−. When these channels open, negatively charged chloride ions can flow across the membrane. The postsynaptic neuron has a resting potential of −60 mV and an action potential threshold of −40 mV. Transmitter release at this synapse will inhibit the postsynaptic cell. Since Ea is more negative than the action potential threshold, e.g., −70 mV, it reduces the probability that the postsynaptic ell will fire an action potential.
Some types of neurotransmitters, such as glutamate, consistently result in EPSPs. Others, such as GABA, consistently result in IPSPs. The action potential lasts about one millisecond (1 msec). In contrast, the EPSPs and IPSPs can last as long as 5 to 10 msec. This allows the effect of one postsynaptic potential to build upon the next and so on.
Membrane leakage, and to a lesser extent, potentials per se, can be influenced by external electrical and magnetic fields. These fields may be generated focally, such as through implanted electrodes, or less specifically, such as through transcranial stimulation. Transcranial stimulation may be subthreshold or suprathreshold. In the former case, the external stimulation acts to modulate resting membrane potential, making nerves more or less excitable. Such stimulation may be direct current or alternating current. In the latter case, this will tend to synchronize neuron depolarization with the signals. Suprathreshold stimulation can be painful (at least because the stimulus directly excites pain neurons) and must be pulsed. Since this has correspondence to electroconvulsive therapy, suprathreshold transcranial stimulation is sparingly used.
Neural Correlates A neural correlate of a mental state is an electro-neuro-biological state or the state assumed by some biophysical subsystem of the brain, whose presence necessarily and regularly correlates with such specific mental state. All properties credited to the mind, including consciousness, emotion, and desires are thought to have direct neural correlates. For our purposes, neural correlates of a mental state can be defined as the minimal set of neuronal oscillations that correspond to the given mental state. Neuroscientists use empirical approaches to discover neural correlates of subjective mental states.
Brainwaves At the root of all our thoughts, emotions and behaviors is the communication between neurons within our brains, a rhythmic or repetitive neural activity in the central nervous system. The oscillation can be produced by a single neuron or by synchronized electrical pulses from ensembles of neurons communicating with each other. The interaction between neurons can give rise to oscillations at a different frequency than the firing frequency of individual neurons. The synchronized activity of large numbers of neurons produces macroscopic oscillations, which can be observed in an electroencephalogram. They are divided into bandwidths to describe their purported functions or functional relationships. Oscillatory activity in the brain is widely observed at different levels of organization and is thought to play a key role in processing neural information. Numerous experimental studies support a functional role of neural oscillations. A unified interpretation, however, is still not determined. Neural oscillations and synchronization have been linked to many cognitive functions such as information transfer, perception, motor control and memory. Electroencephalographic (EEG) signals are relatively easy and safe to acquire, have a long history of analysis, and can have high dimensionality, e.g., up to 128 or 256 separate recording electrodes. While the information represented in each electrode is not independent of the others, and the noise in the signals high, there is much information available through such signals that has not been fully characterized to date.
Brain waves have been widely studied in neural activity generated by large groups of neurons, mostly by EEG. In general, EEG signals reveal oscillatory activity in specific frequency bands: alpha (7.5-12.5 Hz) that can be detected from the occipital lobe during relaxed wakefulness and which increases when the eyes are closed; delta (1-4 Hz), theta (4-8 Hz), beta (13-30 Hz), low gamma (30-70 Hz), and high gamma (70-150 Hz) frequency bands, where faster rhythms such as gamma activity have been linked to cognitive processing. Neural oscillations of specific characteristics have been linked to cognitive states, such as awareness and consciousness and different sleep stages. It is a useful analogy to think of brainwaves as musical notes. Like in symphony, the higher and lower frequencies link and cohere with each other through harmonics. Oscillatory activity is observed throughout the central nervous system at all levels of organization. The dominant neuro oscillation frequency determines a mental state.
The functions of brain waves are wide-ranging and vary for different types of oscillatory activity. Neural oscillations also play an important role in many neurological disorders.
In standard EEG recording practice, 19 recording electrodes are placed uniformly on the scalp (the International 10-20 System). In addition, one or two reference electrodes (often placed on earlobes) and a ground electrode (often placed on the nose to provide amplifiers with reference voltages) are required. However, additional electrodes may add minimal useful information unless supplemented by computer algorithms to reduce raw EEG data to a manageable form. When large numbers of electrodes are employed, the potential at each location may be measured with respect to the average of all potentials (the common average reference), which often provides a good estimate of potential at infinity. The common average reference is not appropriate when electrode coverage is sparse (perhaps less than 64 electrodes. Dipole localization algorithms may be useful to determine spatial emission patterns in EEG.
Scalp potential may be expressed as a volume integral of dipole moment per unit volume over the entire brain provided P(r,t) is defined generally rather than in columnar terms. For the important case of dominant cortical sources, scalp potential may be approximated by the following integral over the cortical volume Θ, VS(r,t)=∫∫∫ΘG(r,r′)−P(r′,t)dΘ(r′). If the volume element dΘ(r′) is defined in terms of cortical columns, the volume integral may be reduced to an integral over the folded cortical surface. The time-dependence of scalp potential is the weighted sum of all dipole time variations in the brain, although deep dipole volumes typically make negligible contributions. The vector Green's function G(r,r′) contains all geometric and conductive information about the head volume conductor and weights the integral accordingly. Thus, each scalar component of the Green's function is essentially an inverse electrical distance between each source component and scalp location. For the idealized case of sources in an infinite medium of constant conductivity, the electrical distance equals the geometric distance. The Green's function accounts for the tissue's finite spatial extent and its inhomogeneity and anisotropy. The forward problem in EEG consists of choosing a head model to provide G(r,r′) and carrying out the integral for some assumed source distribution. The inverse problem consists of using the recorded scalp potential distribution VS(r,t) plus some constraints (usual assumptions) on P(r,t) to find the best fit source distribution P(r,t). Since the inverse problem has no unique solution, any inverse solution depends critically on the chosen constraints, for example, only one or two isolated sources, distributed sources confined to the cortex, or spatial and temporal smoothness criteria. High-resolution EEG uses the experimental scalp potential VS(r,t) to predict the potential on the dura surface (the unfolded membrane surrounding the cerebral cortex) VD(r,t). This may be accomplished using a head model Green's function G(r,r′) or by estimating the surface Laplacian with either spherical or 3D splines. These two approaches typically provide very similar dura potentials VD(r,t); the estimates of dura potential distribution are unique subject to head model, electrode density, and noise issues.
In an EEG recording system, each electrode is connected to one input of a differential amplifier (one amplifier per pair of electrodes); a common system reference electrode (or synthesized reference) is connected to the other input of each differential amplifier. These amplifiers amplify the voltage between the active electrode and the reference (typically 1,000-100,000 times, or 60-100 dB of voltage gain). The amplified signal is digitized via an analog-to-digital converter, after being passed through an anti-aliasing filter. Analog-to-digital sampling typically occurs at 256-512 Hz in clinical scalp EEG; sampling rates of up to 20 kHz are used in some research applications. The EEG signals can be captured with open source hardware such as OpenBCI, and the signal can be processed by freely available EEG software such as EEGLAB or the Neurophysiological Biomarker Toolbox. A typical adult human EEG signal is about 10 μV to 100 μV in amplitude when measured from the scalp and is about 10-20 mV when measured from subdural electrodes.
Delta is the frequency range up to 4 Hz. It tends to be the highest in amplitude and the slowest waves. It is normally seen in adults in slow-wave sleep. It is also seen normally in babies. It may occur focally with subcortical lesions and in general distribution with diffuse lesions, metabolic encephalopathy hydrocephalus or deep midline lesions. It is usually most prominent frontally in adults (e.g., FIRDA-frontal intermittent rhythmic delta) and posteriorly in children (e.g., OIRDA-occipital intermittent rhythmic delta).
Theta is the frequency range from 4 Hz to 7 Hz. Theta is normally seen in young children. It may be seen in drowsiness or arousal in older children and adults; it can also be seen in meditation. Excess theta for age represents abnormal activity. It can be seen as a focal disturbance in focal subcortical lesions; it can be seen in generalized distribution in diffuse disorder or metabolic encephalopathy or deep midline disorders or some instances of hydrocephalus. On the contrary, this range has been associated with reports of relaxed, meditative, and creative states.
Alpha is the frequency range from 7 Hz to 14 Hz. This was the “posterior basic rhythm” (also called the “posterior dominant rhythm” or the “posterior alpha rhythm”), seen in the posterior regions of the head on both sides, higher in amplitude on the dominant side. It emerges with the closing of the eyes and with relaxation and attenuates with eye opening or mental exertion. The posterior basic rhythm is actually slower than 8 Hz in young children (therefore technically in the theta range). In addition to the posterior basic rhythm, there are other normal alpha rhythms such as the sensorimotor, or mu rhythm (alpha activity in the contralateral sensory and motor cortical areas) that emerges when the hands and arms are idle; and the “third rhythm” (alpha activity in the temporal or frontal lobes). Alpha can be abnormal; for example, an EEG that has diffuse alpha occurring in coma and is not responsive to external stimuli is referred to as “alpha coma.”
Beta is the frequency range from 15 Hz to about 30 Hz. It is usually seen on both sides in symmetrical distribution and is most evident frontally. Beta activity is closely linked to motor behavior and is generally attenuated during active movements. Low-amplitude beta with multiple and varying frequencies is often associated with active, busy or anxious thinking and active concentration. Rhythmic beta with a dominant set of frequencies is associated with various pathologies, such as Dup15q syndrome, and drug effects, especially benzodiazepines. It may be absent or reduced in areas of cortical damage. It is the dominant rhythm in patients who are alert or anxious or who have their eyes open.
Gamma is the frequency range approximately 30-100 Hz. Gamma rhythms are thought to represent binding of different populations of neurons together into a network to carry out a certain cognitive or motor function.
Mu range is 8-13 Hz and partly overlaps with other frequencies. It reflects the synchronous firing of motor neurons in a rest state. Mu suppression is thought to reflect motor mirror neuron systems, because when an action is observed, the pattern extinguishes, possibly because of the normal neuronal system and the mirror neuron system “go out of sync” and interfere with each other. (en.wikipedia.org/wiki/Electroencephalography)
Neurodynamics Neurodynamics is the mobilization of the nervous system as an approach to physical treatment. The method relies on influencing pain and other neural physiology via mechanical treatment of neural tissues and the non-neural structures surrounding the nervous system. The body presents the nervous system with a mechanical interface via the musculoskeletal system. With movement, the musculoskeletal system exerts non-uniform stresses and movement in neural tissues, depending on the local anatomical and mechanical characteristics and the pattern of body movement. This activates an array of mechanical and physiological responses in neural tissues. These responses include neural sliding, pressurization, elongation, tension and changes in intraneural microcirculation, axonal transport and impulse traffic.
EEG and qEEG EEG (electroencephalography) and MEG (magnetoencephalography) are available technologies to monitor brain electrical activity. Each generally has sufficient temporal resolution to follow dynamic changes in brain electrical activity. Electroencephalography (EEG) and quantitative electroencephalography (gEEG) are electrophysiological monitoring methods that analyze the electrical activity of the brain to measure and display patterns that correspond to cognitive states and/or diagnostic information. It is typically noninvasive, with the electrodes placed on the scalp, although invasive electrodes are also used in some cases. EEG signals may be captured and analyzed by a mobile device, often referred as “brain wearables”. There are a variety of “brain wearables” readily available on the market today. EEGs can be obtained with a non-invasive method where the aggregate oscillations of brain electric potentials are recorded with numerous electrodes attached to the scalp of a person. Most EEG signals originate in the brain's outer layer (the cerebral cortex), believed largely responsible for our thoughts, emotions, and behavior. Cortical synaptic action generates electrical signals that change in the 10 to 100-millisecond range. Transcutaneous EEG signals are limited by the relatively insulating nature of the skull surrounding the brain, the conductivity of the cerebrospinal fluid and brain tissue, relatively low amplitude of individual cellular electrical activity, and distances between the cellular current flows and the electrodes. EEG is characterized by: (1) Voltage; (2) Frequency; (3) Spatial location; (4) Inter-hemispheric symmetries; (5) Reactivity (reaction to state change); (6) Character of waveform occurrence (random, serial, continuous); and (7) Morphology of transient events. EEGs can be separated into two main categories. Spontaneous EEG which detect brainwaves that occur in the absence of specific sensory stimuli and evoked potentials (EPs) which are associated with sensory stimuli like repeated light flashes, auditory tones, finger pressure or mild electric shocks. The latter is recorded for example by time averaging to remove effects of spontaneous EEG. Non-sensory triggered potentials are also known. EP's typically are time synchronized with the trigger, and thus have an organization principle. Event-related potentials (ERPs) provide evidence of a direct link between cognitive events and brain electrical activity in a wide range of cognitive paradigms. It has generally been held that an ERP is the result of a set of discrete stimulus-evoked brain events. Event-related potentials (ERPs) are recorded in the same way as EPs, but occur at longer latencies from the stimuli and are more associated with an endogenous brain state.
EEG-based studies of emotional specificity at the single-electrode level demonstrated that asymmetric activity at the frontal site, especially in the alpha (8-12 Hz) band, is associated with emotion. Voluntary facial expressions of smiles of enjoyment produce higher left frontal activation. Decreased left frontal activity is observed during the voluntary facial expressions of fear. In addition to alpha band activity, theta band power at the frontal midline (Fm) has also been found to relate to emotional states. Pleasant (as opposed to unpleasant) emotions are associated with an increase in frontal midline theta power. Many studies have sought to utilize pattern classification, such as neural networks, statistical classifiers, clustering algorithms, etc., to differentiate between various emotional states reflected in EEG.
Detecting different emotional states by EEG may be more appropriate using EEG-based functional connectivity. There are various ways to estimate EEG-based functional brain connectivity: correlation, coherence and phase synchronization indices between each pair of EEG electrodes had been used. The assumption is that a higher correlation map indicates a stronger relationship between two signals. Coherence gives information similar to correlation, but also includes the covariation between two signals as a function of frequency. The assumption is that higher correlation indicates a stronger relationship between two signals. Phase synchronization among the neuronal groups estimated based on the phase difference between two signals is another way to estimate the EEG-based functional connectivity among brain areas.
A number of groups have examined emotional specificity using EEG-based functional brain connectivity. When emotional states become more negative at high room temperatures, correlation coefficients between the channels in temporal and occipital sites increase. Coherence decreases in the alpha band during sadness, compared to happiness. EEG coherence between the prefrontal cortex and the posterior cortex increases while viewing highly emotionally arousing (i.e., threatening) images, compared to viewing neutral images. A synchronization index may be applied to detect interaction in different brain sites under different emotional states. Test results showed an overall increase in the synchronization index among frontal channels during emotional stimulation, particularly during negative emotion (i.e., sadness). Furthermore, phase synchronization patterns were found to differ between positive and negative emotions. Sadness was more synchronized than happiness at each frequency band and was associated with a wider synchronization both between the right and left frontal sites and within the left hemisphere. In contrast, happiness was associated with a wider synchronization between the frontal and occipital sites.
Different connectivity indices are sensitive to different characteristics of EEG signals. Correlation is sensitive to phase and polarity, but is independent of amplitudes. Changes in both amplitude and phase lead to a change in coherence. The phase synchronization index is only sensitive to a change in phase. A number of studies have tried to classify emotional states by means of recording and statistically analyzing EEG signals from the central nervous systems.
These emotional states are based on the dimensional theory of emotion, which asserts that there are neutral, positive, and negative emotional states, because numerous studies have suggested that the responses of the central nervous system correlate with emotional valence and arousal. EEG-based functional connectivity change is found to be significantly different among emotional states of neutral, positive, or negative. Furthermore, the connectivity pattern was detected by pattern classification analysis using Quadratic Discriminant Analysis.
To assess a user's state of mind, a computer may be used to analyze the EEG signals produced by the brain of the user. However, the emotional states of a brain are complex, and the brain waves associated with specific emotions seem to change over time. Machine learning may be used to reliably identify the emotional brain states. A set of patterns may be defined that clearly distinguish positive, negative, and neutral emotions.
MEG Magnetoencephalography (MEG) is a functional neuroimaging technique for mapping brain activity by recording magnetic fields produced by electrical currents occurring naturally in the brain, using very sensitive magnetometers. Arrays of SQUIDs (superconducting quantum interference devices) are currently the most common magnetometer, while the SERF (spin exchange relaxation-free) magnetometer is being investigated. MEGs seek to detect the magnetic dipole emission from an electrical discharge in cells, e.g., neural action potentials. Typical sensors for MEGs are superconducting quantum interference devices (SQUIDs). These currently require cooling to liquid nitrogen or liquid helium temperatures and, therefore, are currently suitable only for laboratory environments. However, the development of room temperature, or near room temperature superconductors, and miniature cryocoolers, may permit field deployments and portable or mobile detectors. Because MEGs are less influenced by medium conductivity and dielectric properties, and because they inherently detect the magnetic field vector, MEG technology permits volumetric mapping of brain activity and distinction of complementary activity that might suppress detectable EEG signals. MEG technology also supports vector mapping of fields, since magnetic emitters are inherently dipoles, and therefore a larger amount of information is inherently available.
Functional near infrared spectroscopy (fNIRS) fNIR is a non-invasive imaging method involving the quantification of chromophore concentration resolved from the measurement of near infrared (NIR) light attenuation or temporal or phasic changes. NIR spectrum light takes advantage of the optical window in which skin, tissue, and bone are mostly transparent to NIR light in the spectrum of 700-900 nm, while hemoglobin (Hb) and deoxygenated-hemoglobin (deoxy-Hb) are stronger absorbers of light. Differences in the absorption spectra of deoxy-Hb and oxy-Hb allow the measurement of relative changes in hemoglobin concentration through the use of light attenuation at multiple wavelengths. Two or more wavelengths are selected, with one wavelength above and one below the isosbestic point of 810 nm at which deoxy-Hb and oxy-Hb have identical absorption coefficients. Using the modified Beer-Lambert law (mBLL), relative concentration can be calculated as a function of total photon path length. Typically the light emitter and detector are placed ipsilaterally on the subjects skull so recorded measurements are due to back-scattered (reflected) light following elliptical pathways. The use of fNIR as a functional imaging method relies on the principle of neuro-vascular coupling also known as the hemodynamic response or blood-oxygen-level dependent (BOLD) response. This principle also forms the core of fMRI techniques. Through neuro-vascular coupling, neuronal activity is linked to related changes in localized cerebral blood flow. fNIR and fMRI are sensitive to similar physiologic changes and are often comparative methods. Studies relating fMRI and fNIR show highly correlated results in cognitive tasks. fNIR has several advantages in cost and portability over fMRI, but cannot be used to measure cortical activity more than 4 cm deep due to limitations in light emitter power and has more limited spatial resolution. fNIR includes the use of diffuse optical tomography (DOT/NIRDOT) for functional purposes. Multiplexing fNIRS channels can allow 2D topographic functional maps of brain activity (e.g., with Hitachi ETG-4000 or Artinis Oxymon) while using multiple emitter spacings may be used to build 3D tomographic maps.
Neuromodulation/neuroenhancement Neuromodulation is the alteration of nerve activity through targeted delivery of a stimulus, such as electrical stimulation or chemical agents, to specific neurological sites in the body. It is carried out to normalize—or modulate—nervous tissue function. Neuromodulation is an evolving therapy that can involve a range of electromagnetic stimuli such as a magnetic field (rTMS), an electric current, or a drug instilled directly in the subdural space (intrathecal drug delivery). Emerging applications involve targeted introduction of genes or gene regulators and light (optogenetics), and by 2014, these had been at minimum demonstrated in mammalian models, or first-in-human data had been acquired. The most clinical experience has been with electrical stimulation. Neuromodulation, whether electrical or magnetic, employs the body's natural biological response by stimulating nerve cell activity that can influence populations of nerves by releasing transmitters, such as dopamine, or other chemical messengers such as the peptide Substance P, that can modulate the excitability and firing patterns of neural circuits. There may also be more direct electrophysiological effects on neural membranes as the mechanism of action of electrical interaction with neural elements. The end effect is a “normalization” of a neural network function from its perturbed state. Presumed mechanisms of action for neurostimulation include depolarizing blockade, stochastic normalization of neural firing, axonal blockade, reduction of neural firing keratosis, and suppression of neural network oscillations. Although the exact mechanisms of neurostimulation are not known, the empirical effectiveness has led to considerable application clinically.
Neuroenhancement refers to the targeted enhancement and extension of cognitive and affective abilities based on an understanding of their underlying neurobiology in healthy persons who do not have any mental illness. As such, it can be thought of as an umbrella term that encompasses pharmacological and non-pharmacological methods of improving cognitive, affective, and motor functionality, as well as the overarching ethico-legal discourse that accompanies these aims. Critically, for any agent to qualify as a neuroenhancer, it must reliably engender substantial cognitive, affective, or motor benefits beyond normal functioning in healthy individuals, whilst causing few side effects: at most at the level of commonly used comparable legal substances or activities, such as caffeine, alcohol, and sleep-deprivation. Pharmacological neuroenhancement agents include the well-validated nootropics, such as racetam, vinpocetine, and phosphatidylserine, as well as other drugs used for treating patients suffering from neurological disorders. Non-pharmacological measures include non-invasive brain stimulation, which has been employed to improve various cognitive and affective functions, and brain-machine interfaces, which hold much potential to extend the repertoire of motor and cognitive actions available to humans.
Brain-to-Brain interface A brain-brain interface is a direct communication pathway between the brain of one animal and the brain of another animal. Brain to brain interfaces have been used to help rats collaborate with each other. When a second rat was unable to choose the correct lever, the first rat noticed (not getting a second reward), and produced a round of task-related neuron firing that made the second rat more likely to choose the correct lever. Human studies have also been conducted. In 2013, researcher from the University of Washington were able to use electrical brain recordings and a form of magnetic stimulation to send a brain signal to a recipient, which caused the recipient to hit the fire button on a computer game. In 2015, researchers linked up multiple brains, of both monkeys and rats, to form an “organic computer.” It is hypothesized that by using brain-to-brain interfaces (BTBIs) a biological computer, or brain-net, could be constructed using animal brains as its computational units. Initial exploratory work demonstrated collaboration between rats in distant cages linked by signals from cortical microelectrode arrays implanted in their brains. The rats were rewarded when actions were performed by the “decoding rat” which conformed to incoming signals and when signals were transmitted by the “encoding rat” which resulted in the desired action. In the initial experiment the rewarded action was pushing a lever in the remote location corresponding to the position of a lever near a lighted LED at the home location. About a month was required for the rats to acclimate themselves to incoming “brainwaves.”
Brain-to-Computer interface A brain-computer interface (BCI), sometimes called a neural-control interface (NCI), mind-machine interface (MMI), direct neural interface (DNI), or brain-machine interface (BMI), is a direct communication pathway between an enhanced or wired brain and an external device. BCI differs from neuromodulation in that it allows for bidirectional information flow. BCIs are often directed at researching, mapping, assisting, augmenting, or repairing human cognitive or sensory-motor functions.
Brain entrainment Brainwave entrainment, also referred to as brainwave synchronization and neural entrainment, refers to the capacity of the brain to naturally synchronize its brainwave frequencies with the rhythm of periodic external stimuli, most commonly auditory, visual, or tactile. It is hypothesized that listening to these beats of certain frequencies one can induce a desired state of consciousness that corresponds with specific neural activity. It is widely accepted that patterns of neural firing, measured in Hz, correspond with states of alertness such as focused attention, deep sleep, etc.
Neural oscillations are rhythmic or repetitive electrochemical activity in the brain and central nervous system. Such oscillations can be characterized by their frequency, amplitude and phase. Neural tissue can generate oscillatory activity driven by mechanisms within individual neurons, as well as by interactions between them. They may also adjust frequency to synchronize with the periodic vibration of external acoustic or visual stimuli. The functional role of neural oscillations is still not fully understood; however, they have been shown to correlate with emotional responses, motor control, and a number of cognitive functions including information transfer, perception, and memory. Specifically, neural oscillations, in particular theta activity, are extensively linked to memory function, and coupling between theta and gamma activity is considered to be vital for memory functions, including episodic memory. Electroencephalography (EEG) has been most widely used in the study of neural activity generated by large groups of neurons, known as neural ensembles, including investigations of the changes that occur in electroencephalographic profiles during cycles of sleep and wakefulness. EEG signals change dramatically during sleep and show a transition from faster frequencies to increasingly slower frequencies, indicating a relationship between the frequency of neural oscillations and cognitive states including awareness and consciousness.
The term ‘entrainment’ has been used to describe a shared tendency of many physical and biological systems to synchronize their periodicity and rhythm through interaction. This tendency has been identified as specifically pertinent to the study of sound and music generally, and acoustic rhythms specifically. The most ubiquitous and familiar examples of neuromotor entrainment to acoustic stimuli is observable in spontaneous foot or finger tapping to the rhythmic beat of a song. Exogenous rhythmic entrainment, which occurs outside the body, has been identified and documented for a variety of human activities, which include the way people adjust the rhythm of their speech patterns to those of the subject with whom they communicate, and the rhythmic unison of an audience clapping. Even among groups of strangers, the rate of breathing, locomotive and subtle expressive motor movements, and rhythmic speech patterns have been observed to synchronize and entrain, in response to an auditory stimulus, such as a piece of music with a consistent rhythm. Furthermore, motor synchronization to repetitive tactile stimuli occurs in animals, including cats and monkeys as well as humans, with accompanying shifts in electroencephalogram (EEG) readings. Examples of endogenous entrainment, which occurs within the body, include the synchronizing of human circadian sleep-wake cycles to the 24-hour cycle of light and dark, and the synchronization of a heartbeat to a cardiac pacemaker.
Brainwaves, or neural oscillations, share the fundamental constituents with acoustic and optical waves, including frequency, amplitude and periodicity. The synchronous electrical activity of cortical neural ensembles can synchronize in response to external acoustic or optical stimuli and also entrain or synchronize their frequency and phase to that of a specific stimulus. Brainwave entrainment is a colloquialism for such ‘neural entrainment’, which is a term used to denote the way in which the aggregate frequency of oscillations produced by the synchronous electrical activity in ensembles of cortical neurons can adjust to synchronize with the periodic vibration of an external stimuli, such as a sustained acoustic frequency perceived as pitch, a regularly repeating pattern of intermittent sounds, perceived as rhythm, or of a regularly rhythmically intermittent flashing light.
Changes in neural oscillations, demonstrable through electroencephalogram (EEG) measurements, are precipitated by listening to music, which can modulate autonomic arousal ergotropically and trophotropically, increasing and decreasing arousal respectively. Musical auditory stimulation has also been demonstrated to improve immune function, facilitate relaxation, improve mood, and contribute to the alleviation of stress.
The Frequency following response (FFR), also referred to as Frequency Following Potential (FFP), is a specific response to hearing sound and music, by which neural oscillations adjust their frequency to match the rhythm of auditory stimuli. The use of sound with intent to influence cortical brainwave frequency is called auditory driving, by which frequency of neural oscillation is ‘driven’ to entrain with that of the rhythm of a sound source.
Neurofeedback Neurofeedback (NFB), also called neurotherapy or neurobiofeedback, is a type of biofeedback that uses real-time displays of brain activity—most commonly electroencephalography (EEG), to teach self-regulation of brain function. Typically, sensors are placed on the scalp to measure activity, with measurements displayed using video displays or sound. The feedback may be in various other forms as well. Typically, the feedback is sought to be presented through primary sensory inputs, but this is not a limitation on the technique.
The applications of neurofeedback to enhance performance extend to the arts in fields such as music, dance, and acting. A study with conservatoire musicians found that alpha-theta training benefitted the three music domains of musicality, communication, and technique. Historically, alpha-theta training, a form of neurofeedback, was created to assist creativity by inducing hypnagogia, a “borderline waking state associated with creative insights”, through facilitation of neural connectivity. Alpha-theta training has also been shown to improve novice singing in children. Alpha-theta neurofeedback, in conjunction with heart rate variability training, a form of biofeedback, has also produced benefits in dance by enhancing performance in competitive ballroom dancing and increasing cognitive creativity in contemporary dancers. Additionally, neurofeedback has also been shown to instill a superior flow state in actors, possibly due to greater immersion while performing.
Transcranial Stimulation Non-invasive brain stimulation (NIBS) bypasses the correlative approaches of other imaging techniques, making it possible to establish a causal relationship between cognitive processes and the functioning of specific brain areas. NIBS can provide information about where a particular process occurs. NIBS offers the opportunity to study brain mechanisms beyond process localization, providing information about when activity in a given brain region is involved in a cognitive process, and even how it is involved. When using NIBS to explore cognitive processes, it is important to understand not only how NIBS functions but also the functioning of the neural structures themselves. Non-invasive brain stimulation (NIBS) methods, which include transcranial magnetic stimulation (TMS) and transcranial electric stimulation (tES), are used in cognitive neuroscience to induce transient changes in brain activity and thereby alter the behavior of the subject. The application of NIBS aims at establishing the role of a given cortical area in an ongoing specific motor, perceptual or cognitive process. Physically, NIBS techniques affect neuronal states through different mechanisms. In TMS, a solenoid (coil) is used to deliver a strong and transient magnetic field, or “pulse,” to induce a transitory electric current at the cortical surface beneath the coil. The pulse causes the rapid and above-threshold depolarization of cell membranes affected by the current, followed by the transynaptic depolarization or hyperpolarization of interconnected neurons. Therefore, TMS induces a current that elicits action potentials in neurons. A complex set of coils can deliver a complex 3D excitation field. By contrast, in tES techniques, the stimulation involves the application of weak electrical currents directly to the scalp through a pair of electrodes. As a result, tES induces a subthreshold polarization of cortical neurons that is too weak to generate an action potential. However, by changing the intrinsic neuronal excitability, tES can induce changes in the resting membrane potential and the postsynaptic activity of cortical neurons. This, in turn, can alter the spontaneous firing rate of neurons and modulate their response to afferent signals, leading to changes in synaptic efficacy. The typical application of NIBS involves different types of protocols: TMS can be delivered as a single pulse (spTMS) at a precise time, as pairs of pulses separated by a variable interval, or as a series of stimuli in conventional or patterned protocols of repetitive TMS (rTMS). In tES, different protocols are established by the electrical current used and by its polarity, which can be direct (anodal or cathodal transcranial direct current stimulation: tDCS), alternating at a fixed frequency (transcranial alternating current stimulation: tACS) or at random frequencies (transcranial random noise stimulation: tRNS). In general, the final effects of NIBS on the central nervous system depend on a lengthy list of parameters (e.g., frequency, temporal characteristics, intensity, geometric configuration of the coil/electrode, current direction), when it is delivered before (off-line) or during (on-line) the task as part of the experimental procedure. In addition, these factors interact with several variables related to the anatomy (e.g., properties of the brain tissue and its location), as well as physiological (e.g., gender and age) and cognitive states of the stimulated area/subject. The entrainment hypothesis, suggests the possibility of inducing a particular oscillation frequency in the brain using an external oscillatory force (e.g., rTMS, but also tACS). The physiological basis of oscillatory cortical activity lies in the timing of the interacting neurons; when groups of neurons synchronize their firing activities, brain rhythms emerge, network oscillations are generated, and the basis for interactions between brain areas may develop. Because of the variety of experimental protocols for brain stimulation, limits on descriptions of the actual protocols employed, and limited controls, consistency of reported studies is lacking, and extrapolability is limited. Thus, while there is various consensus in various aspects of the effects of extra cranial brain stimulation, the results achieved have a degree of uncertainty dependent on details of implementation. On the other hand, within a specific experimental protocol, it is possible to obtain statistically significant and repeatable results. This implies that feedback control might be effective to control implementation of the stimulation for a given purpose; however, studies that employ feedback control are lacking.
Changes in the neuronal threshold result from changes in membrane permeability, which influence the response of the task-related network. The same mechanism of action may be responsible for both tES methods and TMS, i.e., the induction of noise in the system. However, the neural activity induced by tES will be highly influenced by the state of the system because it is a neuromodulatory method, and its effect will depend on the activity of the stimulated area. Therefore, the final result will depend strongly on the task characteristics, the system state and the way in which tES will interact with such a state. In TMS, the magnetic pulse causes the rapid and above-threshold depolarization of cell membranes affected by the current, leading to the trans-synaptic depolarization or hyperpolarization of connected cortical neurons. Therefore, TMS activates a neural population that, depending on several factors, can be congruent (facilitate) or incongruent (inhibit) with task execution. tES induces a polarization of cortical neurons at a subthreshold level that is too weak to evoke an action potential. However, by inducing a polarity shift in the intrinsic neuronal excitability, tES can alter the spontaneous firing rate of neurons and modulate the response to afferent signals. In this sense, tES-induced effects are even more bound to the state of the stimulated area that is determined by the emotion conditions. In short, NIBS leads to a stimulation-induced modulation of mood that can be substantially defined as noise induction. Induced noise will not be just random activity, but will depend on the interaction of many parameters, from the characteristics of the stimulation to the emotion. The noise induced by NIBS will be influenced by the state of the neural population of the stimulated area. Although the types and number of neurons “triggered” by NIBS are theoretically random, the induced change in neuronal activity is likely to be correlated with ongoing emotion-relevant activity, yet even if we are referring to a non-deterministic process, the noise introduced will not be a totally random element. Because it will be partially determined by the experimental variables, the level of noise that will be introduced by the stimulation and by the emotion can be estimated, as well as the interaction between the two levels of noise (stimulation and emotion). Known transcranial stimulation does not permit stimulation with a focused and highly targeted signal to a clearly defined area of the brain to establish a unique brain-behavior relationship; therefore, the known introduced stimulus activity in the brain stimulation is ‘noise.’
Transcranial Direct Current Stimulation (tDCS) tDCS. Cranial electrotherapy stimulation (CES) is a form of non-invasive brain stimulation that applies a small, pulsed electric current across a person's head to treat a variety of conditions such as anxiety, depression and insomnia. See, en.wikipedia.org/wiki/Cranial electrotherapy stimulation. Transcranial direct current stimulation (tDCS) is a form of neurostimulation that uses constant, low current delivered to the brain area of interest via electrodes on the scalp. It was originally developed to help patients with brain injuries or psychiatric conditions like major depressive disorder. tDCS appears to have some potential for treating depression. See, en.wikipedia.org/wiki/Transcranial direct-current stimulation.
tES (tDCS, tACS, and tRNS) is a noninvasive method of cortical stimulation, using weak direct currents to polarize target brain regions. The most used and best-known method is tDCS, as all considerations for the use of tDCS have been extended to the other tES methods. The hypotheses concerning the application of tDCS in cognition are very similar to those of TMS, with the exception that tDCS was never considered a virtual lesion method. tDCS can increase or decrease cortical excitability in the stimulated brain regions and facilitate or inhibit behavior accordingly. tES does not induce action potentials but instead modulates the neuronal response threshold so that it can be defined as subthreshold stimulation.
High-Definition-tDCS High-Definition transcranial Direct Current Stimulation (HD-tDCS) was invented at The City University of New York with the introduction of the 4×1 HD-tDCS montage. The 4×1 HD-tDCS montage allows precise targeting of cortical structures. The region of current flow is circumscribed by the area of the 4× ring, such that decreasing ring radius increases focality. 4×1 HD-tDCS allows for unifocal stimulation, meaning the polarity of the center 1× electrode will determine the direction of neuromodulation under the ring. This is in contrast to conventional tDCS where the need for one anode and one cathode always produces bidirectional modulation (even when an extra-cephalic electrode is used). 4×1 HD-tDCS thus provides the ability not only to select a cortical brain region to target, but to modulate the excitability of that brain region with a designed polarity without having to consider return counter-electrode flow.
Transcranial Alternative Current Stimulation (tACS) Transcranial alternating current stimulation (tACS) is a noninvasive means by which alternating electrical current applied through the skin and skull entrains in a frequency-specific fashion the neural oscillations of the underlying brain. See, en.wikipedia.org/wiki/Transcranial_alternating_current_stimulation.
Transcranial Magnetic Stimulation (TMS) Transcranial magnetic stimulation (TMS) is a method in which a changing magnetic field is used to cause electric current to flow in a small region of the brain via electromagnetic induction. During a TMS procedure, a magnetic field generator, or “coil”, is placed near the head of the person receiving the treatment. The coil is connected to a pulse generator, or stimulator, that delivers a changing electric current to the coil. TMS is used diagnostically to measure the connection between the central nervous system and skeletal muscle to evaluate damage in a wide variety of disease states, including stroke, multiple sclerosis, amyotrophic lateral sclerosis, movement disorders, and motor neuron diseases. Evidence is available suggesting that TMS is useful in treating neuropathic pain, major depressive disorder, and other conditions.
Low Energy Neurofeedback System (LENS) The LENS, or Low Energy Neurofeedback System, uses a very low power electromagnetic field, to carry feedback to the person receiving it. The feedback travels down the same wires carrying the brain waves to the amplifier and computer. Although the feedback signal is weak, it produces a measurable change in the brainwaves without conscious effort from the individual receiving the feedback. The system is software controlled, to receive input from EEG electrodes, to control the stimulation through the scalp. Neurofeedback uses a feedback frequency that is different from, but correlates with, the dominant brainwave frequency. When exposed to this feedback frequency, the EEG amplitude distribution changes in power. Most of the time the brain waves reduce in power; but at times they also increase in power. In either case the result is a changed brainwave state, and much greater ability for the brain to regulate itself.
Binaural beats Binaural beats are auditory brainstem responses which originate in the superior olivary nucleus of each hemisphere. They result from the interaction of two different auditory impulses, originating in opposite ears, below 1000 Hz and which differ in frequency between one and 30 Hz. For example, if a pure tone of 400 Hz is presented to the right ear and a pure tone of 410 Hz is presented simultaneously to the left ear, an amplitude modulated standing wave of 10 Hz, the difference between the two tones, is experienced as the two wave-forms mesh in and out of phase within the superior olivary nuclei. This binaural beat is not heard in the ordinary sense of the word (the human range of hearing is from 20-20,000 Hz). It is perceived as an auditory beat and theoretically can be used to entrain specific neural rhythms through the frequency-following response (FFR)—the tendency for cortical potentials to entrain to or resonate at the frequency of an external stimulus. Thus, it is theoretically possible to utilize a specific binaural-beat frequency as a consciousness management technique to entrain a specific cortical rhythm. The binaural-beat appears to be associated with an electroencephalographic (EEG) frequency-following response in the brain.
Uses of audio with embedded binaural beats that are mixed with music or various pink or background sound are diverse. They range from relaxation, meditation, stress reduction, pain management, improved sleep quality, decrease in sleep requirements, super learning, enhanced creativity and intuition, remote viewing, telepathy, and out-of-body experience and lucid dreaming. Audio embedded with binaural beats is often combined with various meditation techniques, as well as positive affirmations and visualization.
When signals of two different frequencies are presented, one to each ear, the brain detects phase differences between these signals. “Under natural circumstances a detected phase difference would provide directional information. The brain processes this anomalous information differently when these phase differences are heard with stereo headphones or speakers. A perceptual integration of the two signals takes place, producing the sensation of a third “beat” frequency. The difference between the signals waxes and wanes as the two different input frequencies mesh in and out of phase. As a result of these constantly increasing and decreasing differences, an amplitude-modulated standing wave—the binaural beat—is heard. The binaural beat is perceived as a fluctuating rhythm at the frequency of the difference between the two auditory inputs. Evidence suggests that the binaural beats are generated in the brainstem's superior olivary nucleus, the first site of contralateral integration in the auditory system. Studies also suggest that the frequency-following response originates from the inferior colliculus. This activity is conducted to the cortex where it can be recorded by scalp electrodes. Binaural beats can easily be heard at the low frequencies (<30 Hz) that are characteristic of the EEG spectrum.
Synchronized brain waves have long been associated with meditative and hypnogogic states, and audio with embedded binaural beats has the ability to induce and improve such states of consciousness. The reason for this is physiological. Each ear is “hardwired” (so to speak) to both hemispheres of the brain. Each hemisphere has its own olivary nucleus (sound-processing center) which receives signals from each ear. In keeping with this physiological structure, when a binaural beat is perceived there are actually two standing waves of equal amplitude and frequency present, one in each hemisphere. So, there are two separate standing waves entraining portions of each hemisphere to the same frequency. The binaural beats appear to contribute to the hemispheric synchronization evidenced in meditative and hypnogogic states of consciousness. Brain function is also enhanced through the increase of cross-collosal communication between the left and right hemispheres of the brain.
Isochronic Tones Isochronic tones are regular beats of a single tone that are used alongside monaural beats and binaural beats in the process called brainwave entrainment. At its simplest level, an isochronic tone is a tone that is being turned on and off rapidly. They create sharp, distinctive pulses of sound.
Light Stimulation The functional relevance of brain oscillations in the alpha frequency range (8-13 Hz) has been repeatedly investigated through the use of rhythmic visual stimulation. There are two hypotheses on the origin of steady-state visual evoked potential (SSVEP) measured in EEG during rhythmic stimulation: entrainment of brain oscillations and superposition of event-related responses (ERPs). The entrainment but not the superposition hypothesis justifies rhythmic visual stimulation as a means to manipulate brain oscillations, because superposition assumes a linear summation of single responses, independent from ongoing brain oscillations. Participants stimulated with rhythmic flickering light of different frequencies and intensities, and entrainment was measured by comparing the phase coupling of brain oscillations stimulated by rhythmic visual flicker with the oscillations induced by arrhythmic littered stimulation, varying the time, stimulation frequency, and intensity conditions. Phase coupling was found to be more pronounced with increasing stimulation intensity as well as at stimulation frequencies closer to each participant's intrinsic frequency. Even in a single sequence of an SSVEP, non-linear features (intermittency of phase locking) were found that contradict the linear summation of single responses, as assumed by the superposition hypothesis. Thus, evidence suggests that visual rhythmic stimulation entrains brain oscillations, validating the approach of rhythmic stimulation as a manipulation of brain oscillations. See, Notbohm A, Kurths J, Herrmann C S, Modification of Brain Oscillations via Rhythmic Light Stimulation Provides Evidence for Entrainment but Not for Superposition of Event-Related Responses, Front Hum Neurosci. 2016 Feb. 3; 10:10. doi: 10.3389/fnhum.2016.00010.eCollection 2016.
It is also known that periodic visual stimulation can trigger epileptic seizures.
Principal Component Analysis Principal component analysis (PCA) is a statistical procedure that uses an orthogonal transformation to convert a set of observations of possibly correlated variables into a set of values of linearly uncorrelated variables called principal components. If there are n observations with p variables, then the number of distinct principal components is min(n−1,p). This transformation is defined in such a way that the first principal component has the largest possible variance (that is, accounts for as much of the variability in the data as possible), and each succeeding component in turn has the highest variance possible under the constraint that it is orthogonal to the preceding components. The resulting vectors are an uncorrelated orthogonal basis set. PCA is sensitive to the relative scaling of the original variables. PCA is the simplest of the true eigenvector-based multivariate analyses. Often, its operation can be thought of as revealing the internal structure of the data in a way that best explains the variance in the data. If a multivariate dataset is visualized as a set of coordinates in a high-dimensional data space (1 axis per variable), PCA can supply the user with a lower-dimensional picture, a projection of this object when viewed from its most informative viewpoint. This is done by using only the first few principal components so that the dimensionality of the transformed data is reduced. PCA is closely related to factor analysis. Factor analysis typically incorporates more domain specific assumptions about the underlying structure and solves eigenvectors of a slightly different matrix. PCA is also related to canonical correlation analysis (CCA). CCA defines coordinate systems that optimally describe the cross-covariance between two datasets while PCA defines a new orthogonal coordinate system that optimally describes variance in a single dataset. See, en.wikipedia.org/wiki/Principal_component_analysis
A general model for confirmatory factor analysis is expressed as x=α+Λξ+ε. The covariance matrix is expressed as E[(x−μ)(x−μ)′]=ΛΦΛ′+Θ. If residual covariance matrix Θ=0 and correlation matrix among latent factors Φ=I, then factor analysis is equivalent to principal component analysis and the resulting covariance matrix is simplified to Σ=ΛΛ′. When there are p number of variables and all p components (or factors) are extracted, this covariance matrix can alternatively be expressed into Θ=DΛD′, or Σ=λDAD′, where D=n×p orthogonal matrix of eigenvectors, and Λ=λA, p×p matrix of eigenvalues, where λ is a scalar and A is a diagonal matrix whose elements are proportional to the eigenvalues of Σ. The following three components determine the geometric features of the observed data: λ parameterizes the volume of the observation, D indicates the orientation, and A represents the shape of the observation.
When population heterogeneity is explicitly hypothesized as in model-based cluster analysis, the observed covariance matrix is decomposed into the following general form
Σk=λkDkAkDkT,
where λk parameterizes the volume of the kth cluster, Dk indicates the orientation of that cluster, and Ak represents the shape of that cluster. The subscript k indicates that each component (or cluster) can have different volume, shape, and orientation.
Assume a random vector X, taking values in m, has a mean and covariance matrix of μX and ΣX, respectively. λ1>λ2> . . . >λm>0 are ordered eigenvalues of ΣX, such that the i-th eigenvalue of ΣX means the i-th largest of them. Similarly, a vector αi is the i-th eigenvector of ΣX when it corresponds to the i-th eigenvalue of ΣX. To derive the form of principal components (PCs), consider the optimization problem of maximizing var[α1T X]=α1T ΣX α1, subject to α1Tα1=1. The Lagrange multiplier method is used to solve this question.
Because −ϕ1 is the eigenvalue of ΣX, with α1 being the corresponding normalized eigenvector, var[α1T X] is maximized by choosing α1 to be the first eigenvector of ΣX. In this case, 1=α1T X is named the first PC of X, α1 is the vector of coefficients for z1, and var(z1)=λ1.
To find the second PC, z2=α2T X, we need to maximize var[α2T X]=α2T ΣX α2 subject to z2 being uncorrelated with z1. Because cov(α1T X, α2T X)=0⇒α1T ΣX α2=0⇒α1T α2=0, this problem is equivalently set as maximizing α2T ΣX α2, subject to α1T α2=0, and α2T α2=1. We still make use of the Lagrange multiplier method.
Because −ϕ2 is the eigenvalue of ΣX, with α2 being the corresponding normalized eigenvector, var[α2T X] is maximized by choosing α2 to be the second eigenvector of ΣX. In this case, 2=α2T X is named the second PC of X, α2 is the vector of coefficients for
2, and var(
2)=λ2. Continuing in this way, it can be shown that the i-th PC
i=αiT X is constructed by selecting αi to be the i-th eigenvector of ΣX, and has variance of λi. The key result in regards to PCA is that the principal components are the only set of linear functions of original data that are uncorrelated and have orthogonal vectors of coefficients.
For any positive integer p≤m, let B=[β1, β2, . . . , βp] be an real m×P matrix with orthonormal columns, i.e., βiT=βj=δij, and Y=BT X. Then the trace of covariance matrix of Y is maximized by taking B=[α1, α2, . . . , αp], where αi is the i-th eigenvector of ΣX. Because ΣX is symmetric with all distinct eigenvalues, so {α1, α2, . . . , αm} is an orthonormal basis with αi being the i-th eigenvector of ΣX, and we can represent the columns of B as
i=1, . . . , p, So we have B=PC, where P=[α1, . . . αm], C={cij} is an m×p matrix. Then, PTΣX P=Λ, with Λ being a diagonal matrix whose k-th diagonal element is λk, and the covariance matrix of Y is,
ΣY=BT ΣX B=CT PT ΣX PC=CT ΛC=λ1c1c1+ . . . +λmcmcmT, where is the i-th row of C. So,
Because CTC=BTPPTB=BTB=I, so
and the columns of C are orthonormal. By the Gram-Schmidt method, C can expand to D, such that D has its columns as an orthonormal basis of m and contains C as its first p columns. D is square shape, thus being an orthogonal matrix and having its rows as another orthonormal basis of
m. One row of C is a part of one row of D, so
i=1, . . . , m Considering the constraints
and the objective
We derive that trace(ΣY) is maximized if
for i=1, . . . p, and
for i=p+1, . . . , m. When B=[α1, α2, . . . αp], straightforward calculation yields that C is an all-zero matrix except cii=1, i=1, . . . , p. This fulfills the maximization condition. Actually, by taking B=[γ1, γ2, . . . , γp], where {γ1, γ2, . . . , γp} is any orthonormal basis of the subspace of span{αi, α2, . . . , αp}, the maximization condition is also satisfied, thus yielding the same trace of covariance matrix of Y.
Suppose that we wish to approximate the random vector X by its projection onto a subspace spanned by columns of B, where B=[β1, β2, . . . βp] is a real m×p matrix with orthonormal columns, i.e., βiT βj=δij. If σi2 is the residual variance for each component of X, then
is minimized if B=[α1, α2, . . . , αP], where {α1, α2, . . . , αp} are the first p eigenvectors of ΣX. In other words, the trace of covariance matrix of X−BBT X is minimized if B=[α1, α2, . . . , αp]. When E(X)=0, which is a commonly applied preprocessing step in data analysis methods, this property is saying that E∥X−BBT X∥2 is minimized if B=[α1, α2, . . . , αp].
The projection of a random vector X onto a subspace spanned by columns of B is {circumflex over (X)}=BBT X. Then the residual vector is ε=X−BBT X, which has a covariance matrix Σε=(I−BBT) ΣX(I−BBT). Then,
Also, we know
trace(ΣXBBT)=trace(BBTΣX)=trace(BTΣXB)
trace(BBTΣXBBT)=trace(BTΣXBBTB)=trace(BTΣXB).
The last equation comes from the fact that B has orthonormal columns. So,
To minimize
it suffices to maximize trace(BT ΣX B). This can be done by choosing B=[α1, α2, . . . , αp], where {α1, α2, . . . , αp} are the first P eigenvectors of ΣX, as above.
See, Pietro Amenta, Luigi D'Ambra, “Generalized Constrained Principal Component Analysis with External Information,” (2000). We assume that data on K sets of explanatory variables and S criterion variables of n statistical units are collected in matrices Xk (k=1, . . . , K) and Ys (s=1, . . . , S) of orders (n×p1), . . . , (n×pK) and (n×q1), . . . , (n×qS) respectively. We suppose, without loss of generality, identity matrices for the metrics of the spaces of variables of Xk and Ys with Dn=diag(1/n), weight matrix of statistical units. We assume, moreover, that Xk's and Ys's are centered as to the weights Dn.
Let X=[X1| . . . |XK] and Y=[Y1| . . . |YS], respectively, be K and S matrices column linked of orders (n×Σk pk) and (n×Σs qs). Let be, also, WY=YY′ while we denote vk the coefficients vector (pk, 1) of the linear combination for each Xk such that k=Xkvk. Let Ck be the matrix of dimension pk×m (m≤pk), associated to the external information explanatory variables of set k.
Generalized CPCA (GCPCA) (Amenta, D'Ambra, 1999) with external information consists in seeking for K coefficients vectors vk (or, in same way, K linear combinations k) subject to the restriction Ck′vk=0 simultaneously, such that:
where A=Y′X, B=diag(X1′X1, . . . , XK′XK) C′=[C1′| . . . |Ck′], v′=(v1′| . . . |vk′) and f=B0.5v, with
The constrained maximum problem (1) turns out to be an extension of criterion
(Sabatier, 1993) with more sets of criterion variables with external information. The solution of this constrained maximum problem leads to solve the eigen-equation
(PX−PXB
where g=Xv, PX−PXBn
n=Im(PX−PXB
with PX
(PX
(which is obtained from the expression (I−PC)X′WYg=λBv) the coefficients vectors vk and the linear combinations k=Xkvk maximizing (1) can be given by the relations
respectively.
The solution eigenvector g can be written, according to (2) and (3), as sum of the linear combinations k: g=ΣkXkvk. Notice that the eigenvalues associated to the eigen-system (2) are, according to the Sturm theorem, lower or equal than those of GCPCA eigen-system: Σk=1KPx
Spatial Principal Component Analysis
We introduce the following notation. Let J(t,i;α,s) be the current density in voxel i, as estimated by LORETA, in condition α at t time-frames after stimulus onset for subject s. Let area:Voxel→fBA be a function, which assigns to each voxel i∈Voxel the corresponding fBA b∈bBA. In a first pre-processing step, we calculate for each subject s the value of the current density averaged over each Fba
where Nb is the number of voxels in the fBA b, in condition α for subject s.
In the second analysis stage, the mean current density x(t,b;α,s) from each fBA b, for every subject sand condition α, was subjected to spatial PCA analysis of the correlation matrix and varimax rotation
In the present study the spatial PCA uses the above-defined fBAs as variables sampled along the time epoch for which EEG has been sampled (0-1000 ms; 512 time-frames), and the inverse solution was estimated. Spatial matrices (each matrix was sized b×t=36×512 elements) for every subject and condition were collected, and subjected to PCA analyses, including the calculation of the covariance matrix; eigenvalue decomposition and varimax rotation, in order to maximize factor loadings. In other words, in the spatial PCA analysiswe approximate the mean current density for each subject in each condition as
where here x(t;α,s)∈R36 is a vector, which denotes the time-dependent activation of the fBAs, x0(α,s) is their mean activation, and xk(α,s) and ck are the principal components and their corresponding coefficients (factor loadings) as computed using the principal component analysis.
EEG analysis approaches have emerged, in which event-related changes in EEG dynamics in single event-related data records are analyzed. See Allen D. Malony et al., Computational Neuroinformatics for Integrated Electromagnetic Neuroimaging and Analysis, PAR-99-138. Pfurtscheller, reported a method for quantifying the average transient suppression of alpha band (circa 10-Hz) activity following stimulation. Event-related desynchronization (ERD, spectral amplitude decreases), and event-related synchronization (ERS, spectral amplitude increases) are observed in a variety of narrow frequency bands (4-40 Hz) which are systematically dependent on task and cognitive state variables as well as on stimulus parameters. Makeig (1993) was reported event-related changes in the full EEG spectrum, yielding a 2-D time/frequency measure he called the event-related spectral perturbation (ERSP). This method avoided problems associated with analysis of a priori narrow frequency bands, since bands of interest for the analysis could be based on significant features of the complete time/frequency transform. Rappelsburger et al. introduced event-related coherence (ERCOH).
A wide variety of other signal processing measures have been tested for use on EEG and/or MEG data, including dimensionality measures based on chaos theory and the bispectrum. Use of neural networks has also been proposed for EEG pattern recognition applied to clinical and practical problems, though usually these methods have not been employed with an aim of explicitly modeling the neurodynamics involved.
The availability of and interest in larger and larger numbers of EEG (and MEG) channels led immediately to the question of how to combine data from different channels. Donchin advocated the use of linear factor analysis methods based on principal component analysis (PCA) for this purpose. Temporal PCA assumes that the time course of activation of each derived component is the same in all data conditions. Because this is unreasonable for many data sets, spatial PCA (usually followed by a component rotation procedure such as Varimax or Promax) is of potentially greater interest. To this end, several variants of PCA have been proposed for ERP decomposition.
Bell and Sejnowski published an iterative algorithm based on information theory for decomposing linearly mixed signals into temporally independent by minimizing their mutual information. First approaches to blind source separation minimized third and fourth-order correlations among the observed variables and achieved limited success in simulations. A generalized approach uses a simple neural network algorithm that used joint information maximization or ‘infomax’ as a training criterion. By using a compressive nonlinearity to transform the data and then following the entropy gradient of the resulting mixtures, ten recorded voice and music sound sources were unmixed. A similar approach was used for performing blind deconvolution, and the ‘infomax’ method was used for decomposition of visual scenes.
The first applications of blind decomposition to biomedical time series analysis applied the infomax independent component analysis (ICA) algorithm to decomposition of EEG and event-related potential (ERP) data and reported the use of ICA to monitor alertness. This separated artifacts, and EEG data into constituent components defined by spatial stability and temporal independence. ICA can also be used to remove artifacts from continuous or event-related (single-trial) EEG data prior to averaging. Vigario et al. (1997), using a different ICA algorithm, supported the use of ICA for identifying artifacts in MEG data. Meanwhile, widespread interest in ICA has led to multiple applications to biomedical data as well as to other fields (Jung et al., 2000b). Most relevant to EEG/MEG analysis, ICA is effective in separating functionally independent components of functional magnetic resonance imaging (fMRI) data
Since the publication of the original infomax ICA algorithm, several extensions have been proposed. Incorporation of a ‘natural gradient’ term avoided matrix inversions, greatly speeding the convergence of the algorithm and making it practical for use with personal computers on large data EEG and fMRI data sets. An initial ‘sphering’ step further increased the reliability of convergence of the algorithm. The original algorithm assumed that sources have ‘sparse’ (super-Gaussian) distributions of activation values. This restriction has recently been relaxed in an ‘extended-ICA’ algorithm that allows both super-Gaussian and sub-Gaussian sources to be identified. A number of variant ICA algorithms have appeared in the signal processing literature. In general, these make more specific assumptions about the temporal or spatial structure of the components to be separated, and typically are more computationally intensive than the infomax algorithm.
Since individual electrodes (or magnetic sensors) each record a mixture of brain and non-brain sources, spectral measures are difficult to interpret and compare across scalp channels. For example, an increase in coherence between two electrode signals may reflect the activation of a strong brain source projecting to both electrodes, or the deactivation of a brain generator projecting mainly to one of the electrodes. If independent components of the EEG (or MEG) data can be considered to measure activity within functionally distinct brain networks, however, event-related coherence between independent components may reveal transient, event-related changes in their coupling and decoupling (at one or more EEG/MEG frequencies). ERCOH analysis has been applied to independent EEG components in a selective attention task.
Nonlinear Dimensionality Reduction High-dimensional data, meaning data that requires more than two or three dimensions to represent, can be difficult to interpret. One approach to simplification is to assume that the data of interest lie on an embedded non-linear manifold within the higher-dimensional space. If the manifold is of low enough dimension, the data can be visualized in the low-dimensional space. Non-linear methods can be broadly classified into two groups: those that provide a mapping (either from the high-dimensional space to the low-dimensional embedding or vice versa), and those that just give a visualization. In the context of machine learning, mapping methods may be viewed as a preliminary feature extraction step, after which pattern recognition algorithms are applied. Typically, those that just give a visualization are based on proximity data—that is, distance measurements. Related Linear Decomposition Methods include Independent component analysis (ICA), Principal component analysis (PCA) (also called Karhunen-Loéve transform—KLT), Singular value decomposition (SVD), and Factor analysis.
The self-organizing map (SOM, also called Kohonen map) and its probabilistic variant generative topographic mapping (GTM) use a point representation in the embedded space to form a latent variable model based on a non-linear mapping from the embedded space to the high-dimensional space. These techniques are related to work on density networks, which also are based around the same probabilistic model.
Principal curves and manifolds give the natural geometric framework for nonlinear dimensionality reduction and extend the geometric interpretation of PCA by explicitly constructing an embedded manifold, and by encoding using standard geometric projection onto the manifold. How to define the “simplicity” of the manifold is problem-dependent, however, it is commonly measured by the intrinsic dimensionality and/or the smoothness of the manifold. Usually, the principal manifold is defined as a solution to an optimization problem. The objective function includes a quality of data approximation and some penalty terms for the bending of the manifold. The popular initial approximations are generated by linear PCA, Kohonen's SOM or autoencoders. The elastic map method provides the expectation-maximization algorithm for principal manifold learning with minimization of quadratic energy functional at the “maximization” step.
An autoencoder is a feed-forward neural network which is trained to approximate the identity function. That is, it is trained to map from a vector of values to the same vector. When used for dimensionality reduction purposes, one of the hidden layers in the network is limited to contain only a small number of network units. Thus, the network must learn to encode the vector into a small number of dimensions and then decode it back into the original space. Thus, the first half of the network is a model which maps from high to low-dimensional space, and the second half maps from low to high-dimensional space. Although the idea of autoencoders is quite old, training of deep autoencoders has only recently become possible through the use of restricted Boltzmann machines and stacked denoising autoencoders. Related to autoencoders is the NeuroScale algorithm, which uses stress functions inspired by multidimensional scaling and Sammon mappings (see below) to learn a non-linear mapping from the high-dimensional to the embedded space. The mappings in NeuroScale are based on radial basis function networks.
Gaussian process latent variable models (GPLVM) are probabilistic dimensionality reduction methods that use Gaussian Processes (GPs) to find a lower dimensional non-linear embedding of high dimensional data. They are an extension of the Probabilistic formulation of PCA. The model is defined probabilistically and the latent variables are then marginalized and parameters are obtained by maximizing the likelihood. Like kernel PCA they use a kernel function to form a nonlinear mapping (in the form of a Gaussian process). However, in the GPLVM the mapping is from the embedded (latent) space to the data space (like density networks and GTM) whereas in kernel PCA it is in the opposite direction. It was originally proposed for visualization of high dimensional data but has been extended to construct a shared manifold model between two observation spaces. GPLVM and its many variants have been proposed specially for human motion modeling, e.g., back constrained GPLVM, GP dynamic model (GPDM), balanced GPDM (B-GPDM) and topologically constrained GPDM. To capture the coupling effect of the pose and gait manifolds in the gait analysis, a multi-layer joint gait-pose manifolds was proposed.
Curvilinear component analysis (CCA) looks for the configuration of points in the output space that preserves original distances as much as possible while focusing on small distances in the output space (conversely to Sammon's mapping which focus on small distances in original space). It should be noticed that CCA, as an iterative learning algorithm, actually starts with focus on large distances (like the Sammon algorithm), then gradually change focus to small distances. The small distance information will overwrite the large distance information, if compromises between the two have to be made. The stress function of CCA is related to a sum of right Bregman divergences. Curvilinear distance analysis (CDA) trains a self-organizing neural network to fit the manifold and seeks to preserve geodesic distances in its embedding. It is based on Curvilinear Component Analysis (which extended Sammon's mapping), but uses geodesic distances instead. Diffeomorphic Dimensionality Reduction or Diffeomap learns a smooth diffeomorphic mapping which transports the data onto a lower-dimensional linear subspace. The method solves for a smooth time indexed vector field such that flows along the field which start at the data points will end at a lower-dimensional linear subspace, thereby attempting to preserve pairwise differences under both the forward and inverse mapping.
Perhaps the most widely used algorithm for manifold learning is Kernel principal component analysis (kernel PCA). It is a combination of Principal component analysis and the kernel trick. PCA begins by computing the covariance matrix of the M×n Matrix X. It then projects the data onto the first k eigenvectors of that matrix. By comparison, KPCA begins by computing the covariance matrix of the data after being transformed into a higher-dimensional space. It then projects the transformed data onto the first k eigenvectors of that matrix, just like PCA. It uses the kernel trick to factor away much of the computation, such that the entire process can be performed without actually computing ϕ(x). Of course ϕ must be chosen such that it has a known corresponding kernel.
Laplacian Eigenmaps, LLE) are special cases of kernel PCA, performed by constructing a data-dependent kernel matrix. KPCA has an internal model, so it can be used to map points onto its embedding that were not available at training time. Laplacian Eigenmaps uses spectral techniques to perform dimensionality reduction. This technique relies on the basic assumption that the data lies in a low-dimensional manifold in a high-dimensional space. This algorithm cannot embed out of sample points, but techniques based on Reproducing kernel Hilbert space regularization exist for adding this capability. Such techniques can be applied to other nonlinear dimensionality reduction algorithms as well. Traditional techniques like principal component analysis do not consider the intrinsic geometry of the data. Laplacian eigenmaps builds a graph from neighborhood information of the data set. Each data point serves as a node on the graph and connectivity between nodes is governed by the proximity of neighboring points (using e.g., the k-nearest neighbor algorithm). The graph thus generated can be considered as a discrete approximation of the low-dimensional manifold in the high-dimensional space. Minimization of a cost function based on the graph ensures that points close to each other on the manifold are mapped close to each other in the low-dimensional space, preserving local distances. The eigenfunctions of the Laplace-Beltrami operator on the manifold serve as the embedding dimensions, since under mild conditions this operator has a countable spectrum that is a basis for square integrable functions on the manifold (compare to Fourier series on the unit circle manifold). Attempts to place Laplacian eigenmaps on solid theoretical ground have met with some success, as under certain nonrestrictive assumptions, the graph Laplacian matrix has been shown to converge to the Laplace-Beltrami operator as the number of points goes to infinity. In classification applications, low dimension manifolds can be used to model data classes which can be defined from sets of observed instances. Each observed instance can be described by two independent factors termed ‘content’ and ‘style’, where ‘content’ is the invariant factor related to the essence of the class and ‘style’ expresses variations in that class between instances. Unfortunately, Laplacian Eigenmaps may fail to produce a coherent representation of a class of interest when training data consist of instances varying significantly in terms of style. In the case of classes which are represented by multivariate sequences, Structural Laplacian Eigenmaps has been proposed to overcome this issue by adding additional constraints within the Laplacian Eigenmaps neighborhood information graph to better reflect the intrinsic structure of the class. More specifically, the graph is used to encode both the sequential structure of the multivariate sequences and, to minimize stylistic variations, proximity between data points of different sequences or even within a sequence, if it contains repetitions. Using dynamic time warping, proximity is detected by finding correspondences between and within sections of the multivariate sequences that exhibit high similarity.
Like LLE, Hessian LLE is also based on sparse matrix techniques. It tends to yield results of a much higher quality than LLE. Unfortunately, it has a very costly computational complexity, so it is not well-suited for heavily sampled manifolds. It has no internal model. Modified LLE (MLLE) is another LLE variant which uses multiple weights in each neighborhood to address the local weight matrix conditioning problem which leads to distortions in LLE maps. MLLE produces robust projections similar to Hessian LLE, but without the significant additional computational cost.
Manifold alignment takes advantage of the assumption that disparate data sets produced by similar generating processes will share a similar underlying manifold representation. By learning projections from each original space to the shared manifold, correspondences are recovered and knowledge from one domain can be transferred to another. Most manifold alignment techniques consider only two data sets, but the concept extends to arbitrarily many initial data sets. Diffusion maps leverages the relationship between heat diffusion and a random walk (Markov Chain); an analogy is drawn between the diffusion operator on a manifold and a Markov transition matrix operating on functions defined on the graph whose nodes were sampled from the manifold. Relational perspective map is a multidimensional scaling algorithm. The algorithm finds a configuration of data points on a manifold by simulating a multi-particle dynamic system on a closed manifold, where data points are mapped to particles and distances (or dissimilarity) between data points represent a repulsive force. As the manifold gradually grows in size the multi-particle system cools down gradually and converges to a configuration that reflects the distance information of the data points. Local tangent space alignment (LISA) is based on the intuition that when a manifold is correctly unfolded, all of the tangent hyperplanes to the manifold will become aligned. It begins by computing the k-nearest neighbors of every point. It computes the tangent space at every point by computing the d-first principal components in each local neighborhood. It then optimizes to find an embedding that aligns the tangent spaces. Local Multidimensional Scaling performs multidimensional scaling in local regions, and then uses convex optimization to fit all the pieces together.
Maximum Variance Unfolding was formerly known as Semidefinite Embedding. The intuition for this algorithm is that when a manifold is properly unfolded, the variance over the points is maximized. This algorithm also begins by finding the k-nearest neighbors of every point. It then seeks to solve the problem of maximizing the distance between all non-neighboring points, constrained such that the distances between neighboring points are preserved. Nonlinear PCA (NLPCA) uses backpropagation to train a multi-layer perceptron (MLP) to fit to a manifold. Unlike typical MLP training, which only updates the weights, NLPCA updates both the weights and the inputs. That is, both the weights and inputs are treated as latent values. After training, the latent inputs are a low-dimensional representation of the observed vectors, and the MLP maps from that low-dimensional representation to the high-dimensional observation space. Manifold Sculpting uses graduated optimization to find an embedding. Like other algorithms, it computes the k-nearest neighbors and tries to seek an embedding that preserves relationships in local neighborhoods. It slowly scales variance out of higher dimensions, while simultaneously adjusting points in lower dimensions to preserve those relationships.
Ruffini (2015) discusses Multichannel transcranial current stimulation (tCS) systems that offer the possibility of EEG-guided optimized, non-invasive brain stimulation. A tCS electric field realistic brain model is used to create a forward “lead-field” matrix and, from that, an EEG inverter is employed for cortical mapping. Starting from EEG, 2D cortical surface dipole fields are defined that could produce the observed EEG electrode voltages.
Schestatsky et al. (2017) discuss transcranial direct current stimulation (tDCS), which stimulates through the scalp with a constant electric current that induces shifts in neuronal membrane excitability, resulting in secondary changes in cortical activity. Although tDCS has most of its neuromodulatory effects on the underlying cortex, tDCS effects can also be observed in distant neural networks. Concomitant EEG monitoring of the effects of tDCS can provide valuable information on the mechanisms of tDCS. EEG findings can be an important surrogate marker for the effects of tDCS and thus can be used to optimize its parameters. This combined EEG-tDCS system can also be used for preventive treatment of neurological conditions characterized by abnormal peaks of cortical excitability, such as seizures. Such a system would be the basis of a non-invasive closed-loop device. tDCS and EEG can be used concurrently.
The present technology provides a method of replicating a mental state of a first subject in a second subject. (In some embodiments, a first and a second subject may be the same subject at different points in time.) The mental state typically is not a state of consciousness or an idea, but rather a subconscious (in a technical sense) state, representing an emotion, readiness, receptivity, or other state, often independent of particular thoughts or ideas. In essence, a mental state of the first subject (a “trainer” or “donor” who is in a desired mental state) is captured by recording neural correlates of the mental state, e.g., as expressed by brain activity patterns, such as EEG or MEG signals. The neural correlates of the first subject, either as direct or recorded representations, may then be used to control a stimulation of the second subject (a “trainee” or “recipient”), seeking to induce the same brain activity patterns in the second subject (recipient/trainee) as were present in the first subject (donor/trainer) to assist the second subject (recipient/trainee) to attain the desired mental state that had been attained by the donor/trainer. In an alternative embodiment, the signals from the first subject (donor/trainer) being in the first mental state are employed to prevent the second subject (recipient/trainee) from achieving a second mental state, wherein the second mental state is an undesirable one.
In some embodiments, the acquiring of the mental state information is preceded by or followed by identifying the mental state, by direct annotation by the first subject or an observer, or by automated analysis of the brain activity patterns, or both.
In other embodiments, the processing of the brain activity patterns does not seek to classify or characterize it, but rather to filter and transform the information to a form suitable for control of the stimulation of the second subject. In particular, according to this embodiment, the subtleties that are not yet reliably classified in traditional brain activity pattern analysis are respected. For example, it is understood that all brain activity is reflected in synaptic urrents and other neural modulation, and therefore to a large extent, conscious and subconscious information is in theory accessible through brain activity pattern analysis. Since the available processing technology generally fails to distinguish a large number of different brain activity patterns, that available processing technology, is necessarily deficient. However, just because a computational algorithm is unavailable to extract the information, does not mean that the information is absent. Therefore, this embodiment employs relatively raw brain activity pattern data, such as filtered EEGs, to control the stimulation of the second subject, without a full comprehension or understanding of exactly what information of significance is present. Typically, the stimulation is a relatively low dimensionality stimulus, such as stereooptic, binaural, tactile, or other sensory stimulation, operating bilaterally, and with control over frequency and phase and/or waveform.
Likewise, a lack present understanding of the essential characteristics of the signal components in the brain activity patterns does not prevent their acquisition, storage, communication, and processing (to some extent). The stimulation may be direct, i.e., a visual or auditory stimulus corresponding to the brain activity pattern, or a derivative or feedback control based on the second subject's brain activity pattern.
According to one embodiment, the stimulation of the second subject is associated with a feedback process, to verify that the second subject has appropriately responded to the stimulation, e.g., has a predefined similarity to the mental state as the first subject, has a mental state with a predefined difference from the first subject, or has a desire change from a baseline mental state. Advantageously, the stimulation may be adaptive to the feedback. In some cases, the feedback may be functional, i.e., not based on brain activity per se, or neural correlates of mental state, but rather physical, psychological, or behavioral effects that may be reported or observed.
The feedback typically is provided to a model-based controller for the stimulator, which alters stimulation parameters to optimize the stimulation.
For example, it is believed that brainwaves represent a form of resonance, where ensembles of neurons interact in a coordinated fashion. The frequency of the wave is related to neural responsivity to neurotransmitters, distances along neural pathways, diffusion limitations, etc. That is, the same mental state may be represented by different frequencies in two different individuals, based on differences in the size of their brains, neuromodulators present, physiological differences, etc. These differences may be measured in microseconds or less, resulting in fractional changes in frequency. However, if the stimulus is different from the natural or resonant frequency of the target process, the result may be different from that expected. Therefore, the model-based controller can determine the parameters of neural transmission and ensemble characteristics, vis-à-vis stimulation, and resynthesize the stimulus wave to match the correct waveform, with the optimization of the waveform adaptively determined. This may not be as simple as speeding up or slowing down playback of the signal, as different elements of the various waveforms representing neural correlates of mental state may have different relative differences between subjects.
Thus, a hybrid approach is provided, with use of source-derived waveforms, on one hand, which may be extracted from the brain activity readings of the first subject, processed by principal component analysis, or spatial principal component analysis, autocorrelation, or other statistical processing technique which separates components of brain activity, which can then be modified or modulated based on high-level parameters, e.g., abstractions. However, in the general case, the present technology maintains use of components or subcomponents of the source subject brain activity readings, e.g., EEG or MEG, and does not seek to characterize or abstract them to a semantic level
Of course, in some cases, one or more components of the stimulation of the target subject may be represented as abstract or semantically defined signals, and more generally the processing of the signals to define the stimulation will involve high level modulation or transformation between the source signal received from the first subject, to define the target signal for stimulation of the second subject.
Preferably, each component represents a subset of the neural correlates reflecting brain activity that have a high autocorrelation in space and time, or in a hybrid representation such as wavelet. For example, one signal may represent a modulated 10.2 Hz signal, while another signal represents a superposed modulated 15.7 Hz signal, with respectively different spatial origins. These may be separated by optimal filtering (e.g., spatial PCA), once the characteristics of the signal are known, and bearing in mind that the signal is accompanied by a modulation pattern, and that the two components themselves may have some weak coupling and interaction.
In some cases, the base frequency, modulation, coupling, noise, phase jitter, or other characteristic of the signal may be substituted. For example, if the first subject is listening to music, there will be significant components of the neural correlates that are synchronized with the particular music. On the other hand, the music per se may not be part of the desired stimulation of the target subject. Therefore, through signal analysis and decomposition, the components of the signal from the first subject, which have a high temporal correlation with the music, may be extracted or suppressed from the resulting signal. Further, the target subject may be in a different acoustic environment, and it may be appropriate to modify the residual signal dependent on the acoustic environment of the target subject, so that the stimulation is appropriate for achieving the desired effect, and does not represent phantoms, distractions, or irrelevant or inappropriate content. In order to perform signal processing, it is convenient to store the signals or a partially processed representation, though a complete real-time signal processing chain may be implemented. Such a real-time signal processing chain is generally characterized in that the average size of a buffer remains constant, i.e., the lag between output and input is relatively constant, bearing in mind that there may be periodicity to the processing.
According to another embodiment, the mental state of the first subject (trainer or donor) is identified, and the neural correlates of brain activity are captured, and the second subject (trainee or recipient) is subject to stimulation based on the captured neural correlates and the identified mental state. The mental state is typically represented as a semantic variable, within a limited classification space. The mental state identification need not be through analysis of the neural correlates signal, and may be a volitional self-identification by the first subject, or a manual classification by third parties using, for example, fMRI or psychological assessment. The identified mental state is useful, for example, because it represents a target toward (or, in some cases, against) which the second subject (trainee or recipient) can be steered.
The stimulation may be one or more stimulus applied to the second subject (trainee or recipient), which may be an electrical or magnetic transcranial stimulation, sensory stimulation (e.g., visual, auditory or tactile), mechanical stimulation, ultrasonic stimulation, etc., and controlled with respect to waveform, intensity/amplitude, duration, or controlled via feedback, self-reported effect by the second subject, manual classification by third parties, automated analysis of brain activity, behavior, physiological parameters, etc. of the second subject.
Typically, the goal of the process is to induce in the second subject (trainee or recipient) neural correlates of the mental state of the first subject (trainer or donor) corresponding to the mental state of the first subject, through the use of stimulation parameters comprising a waveform over a period of time derived from the neural correlates of the mental state of the first subject.
Typically, the first and the second subjects are spatially remote from each other and may be temporally remote as well. In some cases, the first and second subject are the same subject (human or animal), temporally displaced. In other cases, the first and the second subject are spatially proximate to each other. These different embodiments differ principally in the transfer of the signal from the first subject to the second subject. However, when the first and the second subjects share a common environment, the signal processing of the neural correlates and, especially of real-time feedback of neural correlates from the second subject, may involve interactive algorithms with the neural correlates of the first subject.
According to another embodiment, the first and second subjects are each subject to stimulation. In one particularly interesting embodiment, the first subject and the second subject communicate with each other in real-time, with the first subject receiving stimulation based on the second subject, and the second subject receiving feedback based on the first subject. This can lead to synchronization of neural correlates (e.g., neuronal oscillations, or brainwaves) and, consequently, of mental state between the two subjects. However, the first subject need not receive stimulation based on real-time signals from the second subject, as the stimulation may derive from a third subject, or the first or second subjects at different points in time.
The neural correlates may be neuronal oscillations resulting in brainwaves that are detectable as, for example, EEG, gEEG, or MEG signals. Traditionally, these signals are found to have dominant frequencies, which may be determined by various analyses, such as spectral analysis, wavelet analysis, or principal part analysis (PCA), for example. One embodiment provides that the modulation pattern of a brainwave of the first subject is determined independent of the dominant frequency of the brainwave (though, typically, within the same class of brainwaves), and this modulation imposed on a brainwave corresponding to the dominant frequency of the second subject. That is, once the second subject achieves that same brainwave pattern as the first subject (which may be achieved by means other than electromagnetic, mechanical, or sensory stimulation), the modulation pattern of the first subject is imposed as a way of guiding the mental state of the second subject.
According to another embodiment, the second subject is stimulated with a stimulation signal which faithfully represents the frequency composition of a defined component of the neural correlates of the first subject. The defined component may be determined based on a principal component analysis or related technique.
The stimulation may be performed, for example, by using a tDCS device, a high-definition tDCS device, a tACS device, a TMS device, a deep TMS device, a light source, or a sound source configured to modulate the dominant frequency on the light signal or the sound signal. The stimulus may be a light signal, a sound signal, an electric signal, a magnetic field, olfactory or a tactile stimulation. The electric signal may be a direct current signal or an alternating current signal. The stimulus may be applied via a transcranial electric stimulation, a transcranial magnetic stimulation, a deep magnetic stimulation, a light stimulation, or a sound stimulation. A visual stimulus may be ambient light or a direct light. An auditory stimulus may be binaural beats or isochronic tones.
The technology also provides a processor configured to process the neural correlates of mental state from the first subject (trainer or donor), and to produce or define a stimulation pattern for the second subject (trainee or recipient) selectively dependent on a waveform pattern of the neural correlates from the first subject. Typically, the processor performs signal analysis and calculates at least a dominant frequency of the brainwaves of the first subject, and preferably also spatial and phase patterns within the brain of the first subject. The processor may also perform a PCA, a spatial PCA, an independent component analysis (ICA), eigenvalue decomposition, eigenvector-based multivariate analyses, factor analysis, an autoencoder neural network with a linear hidden layer, linear discriminant analysis, network component analysis, nonlinear dimensionality reduction (NLDR), or another statistical method of data analysis.
A signal is presented to a second apparatus, configured to stimulate the second subject (trainee or recipient), which may be an open loop stimulation dependent on a non-feedback controlled algorithm, or a closed loop feedback dependent algorithm. In other cases, analog processing is employed in part or in whole, wherein the algorithm comprises an analog signal processing chain. The second apparatus receives information from the processor (first apparatus), typically comprising a representation of a portion of a waveform represented in the neural correlates. The second apparatus produces a stimulation intended to induce in the second subject the desired mental state, e.g., representing the same mental state as was present in the first subject.
A typically process performed on the neural correlates is a filtering to remove noise. For example, noise filters may be provided at 50 Hz, 60 Hz, 100 Hz, 120 Hz, and additional overtones (e.g., secondary harmonics). Other environmental signals may also be filtered in a frequency-selective or waveform-selective (temporal) manner. Higher level filtering may also be employed, as is known in the art. The neural correlates, after noise filtering, may be encoded, compressed (lossy or losslessly), encrypted, or otherwise processed or transformed. The stimulator associated with the second subject (trainee or recipient) would typically perform decoding, decompression, decryption, inverse transformation, etc.
Information security and copy protection technology, similar to that employed for audio signals, may be employed to protect the neural correlate signals from copying or content analysis before use. In some cases, it is possible to use the stored encrypted signal in its encrypted for, without decryption. For example, with an asymmetric encryption scheme, which supports distance determination.
In practice, the feedback signal from the second subject may be correspondingly encoded as per the source signal, and the error between the two minimized. In such an algorithm, the signal sought to be authenticated is typically brought within an error tolerance of the encrypted signal before usable feedback is available. One way to accomplish this is to provide a predetermined range of acceptable authenticatable signals which are then encoded, such that an authentication occurs when the putative signal matches any of the predetermined range. In the case of the neural correlates, a large set of digital hash patterns may be provided representing different signals as hash patterns. The net result is relatively weakened encryption, but the cryptographic strength may still be sufficiently high to abate the risks.
According to one embodiment, the processor may perform a noise reduction distinct from a frequency-band filtering. According to one embodiment, the neural correlates is transformed into a sparse matrix, and in the transform domain, components representing high probability noise are masked, while components representing high probability signal are preserved. The distinction may be optimized or adaptive. That is, in some cases, the components which represent modulation that are important may not be known a priori. However, dependent on their effect in inducing the desired response in the second subject, the “important” components may be identified, and the remainder filtered or suppressed. The transformed signal may then be inverse-transformed, and used as a basis for a stimulation signal.
According to another embodiment, a method of mental state modification, e.g., brain entrainment, is provided, comprising: ascertaining a mental state in a plurality of first subjects; acquiring brain waves of the plurality of first subjects (trainer or donors), e.g., using one of EEG and MEG, to create a dataset containing brain waves corresponding to different mental states. The database may be encoded with a classification of mental states, activities, environment, or stimulus patterns, applied to the plurality of first subjects, and the database may include acquired brain waves across a large number of mental states, activities, environment, or stimulus patterns, for example. In many cases, the database records will reflect a characteristic or dominate frequency of the respective brain waves.
The database may be accessed according to its indexing, e.g., mental states, activities, environment, or stimulus patterns, for example, and a stimulation pattern for a second subject (trainee or recipient) defined based on the database records of one or more subjects.
The record(s) thus retrieved are used to define a stimulation pattern for the second subject (trainee or recipient). The selection of records, and their use, may be dependent on the second subject and/or feedback from the second subject. As a relatively trivial example, a female second subject could be stimulated principally dependent on records from female first subjects. Of course, a more nuanced approach is to process the entirety of the database and stimulate the second subject based on a global brain wave-stimulus model, though this is not required, and also, the underlying basis for the model may prove unreliable or inaccurate. In fact, it may be preferred to derive a stimulus waveform from only a single first subject, in order to preserve micro-modulation aspects of the signal, which as discussed above have not been fully characterized. However, the selection of the first subject(s) need not be static, and can change frequently. The selection of first subject records may be based on population statistics of other users of the records, i.e., whether or not the record had the expected effect, collaborative filtering, i.e., filtering the first subjects whose response pattern correlates highest with a given subject, etc. The selection of first subject records may also be based on feedback patterns from the second subject.
The process of stimulation typically seeks to target a desired mental state in the second subject (trainee or recipient), which is automatically or semi-automatically determined or manually entered. That target then represents a part of the query against the database to select the desired record(s). The selection of records may be a dynamic process, and reselection of records may be feedback dependent.
In one embodiment, the records are used to define a modulation waveform of a synthesized carrier or set of carriers, and the process may include a frequency domain multiplexed multi-subcarrier signal (which is not necessarily orthogonal). A plurality of stimuli may be applied concurrently, through the suffered subchannels and/or though different stimulator electrodes, magnetic field generators, mechanical stimulators, sensory stimulators, etc. The stimuli for the different subchannels or modalities need not be derived from the same records.
The stimulus may be applied to achieve brain entrainment (i.e., synchronization) of the second subject (trainee or recipient) with one or more first subjects (trainer or donor). Brain entrainment is not the only possible outcome of this process. If the plurality of first subjects are mutually entrained, then each will have a corresponding brain wave pattern dependent on the basis of brainwave entrainment. This link between first subject may be helpful in determining compatibility between a respective first subject and the second subject. For example, characteristic patterns in the entrained brainwaves may be determined, even for different target mental states, and the characteristic patterns correlated to find relatively close matches and to exclude relatively poor matches.
This technology may also provide a basis for a social network, dating site, employment or vocational testing, or other interpersonal environments, wherein people may be matched with each other based on entrainment characteristics. For example, people who efficiently entrain with each other may have better social relationships than those who do not.
As discussed above, the plurality of first subjects (trainers or donors) may have their respective brain wave patterns stored in association with separate database records. However, they may also be combined into a more global model. One such model is a neural network or deep neural network. Typically, such a network would have recurrent features. Data from a plurality of first subjects (trainers or donors) is used to train the neural network, which is then accessed by inputting the target state and/or feedback information, and which outputs a stimulation pattern or parameters for controlling a stimulator. When multiple first subjects form the basis for the stimulation pattern, it is preferred that the neural network output parameters of the stimulation, derived from and comprising features of the brain wave patterns or other neural correlates of mental state from the plurality of first subjects, which are then used to control a stimulator which, for example, generates its own carrier wave(s) which are then modulated based on the output of the neural network. The neural network need not periodically retrieve records, and therefore may operate in a more time-continuous manner, rather than the more segmented scheme of record-based control.
In any of the feedback dependent methods, the brainwave patterns or other neural correlates of mental states may be processed by a neural network, to produce an output that guides or controls the stimulation. The stimulation, is, for example, at least one of a light signal, a sound signal, an electric signal, a magnetic field, and a vibration or mechanical stimulus. The fields may be static or dynamically varying.
The process may employ a relational database of mental states and brainwave patterns, e.g., frequencies/neural correlate waveform patterns associated with the respective mental states. The relational database may comprise a first table, the first table further comprising a plurality of data records of brainwave patterns, and a second table, the second table comprising a plurality of mental states, each of the mental states being linked to at least one brainwave pattern. Data related to mental states and brainwave patterns associated with the mental states are stored in the relational database and maintained. The relational database is accessed by receiving queries for selected mental states, and data records are returned representing the associated brainwave pattern. The brainwave pattern retrieved from the relational database may then be used for modulating a stimulator seeking to produce an effect selectively dependent on the mental state at issue.
A further aspect of the technology provides a computer apparatus for creating and maintaining a relational database of mental states and frequencies associated with the mental state. The computer apparatus may comprise a non-volatile memory for storing a relational database of mental states and neural correlates of brain activity associated with the mental states, the database comprising a first table comprising a plurality of data records of neural correlates of brain activity associated with the mental states, and a second table comprising a plurality of mental states, each of the mental states being linked to one or more records in the first table; a processor coupled with the non-volatile memory, and being configured to process relational database queries, which are then used for searching the database; RAM coupled with the processor and the non-volatile memory for temporary holding database queries and data records retrieved from the relational database; and an IO interface configured to receive database queries and deliver data records retrieved from the relational database. A SQL or noSQL database may also be used to store and retrieve records. A relational database described above maintained and operated by a general purpose computer, improves the operations of the general purpose computer by making searches of specific mental states and brainwaves associated therewith more efficient.
A further aspect of the technology provides a method of brain entrainment comprising: ascertaining a mental state in a first subject; recording brain waves of the plurality of subjects using at least one channel one of EEG and MEG; storing the recorded brain waves in a physical memory device; retrieving the brain waves from the memory device; applying a stimulus signal comprising a brainwave pattern derived from at least one-channel one of the EEG and MEG to a second subject via transcranial stimulation, whereby the mental state desired by the second subject is achieved. The stimulation may be of the same order (number of channels) as the EEG or MEG, or a different number of channels, typically reduced. For example, the EEG or MEG may comprise 128 or 256 channels, while the transcranial stimulator may have 8 or fewer channels. Sensory stimulation of various modalities and patterns may accompany the transcranial stimulation.
It is therefore an object to provide a method of inducing sleep in a mammal, comprising: retrieving a stored modulation sequence for at least one of an audio signal and a visual signal derived from captured neural correlates of at least one sleep cycle, which is adapted, when used to stimulate the mammal, to induce at least one sleep cycle in the mammal; and presenting the at least one of the audio signal and the visual signal having the modulation sequence to the mammal, to induce the mammal to enter a respective sleep cycle.
It is a further object to provide a system for inducing sleep in a subject, comprising: a memory configured to store a waveform, derived from neural correlates of at least one sleep stage, processed to derive at least one of a temporal and a spatial pattern transformed into the waveform for modulating at least one of a visual stimulus and an auditory stimulus; and at least one of a visual stimulator and an auditory stimulator, configured to retrieve the waveform and modulate the waveform on at least one of a visual stimulus and an auditory stimulus, for use in stimulating a subject to induce a sleep.
It is another object to provide a computer readable medium, storing non-transitory instructions for controlling a programmable processor for presenting at least one of a visual stimulus and an auditory stimulus to a subject to induce sleep, comprising: instructions to define a waveform pattern derived from neural correlates of at least one sleep stage, processed to derive at least one of a temporal and a spatial pattern transformed into the waveform; instructions to modulate the waveform on at least one of a visual stimulus and an auditory stimulus for presentation to the subject; and instructions for controlling the modulation of the waveform according to a sleep cycle.
It is also an object to provide a method of inducing a sleep cycle in a subject, comprising modulating a waveform on at least one of a visual stimulus and an auditory stimulus for use in visual or auditory stimulation of the subject, to induce the sleep cycle in the subject, wherein the waveform is generated by recording neural correlates of at least one sleep cycle of a donor, processing the recorded neural correlates of the at least one sleep cycle to decode at least one of a temporal and a spatial pattern; and transforming the at least one of the temporal and the spatial pattern into the waveform.
The recording of the neural correlates of at least one sleep stage of a donor may be done by recording at least one of an electroencephalogram (EEG), a magnetoencephalogram (MMG), and a functional magnetic resonance (fMRI). The recorded neural correlates of at the sleep stage may be processed using a statistical analysis technique. The statistical analysis technique may be one of a Principal Component Analysis (PCA), a Spatial Principal Component Analysis (Spatial PCA), a Kernel Principal Component Analysis (Kernel PCA), a Nonlinear Principal Component Analysis (NLPCA), an Independent Component Analysis (ICA), Singular Value Decomposition (SVD), Factor Analysis, a Gaussian Process Latent Variable Model (GPLVM), a Curvilinear Component Analysis (CCA), a Diffeomorphic Dimensionality Reduction (Diffeomap), a Gelfand transform, a Fourier transform, a Fourier-Pontryagin transform, a Laplace transform, a short-time Fourier transform (SIFT), a fractional Fourier transform (FRFT), Laplacian Eigenmaps, a spectral analysis, a wavelet analysis, an eigenvector-based multivariable analysis, a factor analysis, canonical correlation analysis (CCA), and a nonlinear dimensionality reduction (NLDR).
The transforming of the at least one of the temporal and a spatial pattern into the waveform may comprise performing an inversion transform on the at least one of the temporal and the spatial pattern of the donor into the waveform, adapted to expose the at least one of the temporal and the spatial pattern to the subject.
The subject and the donor may be a single human at different times.
The at least one of a light stimulus and a sound stimulus may be frequency modulated, amplitude modulated, pulse rate modulate, pulse frequency modulated, and/or phase modulated corresponding to the waveform.
The at least one of the light stimulus and the sound stimulus may be modulated to provide at least one of a binaural stimulation and an isochronic tones stimulation. The sound stimulus may comprise at least one of an audible sound frequency, an ultrasonic sound frequency, and an infrasonic sound frequency. The light stimulus may be at least one of amplitude modulated and pulse modulated. The sound stimulus may comprise at least one of a random noise, music, a sound of rainfall, a sound of ocean waves, and a sound of a human voice.
The sleep cycle may comprise a plurality of sleep stages, comprising a REM sleep stage, and/or a non-REM sleep stage. The sleep cycle may comprise a natural sequence of sleep stages comprising at least one full sleep cycle, e.g., at least two recorded sleep cycles. The method may comprise monitoring the sleep stage of the subject, distinct from recording the neural correlates, and adapting the waveform selectively dependent on the monitored sleep stage.
The subject may be stimulated with at least one of a visual stimulus and an auditory stimulus modulated according to a first portion of the waveform, correlating the stimulation of the subject with a respective sleep stage of the subject during stimulation, and selecting a second portion of the waveform representative of the at least one sleep stage of the donor that corresponds to the respective sleep stage of the subject. The method may further comprise stimulating the subject with the at least one of the visual stimulus and the auditory stimulus, and modifying the stimulation after determining that the subject is not in the sleep stage sought to be induced corresponding to the at least one sleep stage of the donor.
The method may further comprise adjusting an ambient temperature surrounding the subject for a specific sleep stage corresponding to the at least one sleep stage of the donor.
The method may further comprise superimposing on said at least one of the light stimulus and the sound stimulus a signal having a rhythm having a frequency less than approximately 100 Hz. The frequency may be approximately 40 Hz. In this case, the approximation means, for example, within 13%, or a signal having a corresponding physiological effect in the subject as a signal of that frequency in a mean normal male Caucasian adult, to the extent that the physiological effect varies across gender, race, size, condition, of other characteristics. The at least one channel may be less than six channels and the placement of electrodes used for transcranial stimulation may be approximately the same as the placement of electrodes used in recording of said one of EEG and MEG.
The present technology may be responsive to chronobiology, and in particular to the subjective sense of time. For a subject, this may be determined volitionally subjectively, but also automatically, for example by judging attention span, using e.g., eye movements, and analyzing persistence of brainwave patterns after a discrete stimulus. Further, timeconstants of the brain, reflected by delays and phase may also be analyzed. Further, the contingent negative variation (CNV) preceding a volitional act may be used, both to sense conscious action timing, and also the time relationships between thought and action more generally.
Typically, brainwave activity is measured with a large number of EEG electrodes, which each receive signals from a small area on the scalp, or in the case of an MEG, by a number of sensitive magnetic field detectors which are responsive to local field differences. Typically, the brainwave capture is performed in a relatively high number of spatial dimensions, e.g., corresponding to the number of sensors. Typically, it is infeasible to process the brainwave signals to create a source model, given that the brainwaves are created by billions of neurons, connected through axons which have long distances. Further, the neurons are generally no-linear, and interconnected. However, a source model is not required.
Various types of artificial intelligence may be exploited to analyze the neural correlates of mental state represented in the brain activity data, of both the first subject (source) and the second subject (target). The algorithm or implementation need to be the same, though in some cases, it is useful to conform the approach of the source processing and feedback processing so that the feedback does not achieve or seek a suboptimal target mental state. However, given the possible differences in conditions, resources, equipment, and purpose, there is no necessary coordination of these processes. The artificial intelligence may take the form of neural networks or deep neural networks, though rule/expert based systems, hybrids, and more classical statistical analysis may be used. In a typical case, an artificial intelligence process will have at least one aspect which is non-linear in its output response to an input signal, and thus at least the principle of linear superposition is violated. Such systems tend to permit discrimination, since a decision, and the process of decision-making, is ultimately non-linear. An artificially intelligent system requires a base of experience or information upon which to train. This can be a supervised (external labels applied to data), unsupervised (self-discrimination of classes), or semisupervised (a portion of the data is externally labelled). A self-learning or genetic algorithm may be used to tune the system, including both or either the signal processing at the trainer system and the target system. In a genetic algorithm feedback-dependent self-learning system, the responsivity of a subject, e.g., the target, to various kinds of stimuli may be determined over a stimulus space. This stimulation may be in the context of use, with a specific target mental state provided, or unconstrained. The stimulator may operate using a library of stimulus patterns, or seek to generate synthetic patterns or modifications of patterns. Over a period of time, the system will learn to map a desired mental state to optimal context-dependent parameters of the stimulus pattern.
The technology may be used for both the creation of mental states in the target, elimination of existing mental states in the target. In the latter case, a decision of what end state is to be achieved is less constrained, and therefore the optimization is distinct. For example, in the former case, it may be hard to achieve a particular mental state that is desired, requiring a set of transitions to cause the brain of the target to be enabled/prepared to enter the target state. In the case of a system seeking to eliminate an undesired mental state, the issue is principally what path to take to most efficiently leave the current state, bearing in mind the various costs, such as the comfort/discomfort of the stimulation, the time value cost, etc. Therefore, the series of states may differ in the implementation of these distinct goals, even if the endpoints are identical, i.e., the optimal algorithm to achieve state B from state A, may be different from the optimal algorithm to exist state A, and end up at state B.
The technology may be used to address mental states classified as emotions. Typically, an emotional state at a lower level, and operates in different brain regions, than cognitive processes. As such, the biology of these mental state is different. Often, the emotional states have a biochemical or hormonal component, and perhaps a physiological component, that may be attenuated or absent from cognitive states. Therefore, while the general brainwave or other neural correlates acquisition may be similar or identical, the stimulus used on the second subject may be distinct, in modality, spatial location, intensity/waveform, other stimulation parameters, and the types and application of feedback employed. In e medical treatment implementation, in some cases it may be appropriate to administer a drug or pharmacological agent that assists in achieving the target mental state, and for emotional states, this may include certain psychotropic drugs, such as epinephrine, norepinephrine reuptake inhibitors, serotonin reuptake inhibitors, peptide endocrine hormones, such as oxytocin, ACTH fragments, insulin, etc.
The detailed description is described with reference to the accompanying figures. In the figures, the left-most digit(s) of a reference number identifies the figure in which the reference number first appears. The use of the same reference number in different figures indicates similar or identical items.
Hereinafter, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings so that the present disclosure may be readily implemented by those skilled in the art. However, it is to be noted that the present disclosure is not limited to the embodiments but can be embodied in various other ways. In drawings, parts irrelevant to the description are omitted for the simplicity of explanation, and like reference numerals denote like parts through the whole document.
Through the whole document, the term “connected to” or “coupled to” that is used to designate a connection or coupling of one element to another element includes both a case that an element is “directly connected or coupled to” another element and a case that an element is “electronically connected or coupled to” another element via still another element. Further, it is to be understood that the term “comprises or includes” and/or “comprising or including” used in the document means that one or more other components, steps, operation and/or existence or addition of elements are not excluded in addition to the described components, steps, operation and/or elements unless context dictates otherwise.
Through the whole document, the term “unit” or “module” includes a unit implemented by hardware or software and a unit implemented by both of them. One unit may be implemented by two or more pieces of hardware, and two or more units may be implemented by one piece of hardware.
Each of the following references is expressly incorporated herein by reference in its entirety.
U.S. Pat. Nos. 2,858,388; 3,951,134; 4,172,014; 4,296,756; 4,367,527; 4,407,299; 4,408,616; 4,421,122; 4,437,064; 4,493,327; 4,550,736; 4,557,270; 4,562,540; 4,579,125; 4,583,190; 4,585,011; 4,591,787; 4,594,662; 4,610,259; 4,613,817; 4,649,482; 4,689,559; 4,693,000; 4,700,135; 4,705,049; 4,733,180; 4,736,307; 4,736,751; 4,744,029; 4,749,946; 4,753,246; 4,761,611; 4,776,345; 4,792,145; 4,794,533; 4,801,882; 4,846,190; 4,862,359; 4,883,067; 4,907,597; 4,913,152; 4,924,875; 4,937,525; 4,940,058; 4,947,480; 4,949,725; 4,951,674; 4,974,602; 4,977,505; 4,982,157; 4,983,912; 4,996,479; 5,008,622; 5,010,891; 5,012,190; 5,020,538; 5,020,540; 5,027,817; 5,029,082; 5,059,814; 5,061,680; 5,069,218; 5,070,399; 5,083,571; 5,088,497; 5,092,341; 5,092,835; 5,095,270; 5,105,354; 5,109,862; 5,118,606; 5,126,315; 5,136,687; 5,158,932; 5,159,703; 5,159,928; 5,166,614; 5,187,327; 5,198,977; 5,213,338; 5,215,086; 5,218,530; 5,224,203; 5,230,344; 5,230,346; 5,231,988; 5,233,517; 5,241,967; 5,243,281; 5,243,517; 5,263,488; 5,265,611; 5,269,315; 5,269,325; 5,273,038; 5,280,791; 5,282,474; 5,283,523; 5,287,859; 5,291,888; 5,293,187; 5,299,569; 5,303,705; 5,306,228; 5,307,807; 5,309,095; 5,309,917; 5,309,923; 5,311,129; 5,320,109; 5,323,777; 5,325,862; 5,326,745; 5,331,970; 5,335,657; 5,339,811; 5,339,826; 5,343,871; 5,359,363; 5,377,100; 5,384,588; 5,406,956; 5,406,957; 5,409,445; 5,417,211; 5,418,512; 5,422,689; 5,442,289; 5,443,073; 5,447,154; 5,447,166; 5,458,117; 5,458,142; 5,459,536; 5,461,699; 5,469,057; 5,474,082; 5,476,438; 5,491,492; 5,496,798; 5,503,149; 5,513,649; 5,515,301; 5,522,863; 5,546,943; 5,552,375; 5,555,889; 5,568,816; 5,571,150; 5,579,241; 5,594,849; 5,600,243; 5,601,081; 5,611,350; 5,617,856; 5,619,995; 5,622,168; 5,626,145; 5,632,272; 5,640,493; 5,643,325; 5,649,061; 5,650,726; 5,656,937; 5,662,109; 5,671,740; 5,678,561; 5,682,889; 5,685,313; 5,692,517; 5,694,939; 5,699,808; 5,701,909; 5,706,402; 5,706,811; 5,711,305; 5,715,821; 5,719,561; 5,720,619; 5,722,418; 5,724,987; 5,729,046; 5,730,146; 5,736,543; 5,737,485; 5,740,812; 5,742,748; 5,743,854; 5,743,860; 5,747,492; 5,752,514; 5,752,521; 5,752,911; 5,755,227; 5,755,739; 5,761,332; 5,762,611; 5,767,043; 5,771,261; 5,771,893; 5,771,894; 5,771,897; 5,791,342; 5,794,623; 5,795,304; 5,797,840; 5,797,853; 5,810,737; 5,813,993; 5,815,413; 5,816,247; 5,825,830; 5,827,195; 5,840,040; 5,842,986; 5,845,639; 5,846,189; 5,846,208; 5,853,005; 5,857,978; 5,859,533; 5,871,517; 5,877,801; 5,884,626; 5,885,976; 5,891,131; 5,899,867; 5,911,581; 5,916,171; 5,921,245; 5,928,272; 5,938,598; 5,938,688; 5,954,662; 5,970,499; 5,971,923; 5,983,129; 5,995,868; 5,999,856; 6,002,254; 6,002,952; 6,011,990; 6,011,991; 6,016,444; 6,021,345; 6,023,161; 6,026,173; 6,032,072; 6,042,548; 6,044,292; 6,050,940; 6,050,962; 6,052,619; 6,053,739; 6,057,846; 6,066,084; 6,067,462; 6,067,467; 6,069,369; 6,070,098; 6,071,246; 6,080,164; 6,081,735; 6,088,611; 6,092,058; 6,097,980; 6,097,981; 6,099,319; 6,104,956; 6,115,631; 6,117,075; 6,129,681; 6,132,724; 6,144,872; 6,149,586; 6,154,026; 6,155,966; 6,155,993; 6,157,850; 6,157,857; 6,161,031; 6,167,298; 6,167,311; 6,171,239; 6,171,258; 6,182,013; 6,188,924; 6,195,576; 6,196,972; 6,205,359; 6,208,902; 6,224,549; 6,226,418; 6,230,037; 6,236,872; 6,239,145; 6,240,308; 6,241,686; 6,248,126; 6,259,399; 6,263,189; 6,266,453; 6,269,270; 6,272,370; 6,280,393; 6,287,328; 6,290,638; 6,292,688; 6,293,904; 6,294,917; 6,298,259; 6,305,943; 6,306,077; 6,309,342; 6,309,361; 6,315,736; 6,317,627; 6,319,205; 6,322,515; 6,325,475; 6,325,761; 6,331,164; 6,332,087; 6,338,713; 6,339,725; 6,341,236; 6,343,229; 6,354,087; 6,354,299; 6,356,079; 6,356,781; 6,356,788; 6,358,201; 6,364,845; 6,366,813; 6,366,814; 6,370,414; 6,370,423; 6,374,131; 6,375,614; 6,377,833; 6,385,479; 6,385,486; 6,390,979; 6,393,363; 6,394,963; 6,402,520; 6,402,689; 6,408,107; 6,418,344; 6,419,629; 6,427,086; 6,428,490; 6,430,443; 6,435,878; 6,442,421; 6,442,948; 6,466,816; 6,470,220; 6,475,163; 6,482,165; 6,487,441; 6,488,617; 6,490,472; 6,493,577; 6,496,724; 6,497,658; 6,497,699; 6,503,085; 6,507,754; 6,510,340; 6,511,424; 6,516,246; 6,520,905; 6,520,921; 6,522,906; 6,524,249; 6,526,297; 6,526,415; 6,527,715; 6,527,730; 6,529,759; 6,529,773; 6,530,884; 6,534,986; 6,538,436; 6,539,245; 6,539,263; 6,544,170; 6,546,378; 6,547,736; 6,547,746; 6,549,804; 6,551,243; 6,553,252; 6,556,695; 6,556,861; 6,556,868; 6,557,558; 6,560,486; 6,565,518; 6,574,573; 6,587,727; 6,587,729; 6,591,132; 6,591,137; 6,594,524; 6,597,954; 6,602,202; 6,603,502; 6,609,030; 6,611,698; 6,615,158; 6,616,611; 6,622,036; 6,622,047; 6,625,485; 6,626,676; 6,633,686; 6,644,976; 6,648,822; 6,648,880; 6,650,917; 6,652,458; 6,652,470; 6,654,632; 6,654,729; 6,656,137; 6,658,287; 6,663,571; 6,665,552; 6,665,553; 6,665,562; 6,671,555; 6,671,556; 6,678,548; 6,684,098; 6,684,105; 6,687,525; 6,695,761; 6,697,660; 6,699,194; 6,701,173; 6,703,838; 6,708,051; 6,708,064; 6,708,184; 6,709,399; 6,725,080; 6,726,624; 6,728,424; 6,728,564; 6,731,975; 6,735,460; 6,735,467; 6,735,475; 6,740,032; 6,743,167; 6,743,182; 6,745,060; 6,745,156; 6,746,409; 6,751,499; 6,758,813; 6,768,920; 6,773,400; 6,774,929; 6,775,405; 6,782,292; 6,785,409; 6,788,975; 6,791,331; 6,795,724; 6,798,898; 6,801,648; 6,801,803; 6,804,558; 6,804,661; 6,815,949; 6,816,744; 6,819,956; 6,826,426; 6,843,774; 6,853,186; 6,856,830; 6,863,127; 6,865,494; 6,873,872; 6,875,174; 6,876,196; 6,879,859; 6,882,881; 6,885,192; 6,885,886; 6,886,964; 6,893,407; 6,896,655; 6,907,280; 6,915,241; 6,920,357; 6,926,921; 6,928,354; 6,931,274; 6,931,275; 6,936,012; 6,947,790; 6,950,697; 6,950,698; 6,959,215; 6,961,618; 6,963,770; 6,963,771; 6,978,179; 6,980,863; 6,981,947; 6,983,184; 6,983,264; 6,985,769; 6,988,056; 6,990,377; 6,993,380; 6,996,261; 6,996,549; 7,003,352; 7,006,872; 7,010,340; 7,010,351; 7,011,410; 7,011,814; 7,014,613; 7,016,722; 7,022,083; 7,023,206; 7,024,247; 7,030,617; 7,035,686; 7,037,260; 7,038,450; 7,039,266; 7,039,547; 7,043,293; 7,053,610; 7,054,454; 7,062,391; 7,063,535; 7,070,571; 7,079,977; 7,089,927; 7,092,748; 7,099,714; 7,104,947; 7,104,963; 7,105,824; 7,107,090; 7,116,102; 7,117,026; 7,119,553; 7,120,486; 7,123,955; 7,127,100; 7,128,713; 7,130,673; 7,130,675; 7,130,691; 7,145,333; 7,146,211; 7,146,217; 7,146,218; 7,149,572; 7,149,773; 7,150,710; 7,150,715; 7,150,717; 7,150,718; 7,151,961; 7,155,279; 7,163,512; 7,164,941; 7,167,751; 7,170,294; 7,171,252; 7,171,339; 7,174,206; 7,176,680; 7,177,675; 7,177,678; 7,181,505; 7,183,381; 7,184,837; 7,186,209; 7,187,169; 7,190,826; 7,190,995; 7,193,413; 7,196,514; 7,197,352; 7,199,708; 7,203,548; 7,207,948; 7,209,787; 7,209,788; 7,212,851; 7,215,986; 7,215,994; 7,218,104; 7,221,981; 7,222,964; 7,224,282; 7,225,013; 7,228,167; 7,228,169; 7,228,171; 7,228,178; 7,231,245; 7,231,254; 7,236,830; 7,236,831; 7,239,731; 7,239,926; 7,242,983; 7,242,984; 7,252,090; 7,254,433; 7,254,439; 7,254,500; 7,257,439; 7,258,659; 7,260,430; 7,267,644; 7,267,652; 7,269,455; 7,269,456; 7,269,516; 7,276,916; 7,277,758; 7,278,966; 7,280,861; 7,280,867; 7,280,870; 7,282,030; 7,283,861; 7,286,871; 7,288,066; 7,292,890; 7,295,019; 7,297,110; 7,299,088; 7,299,096; 7,302,298; 7,305,268; 7,309,315; 7,313,442; 7,321,837; 7,324,845; 7,324,851; 7,328,053; 7,330,032; 7,333,619; 7,333,851; 7,334,892; 7,338,171; 7,338,455; 7,340,125; 7,340,289; 7,343,198; 7,346,382; 7,346,395; 7,353,064; 7,353,065; 7,355,597; 7,359,837; 7,363,164; 7,366,571; 7,367,807; 7,367,949; 7,369,896; 7,371,365; 7,373,198; 7,376,453; 7,376,459; 7,378,056; 7,381,185; 7,383,070; 7,383,237; 7,386,347; 7,389,144; 7,392,079; 7,394,246; 7,395,292; 7,396,333; 7,399,282; 7,400,984; 7,403,809; 7,403,814; 7,403,815; 7,403,820; 7,407,485; 7,409,321; 7,418,290; 7,420,033; 7,422,555; 7,429,247; 7,437,196; 7,440,789; 7,440,806; 7,444,184; 7,450,986; 7,453,263; 7,454,240; 7,454,243; 7,454,245; 7,454,387; 7,457,653; 7,457,665; 7,461,045; 7,462,151; 7,462,155; 7,463,024; 7,463,142; 7,463,927; 7,466,132; 7,468,040; 7,468,350; 7,469,697; 7,471,971; 7,471,978; 7,478,108; 7,482,298; 7,483,747; 7,486,986; 7,488,294; 7,489,958; 7,489,964; 7,490,085; 7,491,173; 7,493,171; 7,493,172; 7,496,393; 7,497,828; 7,499,741; 7,499,745; 7,499,752; 7,499,894; 7,502,720; 7,509,154; 7,509,161; 7,509,163; 7,510,531; 7,510,699; 7,515,054; 7,530,955; 7,537,568; 7,539,528; 7,539,532; 7,539,533; 7,539,543; 7,547,284; 7,553,810; 7,558,622; 7,559,903; 7,561,918; 7,565,193; 7,565,199; 7,565,200; 7,565,809; 7,567,693; 7,570,054; 7,570,991; 7,572,225; 7,573,264; 7,573,268; 7,574,007; 7,574,254; 7,577,472; 7,577,481; 7,580,798; 7,582,062; 7,583,857; 7,593,767; 7,594,122; 7,594,889; 7,596,535; 7,597,665; 7,603,168; 7,603,174; 7,604,603; 7,606,405; 7,608,579; 7,610,083; 7,610,094; 7,610,096; 7,610,100; 7,613,502; 7,613,519; 7,613,520; 7,617,002; 7,618,381; 7,620,455; 7,620,456; 7,623,912; 7,623,927; 7,623,928; 7,624,293; 7,625,340; 7,627,370; 7,629,889; 7,630,757; 7,634,317; 7,640,055; 7,643,655; 7,643,881; 7,647,097; 7,647,098; 7,648,498; 7,649,351; 7,653,433; 7,654,948; 7,657,316; 7,668,579; 7,668,591; 7,670,838; 7,672,707; 7,672,717; 7,672,730; 7,676,263; 7,678,047; 7,678,061; 7,678,767; 7,680,526; 7,680,540; 7,684,856; 7,684,858; 7,684,866; 7,684,867; 7,697,979; 7,702,387; 7,702,502; 7,706,871; 7,706,992; 7,711,417; 7,711,432; 7,714,936; 7,715,894; 7,715,910; 7,715,919; 7,720,519; 7,720,530; 7,725,174; 7,725,192; 7,727,161; 7,729,740; 7,729,753; 7,729,755; 7,729,773; 7,733,224; 7,733,973; 7,734,334; 7,734,340; 7,734,355; 7,736,382; 7,737,687; 7,738,683; 7,740,592; 7,742,820; 7,746,979; 7,747,318; 7,747,325; 7,747,326; 7,747,551; 7,749,155; 7,751,877; 7,751,878; 7,753,836; 7,754,190; 7,756,564; 7,756,568; 7,756,584; 7,757,690; 7,758,503; 7,763,588; 7,764,987; 7,765,088; 7,766,827; 7,769,424; 7,769,431; 7,769,461; 7,769,464; 7,771,341; 7,771,364; 7,774,052; 7,774,064; 7,775,993; 7,778,490; 7,778,692; 7,778,693; 7,783,362; 7,787,937; 7,787,946; 7,792,575; 7,794,403; 7,794,406; 7,797,040; 7,800,493; 7,801,591; 7,801,592; 7,801,593; 7,801,601; 7,801,686; 7,803,118; 7,803,119; 7,804,441; 7,805,203; 7,809,433; 7,809,434; 7,811,279; 7,819,794; 7,819,812; 7,822,481; 7,829,562; 7,831,302; 7,831,305; 7,834,627; 7,835,787; 7,840,039; 7,840,248; 7,840,250; 7,840,257; 7,840,280; 7,841,986; 7,844,324; 7,848,803; 7,852,087; 7,853,321; 7,853,322; 7,853,323; 7,853,329; 7,856,264; 7,860,548; 7,860,552; 7,860,561; 7,860,570; 7,863,272; 7,865,234; 7,865,235; 7,865,244; 7,869,867; 7,869,884; 7,869,885; 7,872,235; 7,873,411; 7,876,938; 7,878,965; 7,879,043; 7,881,760; 7,881,770; 7,881,780; 7,882,135; 7,884,101; 7,887,493; 7,890,155; 7,890,176; 7,890,185; 7,891,814; 7,892,764; 7,894,890; 7,894,903; 7,895,033; 7,896,807; 7,899,524; 7,899,525; 7,899,539; 7,899,545; 7,901,211; 7,904,134; 7,904,139; 7,904,144; 7,904,151; 7,904,175; 7,904,507; 7,907,994; 7,907,998; 7,908,008; 7,908,009; 7,909,771; 7,912,530; 7,917,199; 7,917,206; 7,917,221; 7,917,225; 7,918,779; 7,920,914; 7,920,915; 7,920,916; 7,925,353; 7,929,693; 7,930,035; 7,932,225; 7,933,645; 7,933,646; 7,933,727; 7,937,138; 7,937,152; 7,937,222; 7,938,782; 7,938,785; 7,941,209; 7,942,824; 7,944,551; 7,945,304; 7,945,316; 7,945,330; 7,957,796; 7,957,797; 7,957,806; 7,957,809; 7,961,922; 7,962,204; 7,962,214; 7,962,219; 7,962,220; 7,970,734; 7,972,278; 7,974,688; 7,974,693; 7,974,696; 7,974,697; 7,974,701; 7,974,787; 7,976,465; 7,983,740; 7,983,741; 7,983,757; 7,983,762; 7,986,991; 7,988,613; 7,988,969; 7,991,461; 7,991,477; 7,993,279; 7,996,075; 7,996,079; 8,000,767; 8,000,773; 8,000,788; 8,000,793; 8,000,794; 8,000,795; 8,001,179; 8,002,553; 8,005,534; 8,005,624; 8,005,894; 8,010,178; 8,010,347; 8,012,107; 8,014,847; 8,014,870; 8,016,597; 8,019,400; 8,019,410; 8,024,029; 8,024,032; 8,025,404; 8,027,730; 8,029,553; 8,031,076; 8,032,209; 8,032,229; 8,032,486; 8,033,996; 8,036,434; 8,036,728; 8,036,736; 8,036,745; 8,041,136; 8,041,418; 8,041,419; 8,046,041; 8,046,042; 8,046,076; 8,050,768; 8,055,348; 8,055,591; 8,059,879; 8,060,181; 8,060,194; 8,064,994; 8,065,011; 8,065,012; 8,065,017; 8,065,240; 8,065,360; 8,066,637; 8,066,647; 8,068,904; 8,068,911; 8,069,125; 8,073,534; 8,073,546; 8,073,631; 8,075,499; 8,079,953; 8,082,031; 8,082,033; 8,082,215; 8,083,786; 8,086,294; 8,086,296; 8,086,563; 8,088,057; 8,089,283; 8,090,164; 8,092,549; 8,095,209; 8,095,210; 8,097,926; 8,099,299; 8,103,333; 8,108,033; 8,108,036; 8,108,038; 8,108,039; 8,108,042; 8,112,148; 8,112,153; 8,114,021; 8,116,874; 8,116,877; 8,116,883; 8,121,361; 8,121,673; 8,121,694; 8,121,695; 8,126,228; 8,126,243; 8,126,528; 8,126,542; 8,126,567; 8,126,568; 8,128,572; 8,131,354; 8,131,526; 8,133,172; 8,135,472; 8,135,957; 8,137,269; 8,137,270; 8,140,152; 8,145,295; 8,145,310; 8,148,417; 8,148,418; 8,150,508; 8,150,523; 8,150,524; 8,150,796; 8,152,732; 8,155,726; 8,155,736; 8,160,273; 8,160,317; 8,160,680; 8,160,689; 8,160,696; 8,165,687; 8,167,784; 8,167,826; 8,170,315; 8,170,347; 8,172,759; 8,172,766; 8,174,430; 8,175,359; 8,175,360; 8,175,686; 8,175,696; 8,175,700; 8,177,724; 8,177,726; 8,177,727; 8,180,125; 8,180,148; 8,180,420; 8,180,436; 8,180,601; 8,185,186; 8,185,207; 8,185,382; 8,187,181; 8,187,201; 8,188,749; 8,190,227; 8,190,248; 8,190,249; 8,190,251; 8,190,264; 8,195,295; 8,195,298; 8,195,300; 8,195,593; 8,197,395; 8,197,437; 8,199,982; 8,199,985; 8,200,319; 8,200,340; 8,204,583; 8,204,603; 8,209,009; 8,209,018; 8,209,019; 8,209,224; 8,211,035; 8,212,556; 8,213,670; 8,214,007; 8,214,035; 8,219,188; 8,221,330; 8,222,378; 8,223,023; 8,224,431; 8,224,433; 8,224,444; 8,224,451; 8,229,540; 8,229,559; 8,233,682; 8,233,689; 8,233,965; 8,233,990; 8,235,907; 8,236,005; 8,236,038; 8,239,014; 8,239,028; 8,239,029; 8,239,030; 8,241,213; 8,244,340; 8,244,341; 8,244,347; 8,244,475; 8,244,552; 8,244,553; 8,248,069; 8,249,316; 8,249,698; 8,249,718; 8,249,815; 8,260,426; 8,262,714; 8,263,574; 8,267,851; 8,270,814; 8,271,077; 8,280,502; 8,280,503; 8,280,505; 8,280,514; 8,280,517; 8,285,351; 8,285,368; 8,290,575; 8,290,596; 8,295,914; 8,295,934; 8,295,935; 8,296,108; 8,298,078; 8,298,140; 8,301,222; 8,301,232; 8,301,233; 8,301,257; 8,303,636; 8,304,246; 8,305,078; 8,306,607; 8,306,610; 8,306,627; 8,308,646; 8,308,661; 8,311,622; 8,311,747; 8,311,748; 8,311,750; 8,313,441; 8,314,707; 8,315,703; 8,315,704; 8,315,710; 8,315,812; 8,315,813; 8,315,814; 8,315,962; 8,315,970; 8,320,649; 8,321,150; 8,323,188; 8,323,189; 8,323,204; 8,326,418; 8,326,420; 8,326,433; 8,328,718; 8,332,017; 8,332,024; 8,332,038; 8,332,041; 8,332,191; 8,334,690; 8,335,561; 8,335,664; 8,335,715; 8,335,716; 8,337,404; 8,340,752; 8,340,753; 8,340,771; 8,343,026; 8,343,027; 8,343,066; 8,346,331; 8,346,342; 8,346,349; 8,346,354; 8,346,365; 8,350,804; 8,352,023; 8,352,031; 8,353,837; 8,354,438; 8,354,881; 8,355,768; 8,356,004; 8,356,594; 8,358,818; 8,359,080; 8,362,780; 8,364,226; 8,364,254; 8,364,255; 8,364,271; 8,364,272; 8,369,940; 8,374,411; 8,374,412; 8,374,690; 8,374,696; 8,374,701; 8,374,703; 8,376,965; 8,379,947; 8,379,952; 8,380,289; 8,380,290; 8,380,296; 8,380,314; 8,380,316; 8,380,658; 8,382,667; 8,386,188; 8,386,244; 8,386,312; 8,386,313; 8,388,529; 8,388,530; 8,388,555; 8,391,942; 8,391,956; 8,391,966; 8,392,250; 8,392,251; 8,392,253; 8,392,254; 8,392,255; 8,396,542; 8,396,545; 8,396,546; 8,396,557; 8,396,565; 8,396,744; 8,398,692; 8,401,624; 8,401,626; 8,401,634; 8,401,654; 8,401,655; 8,401,666; 8,403,848; 8,406,838; 8,406,841; 8,406,848; 8,406,862; 8,406,890; 8,412,334; 8,412,335; 8,412,337; 8,412,338; 8,412,655; 8,415,123; 8,417,344; 8,423,118; 8,423,125; 8,423,144; 8,423,155; 8,423,297; 8,425,415; 8,425,583; 8,428,696; 8,428,703; 8,428,704; 8,428,726; 8,429,225; 8,430,805; 8,430,816; 8,431,537; 8,433,388; 8,433,410; 8,433,414; 8,433,418; 8,435,166; 8,437,843; 8,437,844; 8,437,861; 8,439,845; 8,442,626; 8,444,571; 8,445,021; 8,445,851; 8,447,392; 8,447,407; 8,447,411; 8,449,471; 8,452,387; 8,452,544; 8,454,555; 8,456,164; 8,456,166; 8,456,309; 8,457,730; 8,457,746; 8,457,747; 8,461,988; 8,463,006; 8,463,007; 8,463,349; 8,463,370; 8,463,374; 8,463,378; 8,463,386; 8,463,387; 8,464,288; 8,465,408; 8,467,877; 8,467,878; 8,473,024; 8,473,044; 8,473,306; 8,473,345; 8,475,354; 8,475,368; 8,475,371; 8,475,387; 8,475,506; 8,478,389; 8,478,394; 8,478,402; 8,478,417; 8,478,428; 8,480,554; 8,483,795; 8,483,815; 8,483,816; 8,484,081; 8,484,270; 8,485,979; 8,487,760; 8,489,185; 8,492,336; 8,494,610; 8,494,829; 8,494,857; 8,494,905; 8,496,594; 8,498,697; 8,498,699; 8,498,708; 8,500,282; 8,500,636; 8,504,150; 8,506,469; 8,509,879; 8,509,881; 8,509,885; 8,509,904; 8,512,219; 8,512,221; 8,512,240; 8,515,535; 8,515,538; 8,515,541; 8,515,549; 8,515,550; 8,517,909; 8,517,912; 8,519,705; 8,519,853; 8,520,974; 8,521,284; 8,523,779; 8,525,673; 8,525,687; 8,527,029; 8,527,035; 8,527,435; 8,529,463; 8,531,291; 8,532,756; 8,532,757; 8,533,042; 8,536,667; 8,538,108; 8,538,512; 8,538,513; 8,538,514; 8,538,523; 8,538,536; 8,538,543; 8,538,700; 8,538,705; 8,542,900; 8,542,916; 8,543,189; 8,543,199; 8,543,214; 8,543,219; 8,545,378; 8,545,416; 8,545,420; 8,545,436; 8,548,583; 8,548,594; 8,548,604; 8,548,786; 8,548,852; 8,553,956; 8,554,311; 8,554,325; 8,559,645; 8,560,034; 8,560,041; 8,560,073; 8,562,525; 8,562,526; 8,562,527; 8,562,536; 8,562,540; 8,562,548; 8,562,660; 8,562,951; 8,565,606; 8,565,864; 8,565,867; 8,565,883; 8,565,886; 8,568,231; 8,568,329; 8,571,293; 8,571,629; 8,571,642; 8,571,643; 8,571,653; 8,574,164; 8,574,279; 8,577,103; 8,577,464; 8,577,465; 8,577,466; 8,577,467; 8,577,468; 8,577,472; 8,577,478; 8,579,786; 8,579,793; 8,579,795; 8,579,834; 8,583,238; 8,583,252; 8,585,568; 8,586,019; 8,586,932; 8,587,304; 8,588,486; 8,588,552; 8,588,899; 8,588,929; 8,588,933; 8,588,941; 8,589,316; 8,591,419; 8,591,498; 8,593,141; 8,593,154; 8,594,798; 8,594,800; 8,594,950; 8,597,171; 8,597,193; 8,600,493; 8,600,502; 8,600,513; 8,600,521; 8,600,696; 8,603,790; 8,606,349; 8,606,351; 8,606,356; 8,606,360; 8,606,361; 8,606,530; 8,606,592; 8,612,005; 8,613,695; 8,613,905; 8,614,254; 8,614,873; 8,615,293; 8,615,309; 8,615,479; 8,615,664; 8,618,799; 8,620,206; 8,620,419; 8,626,264; 8,626,301; 8,628,328; 8,628,480; 8,630,699; 8,630,705; 8,630,812; 8,632,465; 8,632,750; 8,634,616; 8,634,922; 8,635,105; 8,636,640; 8,638,950; 8,641,632; 8,641,646; 8,644,754; 8,644,910; 8,644,914; 8,644,921; 8,644,945; 8,644,946; 8,644,954; 8,644,957; 8,647,278; 8,648,017; 8,649,845; 8,649,866; 8,649,871; 8,652,038; 8,652,187; 8,652,189; 8,655,428; 8,655,437; 8,655,817; 8,657,732; 8,657,756; 8,658,149; 8,660,642; 8,660,649; 8,660,666; 8,660,799; 8,664,258; 8,666,099; 8,666,467; 8,666,478; 8,666,501; 8,668,496; 8,670,603; 8,672,852; 8,675,936; 8,675,945; 8,675,983; 8,676,324; 8,676,325; 8,676,330; 8,679,009; 8,680,119; 8,680,991; 8,682,422; 8,682,441; 8,682,449; 8,682,687; 8,684,742; 8,684,900; 8,684,921; 8,684,922; 8,684,926; 8,688,209; 8,690,748; 8,693,756; 8,693,765; 8,694,087; 8,694,089; 8,694,092; 8,694,107; 8,694,118; 8,694,157; 8,696,722; 8,696,724; 8,698,639; 8,700,137; 8,700,141; 8,700,142; 8,700,163; 8,700,167; 8,700,174; 8,700,183; 8,703,114; 8,706,183; 8,706,205; 8,706,206; 8,706,207; 8,706,237; 8,706,241; 8,706,518; 8,708,903; 8,708,934; 8,711,655; 8,712,507; 8,712,512; 8,712,513; 8,712,547; 8,716,447; 8,717,430; 8,718,747; 8,718,776; 8,718,777; 8,718,779; 8,721,695; 8,724,871; 8,725,238; 8,725,243; 8,725,311; 8,725,668; 8,725,669; 8,725,796; 8,727,978; 8,728,001; 8,729,040; 8,731,650; 8,731,656; 8,731,987; 8,733,290; 8,734,356; 8,734,357; 8,734,498; 8,738,121; 8,738,126; 8,738,136; 8,738,140; 8,738,395; 8,744,562; 8,744,563; 8,747,313; 8,747,336; 8,747,382; 8,750,971; 8,750,974; 8,750,992; 8,751,008; 8,751,011; 8,753,296; 8,754,238; 8,755,854; 8,755,856; 8,755,868; 8,755,869; 8,755,871; 8,755,877; 8,755,901; 8,756,017; 8,758,274; 8,761,438; 8,761,866; 8,761,868; 8,761,869; 8,761,889; 8,762,065; 8,762,202; 8,764,651; 8,764,652; 8,764,653; 8,764,673; 8,768,022; 8,768,427; 8,768,431; 8,768,446; 8,768,447; 8,768,449; 8,768,471; 8,768,477; 8,768,718; 8,771,194; 8,774,923; 8,775,340; 8,781,193; 8,781,197; 8,781,557; 8,781,563; 8,781,595; 8,781,597; 8,781,796; 8,784,109; 8,784,322; 8,785,441; 8,786,624; 8,787,637; 8,788,030; 8,788,033; 8,788,044; 8,788,055; 8,788,057; 8,790,255; 8,790,272; 8,790,297; 8,792,972; 8,792,974; 8,792,991; 8,795,175; 8,798,717; 8,798,728; 8,798,735; 8,798,736; 8,798,773; 8,801,620; 8,805,516; 8,805,518; 8,812,126; 8,812,237; 8,812,245; 8,812,246; 8,814,923; 8,815,582; 8,821,376; 8,821,408; 8,821,559; 8,825,149; 8,825,166; 8,825,167; 8,825,428; 8,827,912; 8,827,917; 8,829,908; 8,831,705; 8,831,731; 8,831,732; 8,834,392; 8,834,546; 8,838,201; 8,838,225; 8,838,226; 8,838,227; 8,838,247; 8,843,199; 8,843,201; 8,843,210; 8,845,545; 8,849,390; 8,849,392; 8,849,407; 8,849,409; 8,849,632; 8,849,681; 8,852,073; 8,852,100; 8,852,103; 8,855,758; 8,855,773; 8,855,775; 8,858,440; 8,858,449; 8,861,819; 8,862,196; 8,862,210; 8,862,236; 8,862,581; 8,864,310; 8,864,806; 8,868,148; 8,868,163; 8,868,172; 8,868,173; 8,868,174; 8,868,175; 8,868,177; 8,868,189; 8,868,201; 8,870,737; 8,871,797; 8,872,640; 8,874,205; 8,874,218; 8,874,227; 8,874,439; 8,880,207; 8,880,576; 8,886,299; 8,886,302; 8,888,672; 8,888,673; 8,888,702; 8,888,708; 8,888,723; 8,892,207; 8,893,120; 8,898,037; 8,900,284; 8,902,070; 8,903,479; 8,903,483; 8,903,486; 8,903,494; 8,906,360; 8,907,668; 8,909,345; 8,910,638; 8,913,810; 8,914,100; 8,914,115; 8,914,119; 8,914,122; 8,915,741; 8,915,871; 8,918,162; 8,918,176; 8,918,178; 8,918,183; 8,921,320; 8,922,376; 8,922,788; 8,923,958; 8,924,235; 8,926,959; 8,929,991; 8,929,999; 8,932,218; 8,932,227; 8,932,562; 8,933,696; 8,934,685; 8,934,965; 8,934,967; 8,934,979; 8,934,986; 8,936,629; 8,936,630; 8,938,102; 8,938,289; 8,938,290; 8,938,301; 8,939,903; 8,942,777; 8,942,813; 8,942,817; 8,945,006; 8,948,834; 8,948,849; 8,948,855; 8,948,860; 8,951,189; 8,951,190; 8,951,192; 8,951,203; 8,954,139; 8,954,146; 8,954,293; 8,955,010; 8,955,974; 8,956,277; 8,956,363; 8,958,868; 8,958,870; 8,958,882; 8,961,187; 8,961,385; 8,961,386; 8,962,042; 8,962,589; 8,964,298; 8,965,492; 8,965,513; 8,965,514; 8,968,172; 8,968,176; 8,968,195; 8,968,376; 8,971,936; 8,972,004; 8,972,013; 8,974,365; 8,977,024; 8,977,110; 8,977,362; 8,980,891; 8,983,155; 8,983,591; 8,983,620; 8,983,628; 8,983,629; 8,985,119; 8,986,207; 8,989,835; 8,989,836; 8,989,863; 8,989,867; 8,989,868; 8,989,871; 8,992,230; 8,993,623; 8,996,112; 8,996,120; 8,998,828; 9,002,458; 9,002,471; 9,002,477; 9,004,687; 9,005,102; 9,005,126; 9,005,649; 9,008,367; 9,008,754; 9,008,771; 9,008,780; 9,008,970; 9,011,329; 9,014,216; 9,014,453; 9,014,804; 9,014,811; 9,014,819; 9,014,823; 9,015,057; 9,015,087; 9,020,576; 9,020,582; 9,020,585; 9,020,586; 9,020,598; 9,020,612; 9,020,789; 9,022,930; 9,022,936; 9,025,845; 9,026,194; 9,026,202; 9,026,217; 9,026,218; 9,026,372; 9,028,405; 9,028,412; 9,031,644; 9,031,653; 9,031,655; 9,031,658; 9,033,884; 9,034,055; 9,034,911; 9,034,923; 9,035,657; 9,036,844; 9,037,224; 9,037,225; 9,037,254; 9,037,256; 9,037,530; 9,042,074; 9,042,201; 9,042,952; 9,042,958; 9,042,988; 9,043,001; 9,044,188; 9,044,612; 9,050,469; 9,050,470; 9,050,471; 9,053,516; 9,053,534; 9,055,871; 9,055,974; 9,056,195; 9,058,473; 9,060,671; 9,060,683; 9,060,695; 9,060,722; 9,060,746; 9,061,132; 9,061,133; 9,061,151; 9,061,153; 9,063,183; 9,063,643; 9,064,036; 9,067,052; 9,067,054; 9,067,070; 9,069,031; 9,069,097; 9,070,492; 9,072,449; 9,072,482; 9,072,832; 9,072,870; 9,072,905; 9,074,976; 9,076,212; 9,078,564; 9,078,577; 9,078,584; 9,079,039; 9,079,940; 9,081,488; 9,081,882; 9,081,890; 9,082,169; 9,084,584; 9,084,885; 9,084,896; 9,084,900; 9,087,147; 9,089,310; 9,089,400; 9,089,683; 9,089,707; 9,089,713; 9,089,719; 9,091,785; 9,092,556; 9,092,895; 9,095,266; 9,095,268; 9,095,295; 9,095,303; 9,095,314; 9,095,618; 9,095,713; 9,100,758; 9,101,263; 9,101,276; 9,101,279; 9,101,690; 9,101,759; 9,101,766; 9,102,717; 9,107,586; 9,107,595; 9,108,041; 9,113,777; 9,113,801; 9,113,803; 9,113,830; 9,116,201; 9,116,835; 9,118,775; 9,119,533; 9,119,551; 9,119,583; 9,119,597; 9,119,598; 9,121,964; 9,125,574; 9,125,581; 9,125,788; 9,126,050; 9,131,864; 9,133,024; 9,133,709; 9,135,221; 9,135,400; 9,138,156; 9,138,175; 9,138,183; 9,138,579; 9,138,580; 9,142,145; 9,142,185; 9,144,392; 9,149,195; 9,149,197; 9,149,210; 9,149,214; 9,149,226; 9,149,255; 9,149,577; 9,149,599; 9,149,719; 9,152,757; 9,155,373; 9,155,484; 9,155,487; 9,155,521; 9,161,715; 9,162,051; 9,162,052; 9,165,472; 9,167,970; 9,167,974; 9,167,976; 9,167,977; 9,167,978; 9,167,979; 9,171,353; 9,171,366; 9,173,582; 9,173,609; 9,173,610; 9,174,045; 9,174,055; 9,174,066; 9,175,095; 9,177,379; 9,177,416; 9,179,850; 9,179,854; 9,179,855; 9,179,858; 9,179,875; 9,179,876; 9,183,351; 9,186,060; 9,186,106; 9,186,503; 9,186,510; 9,187,745; 9,192,300; 9,192,309; 9,198,563; 9,198,612; 9,198,621; 9,198,624; 9,198,637; 9,198,707; 9,198,733; 9,204,796; 9,204,835; 9,204,838; 9,204,998; 9,208,430; 9,208,557; 9,208,558; 9,211,076; 9,211,077; 9,211,212; 9,211,411; 9,211,417; 9,213,074; 9,213,076; 9,215,298; 9,215,978; 9,220,910; 9,220,917; 9,221,755; 9,226,672; 9,227,056; 9,229,080; 9,230,065; 9,230,539; 9,232,910; 9,232,984; 9,233,244; 9,233,245; 9,233,246; 9,233,258; 9,235,679; 9,235,685; 9,238,142; 9,238,150; 9,241,647; 9,241,665; 9,242,067; 9,242,092; 9,247,890; 9,247,911; 9,247,924; 9,248,003; 9,248,280; 9,248,286; 9,248,288; 9,248,290; 9,248,291; 9,248,296; 9,249,200; 9,249,234; 9,251,566; 9,254,097; 9,254,099; 9,254,383; 9,254,387; 9,256,982; 9,259,177; 9,259,482; 9,259,591; 9,261,573; 9,265,458; 9,265,660; 9,265,661; 9,265,662; 9,265,663; 9,265,931; 9,265,943; 9,265,946; 9,265,965; 9,265,974; 9,268,014; 9,268,015; 9,268,902; 9,271,651; 9,271,657; 9,271,660; 9,271,674; 9,271,679; 9,272,091; 9,272,139; 9,272,145; 9,272,153; 9,273,035; 9,275,191; 9,275,451; 9,277,871; 9,277,873; 9,278,159; 9,278,231; 9,280,784; 9,282,927; 9,282,930; 9,282,934; 9,283,279; 9,283,394; 9,284,353; 9,285,249; 9,289,143; 9,289,595; 9,289,599; 9,289,603; 9,289,609; 9,292,471; 9,292,858; 9,292,920; 9,295,838; 9,296,382; 9,302,069; 9,302,093; 9,302,103; 9,302,109; 9,302,110; 9,302,114; 9,302,116; 9,305,376; 9,307,925; 9,307,944; 9,308,372; 9,308,392; 9,309,296; 9,310,985; 9,311,335; 9,314,190; 9,314,613; 9,314,633; 9,314,635; 9,320,449; 9,320,450; 9,320,451; 9,320,900; 9,320,913; 9,320,914; 9,322,895; 9,326,705; 9,326,720; 9,326,742; 9,327,069; 9,327,070; 9,328,107; 9,329,758; 9,330,206; 9,330,523; 9,331,841; 9,332,939; 9,333,334; 9,333,347; 9,333,350; 9,336,302; 9,336,535; 9,336,611; 9,339,200; 9,339,227; 9,339,641; 9,339,654; 9,340,589; 9,345,412; 9,345,609; 9,345,886; 9,345,901; 9,348,974; 9,349,178; 9,351,640; 9,351,651; 9,352,145; 9,352,152; 9,352,156; 9,357,240; 9,357,298; 9,357,941; 9,357,949; 9,357,970; 9,358,361; 9,358,381; 9,358,392; 9,358,393; 9,358,398; 9,359,449; 9,360,472; 9,364,462; 9,364,665; 9,364,674; 9,364,679; 9,365,628; 9,367,131; 9,367,738; 9,368,018; 9,368,265; 9,370,309; 9,370,667; 9,375,145; 9,375,151; 9,375,171; 9,375,564; 9,375,571; 9,375,573; 9,377,348; 9,377,515; 9,380,976; 9,381,346; 9,381,352; 9,383,208; 9,387,320; 9,387,338; 9,390,233; 9,392,955; 9,393,406; 9,393,418; 9,394,347; 9,395,425; 9,396,533; 9,396,669; 9,398,873; 9,399,126; 9,399,133; 9,399,134; 9,399,144; 9,401,021; 9,401,033; 9,402,558; 9,402,994; 9,403,000; 9,403,001; 9,403,009; 9,403,010; 9,403,038; 9,405,366; 9,408,530; 9,409,013; 9,409,022; 9,409,028; 9,410,885; 9,411,033; 9,411,935; 9,412,076; 9,412,233; 9,414,029; 9,414,749; 9,414,763; 9,414,764; 9,414,776; 9,414,780; 9,414,907; 9,415,215; 9,415,216; 9,415,219; 9,415,222; 9,415,233; 9,418,368; 9,420,970; 9,421,258; 9,421,372; 9,421,373; 9,421,379; 9,424,761; 9,427,474; 9,427,581; 9,427,585; 9,427,598; 9,430,615; 9,432,777; 9,433,797; 9,434,692; 9,436,989; 9,438,650; 9,439,150; 9,440,063; 9,440,064; 9,440,070; 9,440,084; 9,440,089; 9,440,646; 9,442,088; 9,442,525; 9,443,141; 9,444,998; 9,445,713; 9,445,730; 9,445,739; 9,445,763; 9,446,238; 9,448,289; 9,449,147; 9,451,303; 9,451,734; 9,451,883; 9,451,886; 9,451,899; 9,452,287; 9,453,215; 9,454,646; 9,458,208; 9,459,597; 9,460,400; 9,462,733; 9,462,956; 9,462,975; 9,462,977; 9,463,327; 9,468,541; 9,468,761; 9,470,728; 9,471,978; 9,472,000; 9,474,481; 9,474,852; 9,474,903; 9,475,502; 9,480,402; 9,480,425; 9,480,812; 9,480,841; 9,480,845; 9,480,854; 9,483,117; 9,483,613; 9,486,168; 9,486,381; 9,486,389; 9,486,618; 9,486,632; 9,489,854; 9,492,084; 9,492,114; 9,492,120; 9,492,313; 9,492,656; 9,492,678; 9,495,684; 9,497,017; 9,498,134; 9,498,628; 9,498,634; 9,500,722; 9,501,829; 9,504,390; 9,504,410; 9,504,420; 9,504,788; 9,505,402; 9,505,817; 9,510,790; 9,513,398; 9,517,020; 9,517,031; 9,517,222; 9,519,981; 9,521,958; 9,522,085; 9,522,278; 9,522,282; 9,522,288; 9,526,419; 9,526,902; 9,526,906; 9,526,913; 9,526,914; 9,533,113; 9,533,144; 9,533,147; 9,533,148; 9,533,150; 9,533,151; 9,534,044; 9,538,635; 9,538,948; 9,538,951; 9,539,118; 9,541,383; 9,545,221; 9,545,222; 9,545,225; 9,545,226; 9,545,285; 9,545,510; 9,545,515; 9,549,691; 9,550,064; 9,556,149; 9,556,487; 9,557,439; 9,558,558; 9,560,458; 9,560,967; 9,560,984; 9,560,986; 9,561,380; 9,562,988; 9,563,273; 9,563,740; 9,563,950; 9,566,426; 9,567,327; 9,568,564; 9,568,635; 9,572,996; 9,577,992; 9,578,425; 9,579,035; 9,579,048; 9,579,247; 9,579,457; 9,579,506; 9,582,072; 9,582,152; 9,582,925; 9,584,928; 9,585,581; 9,585,723; 9,586,047; 9,586,053; 9,588,203; 9,588,490; 9,590,986; 9,592,003; 9,592,004; 9,592,384; 9,592,387; 9,592,389; 9,592,409; 9,596,224; 9,597,493; 9,597,494; 9,597,501; 9,597,504; 9,600,138; 9,600,778; 9,604,056; 9,604,067; 9,604,073; 9,607,023; 9,607,377; 9,609,453; 9,610,442; 9,610,456; 9,610,459; 9,612,295; 9,613,184; 9,613,186; 9,615,746; 9,615,749; 9,615,789; 9,616,166; 9,616,227; 9,618,591; 9,622,660; 9,622,672; 9,622,675; 9,622,676; 9,622,700; 9,622,702; 9,622,703; 9,623,240; 9,623,241; 9,626,756; 9,629,548; 9,629,568; 9,629,976; 9,630,004; 9,630,008; 9,630,011; 9,630,029; 9,636,019; 9,636,185; 9,640,167; 9,641,665; 9,642,552; 9,642,553; 9,642,554; 9,642,699; 9,643,015; 9,643,017; 9,643,019; 9,646,248; 9,649,030; 9,649,036; 9,649,439; 9,649,493; 9,649,494; 9,649,501; 9,651,368; 9,651,706; 9,652,626; 9,652,871; 9,655,573; 9,655,669; 9,656,069; 9,656,075; 9,656,078; 9,656,096; 9,659,186; 9,659,229; 9,662,049; 9,662,069; 9,662,083; 9,662,490; 9,662,492; 9,662,502; 9,664,856; 9,665,824; 9,665,987; 9,668,694; 9,669,185; 9,669,239; 9,672,302; 9,672,617; 9,674,621; 9,675,254; 9,675,255; 9,675,292; 9,675,794; 9,675,809; 9,681,814; 9,681,820; 9,682,232; 9,682,241; 9,684,051; 9,684,335; 9,685,600; 9,687,187; 9,687,562; 9,693,684; 9,693,724; 9,693,725; 9,693,734; 9,694,155; 9,694,178; 9,694,197; 9,697,330; 9,697,336; 9,700,256; 9,700,716; 9,700,723; 9,704,205; 9,706,910; 9,706,925; 9,706,957; 9,706,963; 9,707,372; 9,707,390; 9,707,391; 9,707,396; 9,710,788; 9,712,736; 9,713,428; 9,713,433; 9,713,444; 9,713,712; 9,715,032; 9,717,461; 9,717,904; 9,717,920; 9,724,517; 9,729,252; 9,732,039; 9,734,589; 9,734,601; 9,734,632; 9,737,230; 9,740,710; 9,740,946; 9,741,114; 9,743,197; 9,743,835; 9,744,358; 9,763,592; 9,991,026; D627,476; RE34,015; RE38,476; RE38,749; RE44,097; RE44,408; RE45,336; RE45,337; RE45,766; RE46,189; RE46,209; 20010003799; 20010009975; 20010014818; 20010020127; 20010021800; 20010029391; 20010049480; 20010051774; 20010051787; 20020000808; 20020005784; 20020006875; 20020013612; 20020013613; 20020016552; 20020017905; 20020017994; 20020024450; 20020032375; 20020033454; 20020035317; 20020035338; 20020037095; 20020042563; 20020052539; 20020055675; 20020058867; 20020059159; 20020072776; 20020072782; 20020077536; 20020082513; 20020082665; 20020085174; 20020087201; 20020091319; 20020091335; 20020091419; 20020095099; 20020097332; 20020099273; 20020099295; 20020099306; 20020099412; 20020099417; 20020099418; 20020103428; 20020103429; 20020103512; 20020107454; 20020112732; 20020117176; 20020128540; 20020128544; 20020128638; 20020138013; 20020151771; 20020151939; 20020158631; 20020173714; 20020177882; 20020182574; 20020183607; 20020183644; 20020188330; 20020193670; 20030001098; 20030004429; 20030009078; 20030009096; 20030013981; 20030018277; 20030018278; 20030023183; 20030023282; 20030028081; 20030028121; 20030028348; 20030031357; 20030032870; 20030032888; 20030032889; 20030035301; 20030036689; 20030040660; 20030045914; 20030046018; 20030055355; 20030068605; 20030070685; 20030074032; 20030081818; 20030083596; 20030083716; 20030088274; 20030093004; 20030093005; 20030093129; 20030097159; 20030097161; 20030100844; 20030105408; 20030114886; 20030120140; 20030120172; 20030125786; 20030128801; 20030130706; 20030130709; 20030135128; 20030139681; 20030144601; 20030149351; 20030149678; 20030153818; 20030158466; 20030158495; 20030158496; 20030158497; 20030158587; 20030160622; 20030163027; 20030163028; 20030167019; 20030171658; 20030171685; 20030171689; 20030176804; 20030181791; 20030181821; 20030181954; 20030181955; 20030185408; 20030187359; 20030195429; 20030195574; 20030199749; 20030204135; 20030216654; 20030225335; 20030225340; 20030229291; 20030233039; 20030233250; 20030234781; 20030236458; 20030236557; 20030236558; 20040002635; 20040006265; 20040006376; 20040010203; 20040015204; 20040015205; 20040019257; 20040019370; 20040024287; 20040030585; 20040034299; 20040039268; 20040049124; 20040049484; 20040059203; 20040059241; 20040064020; 20040064066; 20040068164; 20040068172; 20040068199; 20040072133; 20040073098; 20040073129; 20040073273; 20040077960; 20040077967; 20040078056; 20040079372; 20040082862; 20040082876; 20040088732; 20040092809; 20040096395; 20040097802; 20040101146; 20040116784; 20040116791; 20040116798; 20040116825; 20040117098; 20040122787; 20040122790; 20040127803; 20040131998; 20040133118; 20040133119; 20040133120; 20040133248; 20040133390; 20040138516; 20040138517; 20040138518; 20040138536; 20040138580; 20040138581; 20040138647; 20040138711; 20040138721; 20040140811; 20040143170; 20040144925; 20040145370; 20040151368; 20040152958; 20040152995; 20040153129; 20040158119; 20040158298; 20040158300; 20040166536; 20040167418; 20040172089; 20040172091; 20040172094; 20040181162; 20040184024; 20040186542; 20040193037; 20040193068; 20040193220; 20040195512; 20040199482; 20040204636; 20040204637; 20040204656; 20040204659; 20040210127; 20040210146; 20040210156; 20040215082; 20040220494; 20040220782; 20040225179; 20040230105; 20040243017; 20040243182; 20040254493; 20040260169; 20040260356; 20040263162; 20040267152; 20050004489; 20050007091; 20050010091; 20050010116; 20050015205; 20050018858; 20050019734; 20050020483; 20050020918; 20050021105; 20050025704; 20050027284; 20050032827; 20050033122; 20050033154; 20050033174; 20050033379; 20050038354; 20050043774; 20050049651; 20050059689; 20050059874; 20050060001; 20050060007; 20050060008; 20050060009; 20050060010; 20050065412; 20050065427; 20050075568; 20050079474; 20050079636; 20050080124; 20050080349; 20050080828; 20050085744; 20050096311; 20050096517; 20050106713; 20050107654; 20050113713; 20050118286; 20050119547; 20050119586; 20050124848; 20050124851; 20050124863; 20050131311; 20050135102; 20050136002; 20050137494; 20050137645; 20050144042; 20050148828; 20050148893; 20050148894; 20050148895; 20050149123; 20050149157; 20050153268; 20050154290; 20050154419; 20050154425; 20050154426; 20050156602; 20050159670; 20050159671; 20050165458; 20050167588; 20050171410; 20050182287; 20050182288; 20050182389; 20050182450; 20050182453; 20050182456; 20050182467; 20050182468; 20050182469; 20050187600; 20050192514; 20050192644; 20050192647; 20050197590; 20050197675; 20050197678; 20050209512; 20050209517; 20050209654; 20050209664; 20050209665; 20050209666; 20050215889; 20050216070; 20050216071; 20050222522; 20050222639; 20050228451; 20050228785; 20050240087; 20050240229; 20050240253; 20050244045; 20050245796; 20050251055; 20050251220; 20050256378; 20050256385; 20050256418; 20050267011; 20050267343; 20050267344; 20050267362; 20050267542; 20050273017; 20050277813; 20050277912; 20050283053; 20050283090; 20060004298; 20060004422; 20060009704; 20060009815; 20060014753; 20060015034; 20060015153; 20060018525; 20060020184; 20060036152; 20060036153; 20060041201; 20060047187; 20060047216; 20060047324; 20060047325; 20060051814; 20060052386; 20060052657; 20060052706; 20060058590; 20060058683; 20060058856; 20060061544; 20060064138; 20060064139; 20060064140; 20060069059; 20060069415; 20060074290; 20060074298; 20060074334; 20060074822; 20060078183; 20060079936; 20060082727; 20060084858; 20060084877; 20060087746; 20060089541; 20060089549; 20060094968; 20060094970; 20060094971; 20060094972; 20060095091; 20060095092; 20060100526; 20060100530; 20060100671; 20060102171; 20060106274; 20060106326; 20060106430; 20060106434; 20060111644; 20060116556; 20060122481; 20060129022; 20060129202; 20060129277; 20060129324; 20060135879; 20060135880; 20060136135; 20060142802; 20060149144; 20060149160; 20060149337; 20060152227; 20060153396; 20060155206; 20060155207; 20060155348; 20060155495; 20060161071; 20060161075; 20060161217; 20060161218; 20060161384; 20060167370; 20060167497; 20060167564; 20060167722; 20060170424; 20060173259; 20060173364; 20060173493; 20060173494; 20060173495; 20060173510; 20060176062; 20060178709; 20060184058; 20060184059; 20060188134; 20060189866; 20060189880; 20060189882; 20060189899; 20060191543; 20060195039; 20060195154; 20060195155; 20060200013; 20060200016; 20060200034; 20060200035; 20060200206; 20060204532; 20060206033; 20060206108; 20060206155; 20060206165; 20060206174; 20060212090; 20060212091; 20060217609; 20060217781; 20060217816; 20060224216; 20060224421; 20060225437; 20060229164; 20060233390; 20060235315; 20060235324; 20060235484; 20060235489; 20060239482; 20060241373; 20060241382; 20060241562; 20060241718; 20060247728; 20060251303; 20060252978; 20060252979; 20060258896; 20060258950; 20060259077; 20060265022; 20060276695; 20060281543; 20060281980; 20060282123; 20060287691; 20060293578; 20060293721; 20060293723; 20070000372; 20070005115; 20070005391; 20070007454; 20070008172; 20070014454; 20070015985; 20070016095; 20070016264; 20070019846; 20070021673; 20070021675; 20070021800; 20070025608; 20070027486; 20070027498; 20070027499; 20070027500; 20070027501; 20070031798; 20070032733; 20070032737; 20070032834; 20070036355; 20070036402; 20070038067; 20070038264; 20070038382; 20070043392; 20070043401; 20070049844; 20070049988; 20070050715; 20070055145; 20070060830; 20070060831; 20070060954; 20070060974; 20070060984; 20070066403; 20070066914; 20070066915; 20070066997; 20070067003; 20070067004; 20070072857; 20070078134; 20070081712; 20070083128; 20070093721; 20070093870; 20070100246; 20070100251; 20070100278; 20070100377; 20070100378; 20070100389; 20070100392; 20070100398; 20070100666; 20070112404; 20070118197; 20070127793; 20070129647; 20070129769; 20070129774; 20070135724; 20070135728; 20070138886; 20070142862; 20070142873; 20070142874; 20070149860; 20070150024; 20070150025; 20070150026; 20070150029; 20070156180; 20070156457; 20070159185; 20070161919; 20070162085; 20070162086; 20070165915; 20070167694; 20070167723; 20070167853; 20070167858; 20070167991; 20070173733; 20070173902; 20070179395; 20070179396; 20070179534; 20070179558; 20070179734; 20070184507; 20070191688; 20070191691; 20070191697; 20070191704; 20070191727; 20070197930; 20070198063; 20070203401; 20070203448; 20070208212; 20070208269; 20070209669; 20070213785; 20070213786; 20070225581; 20070225674; 20070225774; 20070225932; 20070233192; 20070233193; 20070238934; 20070239059; 20070244387; 20070244407; 20070249918; 20070249949; 20070249952; 20070250119; 20070250138; 20070255122; 20070255135; 20070255155; 20070255320; 20070255379; 20070255531; 20070259323; 20070260151; 20070265508; 20070265533; 20070273504; 20070273611; 20070276270; 20070276278; 20070276279; 20070276441; 20070276609; 20070280508; 20070282228; 20070287896; 20070291832; 20070293760; 20070299370; 20070299371; 20080001600; 20080001735; 20080004514; 20080004550; 20080004904; 20080009685; 20080009772; 20080013747; 20080015458; 20080015459; 20080021332; 20080021336; 20080021340; 20080021341; 20080021342; 20080021345; 20080027347; 20080027348; 20080027515; 20080033266; 20080033291; 20080033297; 20080033502; 20080033503; 20080033508; 20080033513; 20080036752; 20080039677; 20080039698; 20080039737; 20080039904; 20080042067; 20080045775; 20080045823; 20080045844; 20080046012; 20080046035; 20080049376; 20080051669; 20080051858; 20080058664; 20080058668; 20080058773; 20080064934; 20080065183; 20080069446; 20080071150; 20080071326; 20080074307; 20080077010; 20080077015; 20080077191; 20080081963; 20080082018; 20080086182; 20080091118; 20080091240; 20080097197; 20080097235; 20080097553; 20080097785; 20080103547; 20080103548; 20080109050; 20080119716; 20080119747; 20080119763; 20080119900; 20080123927; 20080125669; 20080125829; 20080125830; 20080125831; 20080128626; 20080132383; 20080139953; 20080140141; 20080140149; 20080140403; 20080147137; 20080154111; 20080154126; 20080154148; 20080154331; 20080154332; 20080157980; 20080161700; 20080161879; 20080161880; 20080161881; 20080161886; 20080161894; 20080162182; 20080167535; 20080167540; 20080167569; 20080167571; 20080177195; 20080177196; 20080177197; 20080183072; 20080183097; 20080188765; 20080194981; 20080195166; 20080200831; 20080208072; 20080208073; 20080208280; 20080208285; 20080214902; 20080215112; 20080219917; 20080221400; 20080221401; 20080221441; 20080221472; 20080221969; 20080228077; 20080228100; 20080228239; 20080229408; 20080230702; 20080230705; 20080234113; 20080234601; 20080235469; 20080241804; 20080242521; 20080242976; 20080243005; 20080243014; 20080243017; 20080243021; 20080247618; 20080249430; 20080249589; 20080255469; 20080255816; 20080255949; 20080257349; 20080260212; 20080262327; 20080262367; 20080262371; 20080269542; 20080269812; 20080269833; 20080269834; 20080269840; 20080269843; 20080275327; 20080275340; 20080275526; 20080279436; 20080281238; 20080281381; 20080281667; 20080286453; 20080287774; 20080287821; 20080288018; 20080294019; 20080294063; 20080298653; 20080298659; 20080304691; 20080304731; 20080306365; 20080310697; 20080311549; 20080317317; 20080319326; 20080319505; 20090005654; 20090005667; 20090005675; 20090006001; 20090009284; 20090012387; 20090018407; 20090018419; 20090018429; 20090018431; 20090018432; 20090018462; 20090022825; 20090024007; 20090024050; 20090030476; 20090030930; 20090033333; 20090036781; 20090036791; 20090036950; 20090039889; 20090043221; 20090048507; 20090048530; 20090054788; 20090054800; 20090054801; 20090054946; 20090054958; 20090058660; 20090062660; 20090062670; 20090062676; 20090062679; 20090062680; 20090062696; 20090062698; 20090069707; 20090074279; 20090076339; 20090076399; 20090076400; 20090076406; 20090076407; 20090076567; 20090078875; 20090082688; 20090082689; 20090082690; 20090082829; 20090083071; 20090088658; 20090088680; 20090093403; 20090093862; 20090094305; 20090099474; 20090099627; 20090099783; 20090105785; 20090112117; 20090112273; 20090112277; 20090112278; 20090112279; 20090112280; 20090112281; 20090112523; 20090118593; 20090118610; 20090118622; 20090118636; 20090118780; 20090118786; 20090118787; 20090119154; 20090124869; 20090124921; 20090124922; 20090124923; 20090131995; 20090132275; 20090137915; 20090137923; 20090143654; 20090148019; 20090149148; 20090149736; 20090156907; 20090156954; 20090156955; 20090156956; 20090157323; 20090157481; 20090157482; 20090157625; 20090157660; 20090157662; 20090157751; 20090157813; 20090163777; 20090163980; 20090163981; 20090163982; 20090164131; 20090164132; 20090164302; 20090164401; 20090164403; 20090164458; 20090164503; 20090164549; 20090171164; 20090171232; 20090171240; 20090171405; 20090172540; 20090177050; 20090177090; 20090177108; 20090177144; 20090179642; 20090182211; 20090187230; 20090191131; 20090192394; 20090192556; 20090198144; 20090198145; 20090204015; 20090209831; 20090209835; 20090209845; 20090210018; 20090216091; 20090216146; 20090216288; 20090220425; 20090220429; 20090221904; 20090221928; 20090221930; 20090227876; 20090227877; 20090227882; 20090227889; 20090234419; 20090240119; 20090243756; 20090246138; 20090247893; 20090247894; 20090259277; 20090261832; 20090264785; 20090264789; 20090264952; 20090264954; 20090264955; 20090264956; 20090264957; 20090264958; 20090264967; 20090267758; 20090270687; 20090270688; 20090270692; 20090270693; 20090270694; 20090270754; 20090270758; 20090270786; 20090270944; 20090271011; 20090271120; 20090271122; 20090271347; 20090275853; 20090276011; 20090276012; 20090280153; 20090281400; 20090281448; 20090281594; 20090287035; 20090287107; 20090287108; 20090287271; 20090287272; 20090287273; 20090287274; 20090287467; 20090290767; 20090290772; 20090292180; 20090292478; 20090292551; 20090292713; 20090292724; 20090297000; 20090299126; 20090299169; 20090299435; 20090304582; 20090306491; 20090306531; 20090306532; 20090306534; 20090306741; 20090311655; 20090312595; 20090312624; 20090312646; 20090312663; 20090312664; 20090312668; 20090312808; 20090312817; 20090312998; 20090316925; 20090316968; 20090316969; 20090318773; 20090318779; 20090318794; 20090319000; 20090319001; 20090319002; 20090319004; 20090322331; 20090323049; 20090326353; 20090326604; 20090326605; 20090327068; 20100003656; 20100004500; 20100004705; 20100004717; 20100004762; 20100004977; 20100010289; 20100010316; 20100010363; 20100010364; 20100010365; 20100010366; 20100010383; 20100010388; 20100010391; 20100010392; 20100010571; 20100010572; 20100010573; 20100010574; 20100010575; 20100010576; 20100010577; 20100010578; 20100010579; 20100010580; 20100010584; 20100010585; 20100010587; 20100010588; 20100010589; 20100010590; 20100010844; 20100014730; 20100014732; 20100015583; 20100016783; 20100017001; 20100021378; 20100022820; 20100023089; 20100028841; 20100030073; 20100030089; 20100030097; 20100030287; 20100036211; 20100036233; 20100036276; 20100036453; 20100041949; 20100041958; 20100041962; 20100041964; 20100042011; 20100042578; 20100043795; 20100045467; 20100049069; 20100049075; 20100049276; 20100049482; 20100056276; 20100056854; 20100056939; 20100057159; 20100057160; 20100057655; 20100063368; 20100063563; 20100068751; 20100069724; 20100069739; 20100069762; 20100069775; 20100069777; 20100069780; 20100070001; 20100076249; 20100076253; 20100076274; 20100076333; 20100076334; 20100076338; 20100076525; 20100079292; 20100080432; 20100081860; 20100081861; 20100082506; 20100087719; 20100087900; 20100090835; 20100092934; 20100094103; 20100094152; 20100094154; 20100094155; 20100098289; 20100099954; 20100099975; 20100100036; 20100100164; 20100106041; 20100106043; 20100106044; 20100106217; 20100113959; 20100114190; 20100114192; 20100114193; 20100114237; 20100114272; 20100114813; 20100121415; 20100125219; 20100125304; 20100125561; 20100130811; 20100130812; 20100130869; 20100130878; 20100131030; 20100131034; 20100132448; 20100134113; 20100135556; 20100137728; 20100137937; 20100142774; 20100143256; 20100145215; 20100145219; 20100145427; 20100145428; 20100152621; 20100160737; 20100163027; 20100163028; 20100163035; 20100165593; 20100168053; 20100168525; 20100168529; 20100168602; 20100172567; 20100174161; 20100174533; 20100179415; 20100179447; 20100185113; 20100189318; 20100191095; 20100191124; 20100191139; 20100191304; 20100191305; 20100195770; 20100197610; 20100197993; 20100198090; 20100198098; 20100198101; 20100198282; 20100198296; 20100198519; 20100204604; 20100204614; 20100204748; 20100204749; 20100204750; 20100217100; 20100217146; 20100217341; 20100217348; 20100219820; 20100222640; 20100222694; 20100222845; 20100224188; 20100231221; 20100231327; 20100234705; 20100234752; 20100234753; 20100238763; 20100241020; 20100241195; 20100241449; 20100245093; 20100248275; 20100249573; 20100249627; 20100249635; 20100249638; 20100256592; 20100258126; 20100260402; 20100261977; 20100261993; 20100262377; 20100268055; 20100268057; 20100268108; 20100268288; 20100274106; 20100274141; 20100274147; 20100274303; 20100274305; 20100274308; 20100274577; 20100274578; 20100280332; 20100280334; 20100280335; 20100280372; 20100280403; 20100280500; 20100280571; 20100280574; 20100280579; 20100286549; 20100286747; 20100292602; 20100292752; 20100293002; 20100293115; 20100298624; 20100298735; 20100303101; 20100305962; 20100305963; 20100312188; 20100312579; 20100318025; 20100318160; 20100322488; 20100322497; 20100324441; 20100331649; 20100331715; 20100331976; 20110004115; 20110004270; 20110004283; 20110004412; 20110007129; 20110009715; 20110009729; 20110009752; 20110009777; 20110009920; 20110009928; 20110015209; 20110015469; 20110015501; 20110015515; 20110015536; 20110015539; 20110021899; 20110021970; 20110022981; 20110028798; 20110028799; 20110028802; 20110028825; 20110028827; 20110028859; 20110029038; 20110029044; 20110034812; 20110034821; 20110034822; 20110034912; 20110035231; 20110038515; 20110038850; 20110040202; 20110040356; 20110040546; 20110040547; 20110040713; 20110043759; 20110046451; 20110046473; 20110046491; 20110050232; 20110054272; 20110054279; 20110054345; 20110054562; 20110054569; 20110060382; 20110066005; 20110066041; 20110066042; 20110066053; 20110074396; 20110077503; 20110077538; 20110077548; 20110077721; 20110082154; 20110082360; 20110082381; 20110082522; 20110087125; 20110087127; 20110092800; 20110092834; 20110092839; 20110092882; 20110093033; 20110098583; 20110098778; 20110105859; 20110105915; 20110105938; 20110105998; 20110106206; 20110106750; 20110110868; 20110112379; 20110112381; 20110112394; 20110112426; 20110112427; 20110112590; 20110115624; 20110118536; 20110118618; 20110118619; 20110119212; 20110125046; 20110125048; 20110125077; 20110125078; 20110125203; 20110125238; 20110129129; 20110130615; 20110130643; 20110130675; 20110137371; 20110137381; 20110144520; 20110144521; 20110150253; 20110152284; 20110152710; 20110152729; 20110152967; 20110152988; 20110160543; 20110160607; 20110160608; 20110160795; 20110160796; 20110161011; 20110162645; 20110166430; 20110166471; 20110166546; 20110172500; 20110172509; 20110172553; 20110172554; 20110172562; 20110172564; 20110172567; 20110172725; 20110172732; 20110172738; 20110172739; 20110172743; 20110172927; 20110178359; 20110178441; 20110178442; 20110178581; 20110181422; 20110182501; 20110184305; 20110184487; 20110184650; 20110190569; 20110190600; 20110190846; 20110191275; 20110191350; 20110196693; 20110201944; 20110207988; 20110208012; 20110208094; 20110208264; 20110208539; 20110213200; 20110213222; 20110217240; 20110218405; 20110218453; 20110218456; 20110218950; 20110224569; 20110224570; 20110224571; 20110224602; 20110224749; 20110229005; 20110230701; 20110230738; 20110230755; 20110230938; 20110238130; 20110238136; 20110245709; 20110245734; 20110251583; 20110251985; 20110256520; 20110257501; 20110257517; 20110257519; 20110263962; 20110263968; 20110263995; 20110264182; 20110270074; 20110270095; 20110270096; 20110270117; 20110270346; 20110270347; 20110270348; 20110270579; 20110270914; 20110275927; 20110276107; 20110276112; 20110282225; 20110282230; 20110282234; 20110288119; 20110288400; 20110288424; 20110288431; 20110293193; 20110295142; 20110295143; 20110295166; 20110295338; 20110295344; 20110295345; 20110295346; 20110295347; 20110298706; 20110301436; 20110301439; 20110301441; 20110301448; 20110301486; 20110301487; 20110301488; 20110301529; 20110306845; 20110306846; 20110307029; 20110307030; 20110307079; 20110308789; 20110311021; 20110311489; 20110313268; 20110313274; 20110313308; 20110313487; 20110313760; 20110319482; 20110319724; 20110319726; 20110319975; 20120003615; 20120004518; 20120004561; 20120004564; 20120004579; 20120004749; 20120010493; 20120010536; 20120011927; 20120016218; 20120016252; 20120016336; 20120016430; 20120016432; 20120016435; 20120021394; 20120022336; 20120022340; 20120022343; 20120022350; 20120022351; 20120022365; 20120022384; 20120022392; 20120022611; 20120022844; 20120022884; 20120029320; 20120029378; 20120029379; 20120029591; 20120029601; 20120035428; 20120035431; 20120035433; 20120035698; 20120035765; 20120036004; 20120041279; 20120041318; 20120041319; 20120041320; 20120041321; 20120041322; 20120041323; 20120041324; 20120041330; 20120041498; 20120041735; 20120041739; 20120046531; 20120046535; 20120046711; 20120046715; 20120046971; 20120052469; 20120052905; 20120053394; 20120053433; 20120053449; 20120053473; 20120053476; 20120053478; 20120053479; 20120053483; 20120053491; 20120053508; 20120053919; 20120053921; 20120059246; 20120059273; 20120059431; 20120060851; 20120065536; 20120070044; 20120071771; 20120078115; 20120078323; 20120078327; 20120080305; 20120083668; 20120083690; 20120083700; 20120083701; 20120083708; 20120088987; 20120088992; 20120089004; 20120089205; 20120092156; 20120092157; 20120095352; 20120095357; 20120100514; 20120101326; 20120101387; 20120101401; 20120101402; 20120101430; 20120101544; 20120108909; 20120108918; 20120108995; 20120108997; 20120108998; 20120108999; 20120109020; 20120116149; 20120116179; 20120116235; 20120116244; 20120116475; 20120116741; 20120123232; 20120123290; 20120125337; 20120128683; 20120130204; 20120130228; 20120130229; 20120130300; 20120130641; 20120136242; 20120136274; 20120136605; 20120143038; 20120143074; 20120143075; 20120143104; 20120143285; 20120145152; 20120149042; 20120149997; 20120150255; 20120150257; 20120150262; 20120150516; 20120150545; 20120157804; 20120157963; 20120158092; 20120159656; 20120162002; 20120163689; 20120164613; 20120165624; 20120165631; 20120165696; 20120165898; 20120165899; 20120165904; 20120172682; 20120172689; 20120172743; 20120177716; 20120179071; 20120179228; 20120184801; 20120184826; 20120185020; 20120191000; 20120191158; 20120191542; 20120195860; 20120197092; 20120197153; 20120197163; 20120197322; 20120203079; 20120203087; 20120203130; 20120203131; 20120203133; 20120203725; 20120207362; 20120209126; 20120209136; 20120209139; 20120209346; 20120212353; 20120215114; 20120215448; 20120219195; 20120219507; 20120220843; 20120220889; 20120221310; 20120226091; 20120226130; 20120226185; 20120226334; 20120232327; 20120232376; 20120232433; 20120238890; 20120242501; 20120245464; 20120245474; 20120245481; 20120245493; 20120245655; 20120249274; 20120253101; 20120253141; 20120253168; 20120253219; 20120253249; 20120253261; 20120253421; 20120253429; 20120253434; 20120253442; 20120259249; 20120262250; 20120262558; 20120263393; 20120265080; 20120265262; 20120265267; 20120265270; 20120265271; 20120268272; 20120269385; 20120271148; 20120271151; 20120271183; 20120271189; 20120271190; 20120271374; 20120271375; 20120271376; 20120271377; 20120271380; 20120277545; 20120277548; 20120277816; 20120277833; 20120283502; 20120283604; 20120288143; 20120289854; 20120289869; 20120290058; 20120296182; 20120296241; 20120296253; 20120296569; 20120302842; 20120302845; 20120302856; 20120302867; 20120302894; 20120302912; 20120303080; 20120303087; 20120310050; 20120310100; 20120310105; 20120310106; 20120310107; 20120310298; 20120316622; 20120316630; 20120316793; 20120321152; 20120321160; 20120321759; 20120323108; 20120323132; 20120330109; 20120330369; 20130006124; 20130006332; 20130009783; 20130011819; 20130012786; 20130012787; 20130012788; 20130012789; 20130012790; 20130012802; 20130012804; 20130012830; 20130013327; 20130013339; 20130013667; 20130018435; 20130018438; 20130018439; 20130018440; 20130018592; 20130018596; 20130019325; 20130023783; 20130028496; 20130030241; 20130030257; 20130031038; 20130034837; 20130035579; 20130039498; 20130041235; 20130041281; 20130046151; 20130046193; 20130046358; 20130046715; 20130053656; 20130054214; 20130054215; 20130058548; 20130060110; 20130060125; 20130060158; 20130063434; 20130063550; 20130064438; 20130066350; 20130066391; 20130066392; 20130066394; 20130066395; 20130066618; 20130069780; 20130070929; 20130072292; 20130072775; 20130072780; 20130072807; 20130072996; 20130073022; 20130076885; 20130079606; 20130079621; 20130079647; 20130079656; 20130079657; 20130080127; 20130080489; 20130085678; 20130089503; 20130090454; 20130090706; 20130091941; 20130095459; 20130096391; 20130096393; 20130096394; 20130096408; 20130096441; 20130096453; 20130096454; 20130096839; 20130096840; 20130102833; 20130102877; 20130102897; 20130102907; 20130102919; 20130104066; 20130109995; 20130109996; 20130110616; 20130113816; 20130116520; 20130116540; 20130116561; 20130116578; 20130116588; 20130116748; 20130118494; 20130120246; 20130121984; 20130123568; 20130123584; 20130123607; 20130123684; 20130127708; 20130127980; 20130130799; 20130131438; 20130131461; 20130131537; 20130131746; 20130131753; 20130131755; 20130132029; 20130137717; 20130137936; 20130137938; 20130138002; 20130138176; 20130138177; 20130141103; 20130144106; 20130144107; 20130144108; 20130144183; 20130144192; 20130144353; 20130144537; 20130150650; 20130150651; 20130150659; 20130150702; 20130150921; 20130151163; 20130158883; 20130159041; 20130165766; 20130165804; 20130165812; 20130165846; 20130165996; 20130167360; 20130172663; 20130172686; 20130172691; 20130172716; 20130172763; 20130172767; 20130172772; 20130172774; 20130178693; 20130178718; 20130178733; 20130178913; 20130182860; 20130184218; 20130184516; 20130184552; 20130184558; 20130184597; 20130184603; 20130184639; 20130184728; 20130184781; 20130184786; 20130184792; 20130184997; 20130185144; 20130185145; 20130188830; 20130188854; 20130189663; 20130190577; 20130190642; 20130197321; 20130197322; 20130197328; 20130197339; 20130197401; 20130197944; 20130203019; 20130204085; 20130204122; 20130204144; 20130204150; 20130211183; 20130211224; 20130211238; 20130211276; 20130211291; 20130211728; 20130217982; 20130218043; 20130218053; 20130218232; 20130218233; 20130218819; 20130221961; 20130223709; 20130225940; 20130225953; 20130225992; 20130226261; 20130226408; 20130226464; 20130231574; 20130231580; 20130231709; 20130231716; 20130231721; 20130231947; 20130234823; 20130235550; 20130237541; 20130237874; 20130238049; 20130238050; 20130238053; 20130238063; 20130242262; 20130243287; 20130244323; 20130245416; 20130245422; 20130245424; 20130245464; 20130245466; 20130245485; 20130245486; 20130245711; 20130245712; 20130245886; 20130251641; 20130253363; 20130253612; 20130255586; 20130261490; 20130261506; 20130261703; 20130266163; 20130267760; 20130267866; 20130267928; 20130274562; 20130274580; 20130274586; 20130274625; 20130275159; 20130281758; 20130281759; 20130281811; 20130281879; 20130281890; 20130282075; 20130282339; 20130289360; 20130289364; 20130289385; 20130289386; 20130289401; 20130289413; 20130289417; 20130289424; 20130289433; 20130289653; 20130289669; 20130293844; 20130295016; 20130296406; 20130296637; 20130300573; 20130303828; 20130303934; 20130304153; 20130304159; 20130304472; 20130308099; 20130309278; 20130310422; 20130310660; 20130310909; 20130314243; 20130317380; 20130317382; 20130317384; 20130317474; 20130317568; 20130317580; 20130318546; 20130324880; 20130330428; 20130338449; 20130338450; 20130338459; 20130338518; 20130338526; 20130338738; 20130338803; 20130339043; 20130344465; 20130345522; 20130345523; 20140000630; 20140003696; 20140005518; 20140005743; 20140005744; 20140005988; 20140012061; 20140012110; 20140012133; 20140012153; 20140015852; 20140018649; 20140018792; 20140019165; 20140023999; 20140025133; 20140025396; 20140025397; 20140029830; 20140031703; 20140031889; 20140031903; 20140032512; 20140038147; 20140039279; 20140039290; 20140039336; 20140039571; 20140039577; 20140039578; 20140039975; 20140046203; 20140046208; 20140046407; 20140051044; 20140051960; 20140051961; 20140052213; 20140055284; 20140056815; 20140057232; 20140058189; 20140058218; 20140058219; 20140058241; 20140058289; 20140058292; 20140058528; 20140062472; 20140063054; 20140063055; 20140066739; 20140066763; 20140066796; 20140067740; 20140070958; 20140072127; 20140072130; 20140073863; 20140073864; 20140073866; 20140073870; 20140073875; 20140073876; 20140073877; 20140073878; 20140073898; 20140073948; 20140073949; 20140073951; 20140073953; 20140073954; 20140073955; 20140073956; 20140073960; 20140073961; 20140073963; 20140073965; 20140073966; 20140073967; 20140073968; 20140073974; 20140073975; 20140074060; 20140074179; 20140074180; 20140074188; 20140077612; 20140077946; 20140081071; 20140081114; 20140081115; 20140081347; 20140081353; 20140088341; 20140088377; 20140094710; 20140094719; 20140094720; 20140098981; 20140100467; 20140100633; 20140101084; 20140104059; 20140105436; 20140107397; 20140107398; 20140107401; 20140107464; 20140107519; 20140107521; 20140107525; 20140107728; 20140107935; 20140111335; 20140113367; 20140114165; 20140114205; 20140114207; 20140114242; 20140114889; 20140119621; 20140121446; 20140121476; 20140121554; 20140121565; 20140122379; 20140128762; 20140128763; 20140128764; 20140128938; 20140133720; 20140133722; 20140135642; 20140135680; 20140135873; 20140135879; 20140135886; 20140136585; 20140140567; 20140142448; 20140142653; 20140142654; 20140142669; 20140143064; 20140148479; 20140148657; 20140148693; 20140148716; 20140148723; 20140148726; 20140148872; 20140151563; 20140152673; 20140154647; 20140154650; 20140155430; 20140155706; 20140155714; 20140155730; 20140155740; 20140155770; 20140155772; 20140155952; 20140156000; 20140159862; 20140161352; 20140163328; 20140163330; 20140163331; 20140163332; 20140163333; 20140163335; 20140163336; 20140163337; 20140163368; 20140163385; 20140163409; 20140163425; 20140163627; 20140163643; 20140163893; 20140163897; 20140171749; 20140171757; 20140171819; 20140171820; 20140174277; 20140175261; 20140176944; 20140179980; 20140180088; 20140180092; 20140180093; 20140180094; 20140180095; 20140180096; 20140180097; 20140180099; 20140180100; 20140180112; 20140180113; 20140180145; 20140180153; 20140180160; 20140180161; 20140180176; 20140180177; 20140180194; 20140180358; 20140180597; 20140184550; 20140187901; 20140187994; 20140188006; 20140188770; 20140193336; 20140194702; 20140194720; 20140194726; 20140194758; 20140194759; 20140194768; 20140194769; 20140194780; 20140194793; 20140200414; 20140200432; 20140200623; 20140203797; 20140206981; 20140207224; 20140207432; 20140211593; 20140213842; 20140213843; 20140213844; 20140213937; 20140213961; 20140214135; 20140214330; 20140214335; 20140221726; 20140221866; 20140222113; 20140222406; 20140226131; 20140226888; 20140228620; 20140228649; 20140228651; 20140228653; 20140228702; 20140232516; 20140235826; 20140235965; 20140236039; 20140236077; 20140236272; 20140236492; 20140237073; 20140243608; 20140243613; 20140243614; 20140243621; 20140243628; 20140243647; 20140243652; 20140243663; 20140243694; 20140243714; 20140243926; 20140243934; 20140245191; 20140247970; 20140249360; 20140249396; 20140249429; 20140249445; 20140249447; 20140249454; 20140249608; 20140249791; 20140249792; 20140257047; 20140257073; 20140257118; 20140257128; 20140257132; 20140257147; 20140257430; 20140257437; 20140257438; 20140266696; 20140266787; 20140270438; 20140271483; 20140275716; 20140275741; 20140275807; 20140275847; 20140275851; 20140275886; 20140275889; 20140275891; 20140275944; 20140276012; 20140276013; 20140276014; 20140276090; 20140276123; 20140276130; 20140276181; 20140276183; 20140276185; 20140276187; 20140276194; 20140276549; 20140276702; 20140276944; 20140277255; 20140277256; 20140277282; 20140277286; 20140277582; 20140279341; 20140279746; 20140288381; 20140288614; 20140288620; 20140288953; 20140289172; 20140296646; 20140296655; 20140296724; 20140296733; 20140296750; 20140297397; 20140300532; 20140303424; 20140303425; 20140303452; 20140303453; 20140303454; 20140303486; 20140303508; 20140303511; 20140304773; 20140309484; 20140309614; 20140309881; 20140309943; 20140313303; 20140315169; 20140316191; 20140316192; 20140316217; 20140316221; 20140316230; 20140316235; 20140316243; 20140316248; 20140316278; 20140323849; 20140323899; 20140323900; 20140323924; 20140323946; 20140324118; 20140324138; 20140328487; 20140330093; 20140330102; 20140330157; 20140330159; 20140330268; 20140330334; 20140330335; 20140330336; 20140330337; 20140330345; 20140330357; 20140330394; 20140330404; 20140330580; 20140335489; 20140336473; 20140336489; 20140336514; 20140336547; 20140336730; 20140340084; 20140343397; 20140343399; 20140343408; 20140343463; 20140343882; 20140347265; 20140347491; 20140348183; 20140348412; 20140350353; 20140350369; 20140350380; 20140350431; 20140350436; 20140350634; 20140350636; 20140350864; 20140354278; 20140355859; 20140357507; 20140357932; 20140357935; 20140357936; 20140357962; 20140358024; 20140358025; 20140358067; 20140358193; 20140358199; 20140364721; 20140364746; 20140369537; 20140370479; 20140371515; 20140371516; 20140371544; 20140371573; 20140371599; 20140371611; 20140371984; 20140378809; 20140378810; 20140378815; 20140378830; 20140378851; 20140378941; 20140379620; 20150002815; 20150003698; 20150003699; 20150005592; 20150005594; 20150005640; 20150005644; 20150005646; 20150005660; 20150005680; 20150005839; 20150005840; 20150005841; 20150006186; 20150008916; 20150010223; 20150011866; 20150011877; 20150011907; 20150012054; 20150012057; 20150012111; 20150012466; 20150016618; 20150017115; 20150018665; 20150018699; 20150018702; 20150018705; 20150018706; 20150018758; 20150018893; 20150018905; 20150019241; 20150019266; 20150024356; 20150025351; 20150025408; 20150025410; 20150025421; 20150025422; 20150025610; 20150025917; 20150026446; 20150029087; 20150030220; 20150032017; 20150032044; 20150032178; 20150033245; 20150033258; 20150033259; 20150033262; 20150033266; 20150033363; 20150035959; 20150038804; 20150038812; 20150038822; 20150038869; 20150039066; 20150039110; 20150042477; 20150044138; 20150045606; 20150045607; 20150045686; 20150051655; 20150051656; 20150051657; 20150051658; 20150051659; 20150051663; 20150051668; 20150057512; 20150057715; 20150065803; 20150065831; 20150065838; 20150065839; 20150065845; 20150066124; 20150068069; 20150069846; 20150071907; 20150072394; 20150073141; 20150073237; 20150073249; 20150073294; 20150073306; 20150073505; 20150073722; 20150080327; 20150080671; 20150080674; 20150080695; 20150080703; 20150080746; 20150080753; 20150080985; 20150081226; 20150081299; 20150087931; 20150088015; 20150088024; 20150088093; 20150088120; 20150088224; 20150088228; 20150088478; 20150091730; 20150091791; 20150092949; 20150093729; 20150094962; 20150096564; 20150099941; 20150099946; 20150099959; 20150099962; 20150103360; 20150105631; 20150105641; 20150105701; 20150105837; 20150105844; 20150112222; 20150112403; 20150112409; 20150112899; 20150119652; 20150119658; 20150119689; 20150119698; 20150119743; 20150119745; 20150119746; 20150119794; 20150119898; 20150119956; 20150120007; 20150123653; 20150124220; 20150126821; 20150126845; 20150126848; 20150126873; 20150133716; 20150133811; 20150133812; 20150133830; 20150134031; 20150134264; 20150137817; 20150137988; 20150140528; 20150141529; 20150141773; 20150141789; 20150141794; 20150142082; 20150145519; 20150145676; 20150148617; 20150148619; 20150148700; 20150148878; 20150150122; 20150150473; 20150150475; 20150150530; 20150150753; 20150151142; 20150153477; 20150154721; 20150154764; 20150154889; 20150157235; 20150157266; 20150157271; 20150157859; 20150161326; 20150161348; 20150161738; 20150164349; 20150164362; 20150164375; 20150164404; 20150164431; 20150165226; 20150165239; 20150167459; 20150174362; 20150174398; 20150174403; 20150174405; 20150174406; 20150174407; 20150174418; 20150177413; 20150178631; 20150178978; 20150181840; 20150182417; 20150182753; 20150182756; 20150186923; 20150190062; 20150190070; 20150190077; 20150190085; 20150190094; 20150190636; 20150190637; 20150192532; 20150192776; 20150196213; 20150196246; 20150196249; 20150196800; 20150199010; 20150199121; 20150200046; 20150201849; 20150201879; 20150202330; 20150202428; 20150202447; 20150203822; 20150206051; 20150206174; 20150208940; 20150208975; 20150208978; 20150208982; 20150208994; 20150212168; 20150213012; 20150213019; 20150213020; 20150213191; 20150215412; 20150216436; 20150216439; 20150216468; 20150216469; 20150216762; 20150217082; 20150219729; 20150219732; 20150220486; 20150220830; 20150223721; 20150223731; 20150223743; 20150223905; 20150226813; 20150227702; 20150227793; 20150230719; 20150230744; 20150230750; 20150231330; 20150231395; 20150231397; 20150231405; 20150231408; 20150234477; 20150235088; 20150235370; 20150235441; 20150235447; 20150238104; 20150238106; 20150238112; 20150238137; 20150238693; 20150238761; 20150238765; 20150241705; 20150241916; 20150241959; 20150242575; 20150242608; 20150242943; 20150243100; 20150243105; 20150243106; 20150245781; 20150245800; 20150246238; 20150247723; 20150247921; 20150247975; 20150247976; 20150248167; 20150248169; 20150248170; 20150248470; 20150248615; 20150248764; 20150248765; 20150248787; 20150248788; 20150248789; 20150248791; 20150248792; 20150248793; 20150250393; 20150250401; 20150250415; 20150251016; 20150253391; 20150253410; 20150254413; 20150257645; 20150257648; 20150257649; 20150257673; 20150257674; 20150257700; 20150257712; 20150262016; 20150264492; 20150265164; 20150265207; 20150265583; 20150265830; 20150265836; 20150269825; 20150272448; 20150272461; 20150272465; 20150272496; 20150272510; 20150272652; 20150273211; 20150273223; 20150282705; 20150282730; 20150282749; 20150282755; 20150282760; 20150283019; 20150283265; 20150283379; 20150283393; 20150287223; 20150289217; 20150289779; 20150289813; 20150289929; 20150290419; 20150290420; 20150290453; 20150290454; 20150293004; 20150294067; 20150294074; 20150294085; 20150294086; 20150294445; 20150296288; 20150297106; 20150297108; 20150297109; 20150297139; 20150297141; 20150297444; 20150297719; 20150297889; 20150297893; 20150301218; 20150304048; 20150304101; 20150305685; 20150305686; 20150305689; 20150305799; 20150305800; 20150305801; 20150306057; 20150306340; 20150306390; 20150306391; 20150306392; 20150309563; 20150309582; 20150310862; 20150313496; 20150313498; 20150313535; 20150313539; 20150313540; 20150313949; 20150313971; 20150315554; 20150317447; 20150317796; 20150320591; 20150321000; 20150324544; 20150324545; 20150324692; 20150325151; 20150327813; 20150327837; 20150328330; 20150328455; 20150331929; 20150332015; 20150335281; 20150335288; 20150335292; 20150335294; 20150335295; 20150335303; 20150335876; 20150335877; 20150338915; 20150339363; 20150339459; 20150342472; 20150342478; 20150342493; 20150343215; 20150343222; 20150343242; 20150351655; 20150351690; 20150351701; 20150352362; 20150352363; 20150359431; 20150359441; 20150359450; 20150359452; 20150359467; 20150359486; 20150359492; 20150360026; 20150360030; 20150360039; 20150363941; 20150366482; 20150366497; 20150366503; 20150366504; 20150366516; 20150366518; 20150366656; 20150366659; 20150369864; 20150370320; 20150370325; 20150374250; 20150374285; 20150374292; 20150374300; 20150374973; 20150374983; 20150374986; 20150374987; 20150374993; 20150375006; 20150379230; 20150379370; 20150379878; 20150380009; 20160000348; 20160000354; 20160000383; 20160001065; 20160001096; 20160001098; 20160002523; 20160004298; 20160004396; 20160004821; 20160004957; 20160005235; 20160005320; 20160007899; 20160007904; 20160007915; 20160007918; 20160007945; 20160008489; 20160008568; 20160008598; 20160008600; 20160008620; 20160008632; 20160012011; 20160012583; 20160012749; 20160015281; 20160015289; 20160015307; 20160015673; 20160016014; 20160019434; 20160019693; 20160022141; 20160022156; 20160022164; 20160022165; 20160022167; 20160022168; 20160022206; 20160022207; 20160022981; 20160023016; 20160026913; 20160027178; 20160027293; 20160027342; 20160027423; 20160029896; 20160029917; 20160029918; 20160029946; 20160029950; 20160029958; 20160029959; 20160029965; 20160029998; 20160030666; 20160030702; 20160030749; 20160030750; 20160030834; 20160031479; 20160035093; 20160038037; 20160038038; 20160038042; 20160038043; 20160038049; 20160038069; 20160038091; 20160038559; 20160038770; 20160040514; 20160044841; 20160045128; 20160045150; 20160045162; 20160045731; 20160045756; 20160048659; 20160048948; 20160048965; 20160051161; 20160051162; 20160051187; 20160051195; 20160051793; 20160051812; 20160051818; 20160055236; 20160055304; 20160055415; 20160055842; 20160058301; 20160058304; 20160058322; 20160058354; 20160058359; 20160058366; 20160058376; 20160058392; 20160058673; 20160060926; 20160062459; 20160063207; 20160063883; 20160065724; 20160065840; 20160066788; 20160066789; 20160066828; 20160066838; 20160067485; 20160067492; 20160067494; 20160067496; 20160067526; 20160070436; 20160073886; 20160073916; 20160073947; 20160073991; 20160074657; 20160074660; 20160074661; 20160077547; 20160078780; 20160081577; 20160081610; 20160081613; 20160081616; 20160081625; 20160081793; 20160082180; 20160082319; 20160084925; 20160085302; 20160086622; 20160087603; 20160089031; 20160091448; 20160095546; 20160095838; 20160096025; 20160097824; 20160100769; 20160101260; 20160102500; 20160103487; 20160103963; 20160104006; 20160106331; 20160106344; 20160106513; 20160106950; 20160106997; 20160107309; 20160107653; 20160109851; 20160109959; 20160110517; 20160110866; 20160110867; 20160112022; 20160112684; 20160113517; 20160113528; 20160113539; 20160113545; 20160113567; 20160113569; 20160113587; 20160113726; 20160114165; 20160116472; 20160116553; 20160117815; 20160117816; 20160117819; 20160119726; 20160120048; 20160120428; 20160120432; 20160120433; 20160120434; 20160120436; 20160120437; 20160120457; 20160120464; 20160120480; 20160121074; 20160121114; 20160121116; 20160125228; 20160125572; 20160128589; 20160128596; 20160128597; 20160128632; 20160128661; 20160128864; 20160129249; 20160131723; 20160132654; 20160133015; 20160135691; 20160135727; 20160135748; 20160135754; 20160136423; 20160136427; 20160136429; 20160136430; 20160136443; 20160139215; 20160140306; 20160140313; 20160140707; 20160140834; 20160140975; 20160143540; 20160143541; 20160143554; 20160143560; 20160143574; 20160143582; 20160143594; 20160144175; 20160144186; 20160147964; 20160148077; 20160148371; 20160148372; 20160148400; 20160148531; 20160150988; 20160151014; 20160151018; 20160151628; 20160152233; 20160155005; 20160157742; 20160157773; 20160157777; 20160157828; 20160158553; 20160158554; 20160162652; 20160164813; 20160165852; 20160165853; 20160166169; 20160166197; 20160166199; 20160166205; 20160166207; 20160166208; 20160166219; 20160167672; 20160168137; 20160170996; 20160170998; 20160171514; 20160174099; 20160174862; 20160174863; 20160174867; 20160174907; 20160175557; 20160175607; 20160176053; 20160178392; 20160180042; 20160180054; 20160180055; 20160183812; 20160183828; 20160183861; 20160183881; 20160184029; 20160184596; 20160184599; 20160187524; 20160191517; 20160192841; 20160192842; 20160192847; 20160192879; 20160193499; 20160196185; 20160196393; 20160196635; 20160196758; 20160198950; 20160198963; 20160198966; 20160198968; 20160198973; 20160199241; 20160199577; 20160199656; 20160199662; 20160202755; 20160203597; 20160203726; 20160204937; 20160205450; 20160205489; 20160206236; 20160206241; 20160206380; 20160206581; 20160206671; 20160206871; 20160206877; 20160206880; 20160210872; 20160213261; 20160213276; 20160213314; 20160213317; 20160213947; 20160216760; 20160217586; 20160217595; 20160219345; 20160220133; 20160220134; 20160220136; 20160220163; 20160220166; 20160220439; 20160220821; 20160220836; 20160220837; 20160220850; 20160222073; 20160223622; 20160223627; 20160223703; 20160224757; 20160224803; 20160228019; 20160228028; 20160228029; 20160228059; 20160228064; 20160228204; 20160228640; 20160228702; 20160228705; 20160231401; 20160232330; 20160232625; 20160232667; 20160232811; 20160235323; 20160235324; 20160235341; 20160235351; 20160235352; 20160235359; 20160235980; 20160235983; 20160238673; 20160239084; 20160239966; 20160239968; 20160240212; 20160240765; 20160242645; 20160242659; 20160242665; 20160242669; 20160242670; 20160242690; 20160242699; 20160243362; 20160243381; 20160245670; 20160245766; 20160245952; 20160246939; 20160247064; 20160248434; 20160248994; 20160249826; 20160249841; 20160249846; 20160249857; 20160249864; 20160250355; 20160250465; 20160250473; 20160256063; 20160256086; 20160256105; 20160256108; 20160256109; 20160256112; 20160256118; 20160256130; 20160256690; 20160256691; 20160256693; 20160257957; 20160259085; 20160259905; 20160260216; 20160261962; 20160262623; 20160262664; 20160262680; 20160262685; 20160262695; 20160262703; 20160263318; 20160263376; 20160263380; 20160263393; 20160267809; 20160270656; 20160270723; 20160274660; 20160275536; 20160278651; 20160278653; 20160278662; 20160278672; 20160278687; 20160278697; 20160278713; 20160278736; 20160278870; 20160279021; 20160279022; 20160279023; 20160279024; 20160279025; 20160279267; 20160279410; 20160279417; 20160279435; 20160282113; 20160282941; 20160284082; 20160287117; 20160287118; 20160287120; 20160287142; 20160287157; 20160287162; 20160287166; 20160287169; 20160287308; 20160287334; 20160287436; 20160287869; 20160287871; 20160287889; 20160287895; 20160296157; 20160296287; 20160296746; 20160298449; 20160299568; 20160300252; 20160300352; 20160302683; 20160302704; 20160302709; 20160302711; 20160302720; 20160302737; 20160303322; 20160303396; 20160303397; 20160303402; 20160306844; 20160306942; 20160310031; 20160310070; 20160310071; 20160313408; 20160313417; 20160313418; 20160313798; 20160317056; 20160317060; 20160317077; 20160317383; 20160317824; 20160320210; 20160321742; 20160324445; 20160324457; 20160324465; 20160324478; 20160324580; 20160324677; 20160324942; 20160325111; 20160331264; 20160331307; 20160331952; 20160331970; 20160331974; 20160331982; 20160334475; 20160334534; 20160334866; 20160338608; 20160338634; 20160338644; 20160338798; 20160338825; 20160339237; 20160339238; 20160339239; 20160339242; 20160339243; 20160339300; 20160341684; 20160342241; 20160342762; 20160345856; 20160345895; 20160345901; 20160345911; 20160346530; 20160346542; 20160351069; 20160354003; 20160354027; 20160356911; 20160357003; 20160357256; 20160360100; 20160360965; 20160360970; 20160361021; 20160361027; 20160361041; 20160361532; 20160361534; 20160361540; 20160361546; 20160363483; 20160364859; 20160364860; 20160364861; 20160366462; 20160367138; 20160367186; 20160367195; 20160367198; 20160367204; 20160367209; 20160367808; 20160367812; 20160371387; 20160371455; 20160371721; 20160374581; 20160374616; 20160374618; 20160374990; 20160375245; 20160375259; 20160378608; 20160378965; 20170000324; 20170000325; 20170000326; 20170000329; 20170000330; 20170000331; 20170000332; 20170000333; 20170000334; 20170000335; 20170000337; 20170000340; 20170000341; 20170000342; 20170000343; 20170000345; 20170000404; 20170000422; 20170000454; 20170000683; 20170001016; 20170001032; 20170006931; 20170007111; 20170007115; 20170007116; 20170007122; 20170007123; 20170007165; 20170007173; 20170007182; 20170007450; 20170007799; 20170007820; 20170007828; 20170007843; 20170010469; 20170010470; 20170013562; 20170014037; 20170014080; 20170014083; 20170014625; 20170014630; 20170017083; 20170020434; 20170020447; 20170020454; 20170020627; 20170021158; 20170021161; 20170024886; 20170027467; 20170027517; 20170027521; 20170027539; 20170027651; 20170027812; 20170028563; 20170031440; 20170031441; 20170032098; 20170032221; 20170032524; 20170032527; 20170032544; 20170034638; 20170035309; 20170035317; 20170035344; 20170035392; 20170036024; 20170039591; 20170039706; 20170041699; 20170042430; 20170042444; 20170042469; 20170042474; 20170042475; 20170042476; 20170042485; 20170042713; 20170042827; 20170043160; 20170043166; 20170043167; 20170043178; 20170045601; 20170046052; 20170046971; 20170050046; 20170052170; 20170053082; 20170053088; 20170053092; 20170053461; 20170053513; 20170053665; 20170055839; 20170055898; 20170055900; 20170055913; 20170056363; 20170056467; 20170056642; 20170056655; 20170056663; 20170060298; 20170061034; 20170061589; 20170061760; 20170065199; 20170065218; 20170065229; 20170065349; 20170065379; 20170065638; 20170065816; 20170066806; 20170067323; 20170069306; 20170071495; 20170071521; 20170071523; 20170071529; 20170071532; 20170071537; 20170071546; 20170071551; 20170071552; 20170076452; 20170079538; 20170079543; 20170079573; 20170079588; 20170079589; 20170079596; 20170080050; 20170080234; 20170080256; 20170080320; 20170084175; 20170084187; 20170085547; 20170085855; 20170086672; 20170086695; 20170086727; 20170086729; 20170086763; 20170087302; 20170087330; 20170087354; 20170087355; 20170087356; 20170087364; 20170087367; 20170090475; 20170091418; 20170091567; 20170094385; 20170095157; 20170095174; 20170095199; 20170095670; 20170095676; 20170095721; 20170099479; 20170099713; 20170100051; 20170100540; 20170100591; 20170103440; 20170105647; 20170106193; 20170107575; 20170108926; 20170112379; 20170112403; 20170112427; 20170112446; 20170112577; 20170112671; 20170112947; 20170113042; 20170113046; 20170113056; 20170113057; 20170117866; 20170119270; 20170119271; 20170119994; 20170120041; 20170120043; 20170120052; 20170120054; 20170120066; 20170127727; 20170127946; 20170128006; 20170128015; 20170128032; 20170131293; 20170132816; 20170133576; 20170133577; 20170135594; 20170135597; 20170135604; 20170135626; 20170135629; 20170135631; 20170135633; 20170135640; 20170136238; 20170136240; 20170136264; 20170136265; 20170138132; 20170140124; 20170143231; 20170143249; 20170143255; 20170143257; 20170143259; 20170143266; 20170143267; 20170143268; 20170143273; 20170143280; 20170143282; 20170143442; 20170143550; 20170143960; 20170143963; 20170143966; 20170143986; 20170146386; 20170146387; 20170146390; 20170146391; 20170146615; 20170146801; 20170147578; 20170147754; 20170148213; 20170148240; 20170148340; 20170148592; 20170149945; 20170150896; 20170150916; 20170150921; 20170150925; 20170151433; 20170151435; 20170151436; 20170154167; 20170156593; 20170156606; 20170156622; 20170156655; 20170156662; 20170156674; 20170157343; 20170157402; 20170157410; 20170160360; 20170162072; 20170164861; 20170164862; 20170164876; 20170164878; 20170164893; 20170164894; 20170164895; 20170164901; 20170165020; 20170165481; 20170165496; 20170168121; 20170168566; 20170168568; 20170169714; 20170171441; 20170172414; 20170172446; 20170172499; 20170172501; 20170172520; 20170172527; 20170173262; 20170173326; 20170173391; 20170177023; 20170178001; 20170178340; 20170180558; 20170181252; 20170181693; 20170182176; 20170182285; 20170182312; 20170185149; 20170185714; 20170185741; 20170188862; 20170188865; 20170188866; 20170188868; 20170188869; 20170188870; 20170188872; 20170188876; 20170188905; 20170188916; 20170188922; 20170188932; 20170188933; 20170188947; 20170188992; 20170189685; 20170189686; 20170189687; 20170189688; 20170189689; 20170189691; 20170189700; 20170189707; 20170190765; 20170193161; 20170193831; 20170196497; 20170196501; 20170196503; 20170196519; 20170197080; 20170197081; 20170197086; 20170198017; 20170198349; 20170199251; 20170202474; 20170202475; 20170202476; 20170202518; 20170202621; 20170202633; 20170203154; 20170205259; 20170206654; 20170206691; 20170206913; 20170209043; 20170209044; 20170209053; 20170209062; 20170209083; 20170209094; 20170209225; 20170209389; 20170209737; 20170212188; 20170213339; 20170214786; 20170216595; 20170221206; 20170224990; 20170224994; 20170231560; 20170239486; 20170239489; EP1304073A2; EP1304073A3; WO2000025668A1; and WO2001087153A1;
This application is a Continuation of U.S. patent application Ser. No. 16/388,845, filed Apr. 19, 2019, now U.S. Pat. No. 11,364,361, issued Jun. 21, 2022, which is a non-provisional of, and claims benefit of priority from U.S. Provisional Patent Application No. 62/660,839, filed Apr. 20, 2018, which is expressly incorporated herein by reference in its entirety. This Application incorporates by reference the entirety of U.S. Provisional Patent Application No. 62/612,565, filed Dec. 31, 2017, and U.S. patent application Ser. Nos. 16/237,497, 16/237,471, and 16/237,483, filed Dec. 31, 2018.
| Number | Date | Country | |
|---|---|---|---|
| 62660839 | Apr 2018 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | 16388845 | Apr 2019 | US |
| Child | 17844714 | US |
