This application is the U.S. national phase of International Application No. PCT/BR2016/000090 filed Sep. 12, 2016 which designated the U.S., the entire contents of which are hereby incorporated by reference.
None.
The technology herein relates generally to noninvasive systems and methods for diagnostic and intervention that utilize face recognition associated with other physiologic measures to identify the emotions and state-of-mind of a human subject and to suggest a treatment against discomfort or anxiety, which may enhance mental focus or induce a feeling of well-being. More particularly, the technology herein relates to an environment and system for providing a consensual transcranial stimulation to a subject and an associated user interface, as well as providing enhanced physical and mental performance, such as relaxation and concentration, to perform daily activities.
Nowadays, society is anxious and impatient. People are always in a hurry, running from place to place and always being late for appointments. As never before, people use antidepressants and have sleeping problems. On the other hand, there are more and more people concerned about health and wellbeing, searching for alternatives to lower stress and anxiety such as, for example, music applications which may promote concentration and focus.
For example, people can experience substantial anxiety and stress in the work environment. Instead of focusing on their work, people may be distracted by sadness, depression or other emotional states. Employees suffering from high stress-induced emotional states tend to have lower engagement, are less productive and have higher absenteeism levels than those not working under too much pressure. Employers and employees alike have an interest in maintaining focus and productivity in the workplace.
Some people become anxious when riding in a vehicle such as a car, a train or an airplane. Anxiety can be unpleasant, turning a short trip into a long one. It is very humane to relieve another subject's fear and anxiety. Mammals are very skilled at ascertaining and affecting the emotional stage of other mammals, but so far in the past, machines have been much less effective.
However, with the advent of brain stimulation, neurological diseases such as major depression have been effectively treated. The successful development of new interventions for neurological diseases of the brain using cranial stimulation technologies additionally have brought the opportunity to improve cognitive performance and reduce stress.
The brain continuously receives and translates sensory information from multiple sensory sources including, for example, visual auditory, olfactory, and tactile sources. Through processing, movement, and awareness training, subjects have been able to recover and enhance sensory perception, discrimination, and memory, demonstrating a range of untapped capabilities.
Stimulation technologies that affect electric fields and electrochemical signaling in neurons can modulate the pattern of neural activity and cause altered behavior, cognitive states, perception, motor output, and more.
Some devices combine brain and audiovisual stimulation, whereby the brain is stimulated by sound signals entering the ears and/or by visual stimulation using eye glasses with LED diodes or patterns on a screen in front of the eyes of a treated subject. The brain can also be stimulated using sources of magnetic fields near or in contact with the scalp of the head, while fields are changed according to a program that controls their alteration. Devices producing visual stimulation using bright flashing lights along with adjusted sound frequencies provided to subject's ear are also known. All such devices primarily focus on providing stimulation to the brain by using frequencies from external sources.
Although such prior methods and apparatus have been proposed to treat neurological diseases or to improve comfort, relaxation and awareness, these prior systems, methods and apparatus do not provide any diagnostic of the emotional state-of-mind of an individual and do not provide or suggest any treatment for it. For example, at least one transcranial electrical stimulation requires extensive input data concerning the subject in order to function.
Ideally, such systems and methods should improve the convenience of usage and should employ effective strategies for diagnosing the emotional state-of-mind of the subject as well as provide treatments to improve cognitive performance, enhance mental focus attention and/or promote a feeling of well-being and relaxation. Also, there is a need for systems and methods which could be combined with other noninvasive physiological measurements so as to better recognize the particular state-of-mind of a subject.
The following detailed description of exemplary non-limiting illustrative embodiments is to be read in conjunction with the drawings of which:
The example non-limiting technology herein minimizes or eliminates inconvenience by providing a mechanism and environment for real-time monitoring the emotions and state-of-mind of a human and providing treatments for improving cognitive performance and feeling of well-being via in situ brain stimulation.
The non-limiting example embodiments disclosed herein generally relate to a system and method for noninvasive intervention and diagnostics that uses face recognition associated with other physiologic measures to identify a state of mind of a subject, as well as to provide a possible suggested treatment to alleviate that subject's discomfort or anxiety. An integrated system of hardware processing, memory, user interfacing and stimulation devices, camera and/or other body sensors are used to provide a consensual transcranial stimulation to a subject in a manner capable of quickly producing an enhanced physical feeling or improved mental performance (for example, but not limited to, enhanced relaxation and concentration).
For example, a digital camera is used to record and register the complex movements of a subject's face and through the use of conventional mathematical analysis and pattern matching the images are evaluated to obtain a diagnostic indicative of an emotional status of the subject, such as, the following non-limiting examples:
In one non-limiting exemplary implementation, a system of components is provided that is capable of capturing and recording images of a human subject seated in a vehicle and includes a camera, body sensors and a hardware processor (e.g., a programmable computer) that utilizes one or more conventional mathematician algorithms to analyze the emotional state of the seated subject from the captured images. The resulting analysis may be used to initiate operations by other hardware components for providing a transcranial stimulation to the subject to enhance that subject's mental performance and physical well-being. The components of the system may be independently exchanging information as needed or integrated into a single device.
Accordingly, the technology described herein may be embodied as a method, a system, as embedded firmware in a hardware processor or conventional computer, as a whole integrated product or as a set of individual components that work together.
At least one non-limiting example method employing the system of
In addition, system 200 includes a transcranial stimulation device 207 that receives appropriate output control signals from processor device 200 which enable it to provide/produce a brain stimulation effect to the subject. Exemplary transcranial stimulation techniques which are contemplated for use with an appropriate transcranial stimulation device 207 may include, but are not limited to, direct current transcranial stimulation (tDCS) devices, repetitive transcranial magnetic stimulation (rTMS) devices and transcranial pulsed ultrasound (TPU) devices, among others. Further details regarding the mechanism of action and operation of each may be found, for example, from various reference materials including the following each of which is incorporated herein by reference:
For example, transcranial direct current stimulation (tDCS) is a form of neurostimulation which uses constant, low current delivered to the brain area of interest via electrodes on the scalp. Transcranial direct current stimulation works by sending constant, low direct current through the electrodes. Hence, transcranial direct current stimulation (tDCS), is a non-invasive, painless brain stimulation treatment that uses direct electrical currents to stimulate specific parts of the brain. A constant, low intensity current is passed through two electrodes placed over the head which modulates neuronal activity. There are two types of stimulation with tDCS: anodal and cathodal stimulation. Anodal stimulation acts to excite neuronal activity while cathodal stimulation inhibits or reduces neuronal activity. Although tDCS is still an experimental form of brain stimulation, it potentially has several advantages over other brain stimulation techniques. It is cheap, non-invasive, painless and safe. It is also easy to administer and the equipment is easily portable. The most common side effect of tDCS is a slight itching or tingling on the scalp.
In one non-limiting embodiment, electrodes are placed in the region of interest, the current induces intracerebral current flow. This current flow then either increases or decreases the neuronal excitability in the specific area being stimulated based on which type of stimulation is being used. This change of neuronal excitability leads to alteration of brain function, which can be used in various therapies as well as to provide more information about the functioning of the human brain. Transcranial direct current stimulation is a relatively simple technique requiring only a few parts. These include a tDCS device having at least two electrodes and a power source that delivers constant current via the electrodes. The tDCS device has at least one anodal, positively charged electrode and a cathodal, negative electrode. A tDCS device controller, such as for example controller 200, may include appropriate control software used to control the tDCS device to provide differing stimulation types. Preferably, the electrodes are packaged in a way that provides ease of use and good connectivity. For example, they may be provided with adhesives that allow the electrodes to be stuck to the subject's head in specified locations in order to render efficacious therapy.
In biological systems, such as the human head, a current is usually created by the flow of ions, which may be positively or negatively charged—positive ions will flow towards the cathode; negative ions will flow toward the anode. The tDCS device controller may control the current as well as the duration of stimulation. In the example system shown in
Another example technique utilizes repetitive transcranial magnetic stimulation (rTMS). Transcranial magnetic stimulation (TMS) is a magnetic method used to stimulate specific regions of the brain. TMS is a non-invasive method of brain stimulation that relies on electromagnetic induction using an insulated coil placed over the scalp, focused on an area of the brain thought to play a role in mood regulation. The coil generates brief magnetic pulses, which pass easily and painlessly through the skull and into the brain. The pulses generated are of the same type and strength as those generated by magnetic resonance imaging (MRI) machines. When these pulses are administered in rapid succession, it is referred to as “repetitive TMS” or “rTMS”, which can produce longer lasting changes in brain activity. rTMS has been shown to be a safe and well-tolerated procedure that can be an effective treatment for patients with depression.
During a repetitive TMS procedure, a magnetic field generator, or “coil”, is placed near the head of the subject receiving the treatment and repetitive pulsed currents are applied to the coil. The coil produces small electric currents in the region of the brain just under the coil via electromagnetic induction. Conventionally, the coil is connected to a pulse generator, or other stimulator, that delivers pulsed electric current to the coil. In the example system shown in
Another example technique utilizes transcranial pulsed ultrasound (TPU) which employs low intensity, low frequency ultrasound (LILFU) as a method to stimulate the brain. The low-power ultrasound is able to stimulate high neuron activity which allows for the manipulation of the brain waves through an external source such as ultrasonic transducers. TPU is a noninvasive procedure that does not require the use or implantation of electrodes. In the example system shown in
It is contemplated that for the exemplary use environment of
The analysis performed by processor system 201 is used to select the appropriate type, mode or operation of intracranial stimulation to result in inducing a change in the mental state of the subject to bring about a feeling of well-being or a feeling of improved mental focus for such purposes as, for example, to assist the subject in relaxing, reading or focusing their attention for specific mental tasks. Such selection can extend to selecting between modes of stimulation (e.g., duration, frequency, repetition rate, waveform, etc.) and/or different types of stimulating (e.g., direct electrode stimulation, magnetic, etc.)
Referring to
The flowchart of
Phase A—Scanning of Facial Expressions to Realize Physiological Measurements
In phase A of the example process, a camera 106 is installed near to an office or passenger seat and which is capable of acquiring images of the facial expressions (including microexpressions) of the subject and recording these images or providing the images to processor system 201. For example, a video monitor 106 may be installed in front of a seat 103 to enable real-time capture of images of the seated subject.
Also it is contemplated that physiological measuring devices 104 are installed near to the seat 103 which are capable of measuring the physiological information of the subject such as for example, but not limited to, body temperature, respiratory rate, heart rate, pulse oximetry, body posture, etc.
It is also contemplated that an application server or other computer processor system having facial expression analysis software and which is capable of storing acquired image signals from the camera or from any physiological measuring devices or sensors that may be used.
It is contemplated that software installed in the processor system 201 or application server (herein called “User's interface SW”), will execute user interface 205 commands providing a touch display of the video monitor or other input device for the subject to interact with the system.
Accordingly, the subject may voluntarily start the process for scanning or capturing an image of his/her face and for capturing other physiological measurements. The signals are then acquired by the digital camera and biological sensors and sent to the video monitor as well as being digitized and stored in the application server.
If after starting, the processor system 201 cannot obtain identification and recognition of a human face or signal from biological sensors then. User's Interface SW will send a real-time image the seat 103 which will appear in the user interface display device 205 with no additional message for indicating a diagnostic of the emotional status.
If processor system 201 obtains an identification and recognition of a human face and signals from biological sensors then, User's interface SW will send a real-time image of the subject and provide a display of the image along with a message such as; “Please Wait. The system is performing a diagnostic of your emotional State,” and then will start the process of Phase B: Diagnostic of the Emotional. State.
Phase B—Diagnostic of the Emotional State
In Phase B of the exemplary process, it is assumed that software/firmware program instructions (herein called “Diagnostic SW”) are installed in the memory of processor system 201 which will execute the program and perform an analysis of captured images of the complex movements (including microexpressions) of the subject's face and evaluate any other sensor obtained physiological information to produce a diagnostic of the subject's emotional state.
In the example
User's interface SW starts the Diagnostic SW for an analysis of facial expression associated with physiological measurements defining a diagnosis of emotional state.
Diagnostic SW promotes the execution of several algorithms to perform a mathematical analysis. These algorithms perform a diagnosis of emotional state and the results are sent to and stored by processor system 201 or a remote application server.
Some example types of emotional states which may be indicated as a result of the analysis may include by way of non-limiting example:
Phase C—Results of the Analysis of Emotional State and Consensual Process
After obtaining an analysis result, the Users interface SW performs classification in order to fit the result into one of the groups described below:
If the resulting analysis is in Group 1, User's interface SW sends a real-time image of the subject to display 205 with a message indicating: “You find yourself in the emotional state: (example: Neutral/Happy)”.
User's interface SW sends a real-time image of the subject to display 205 with a message indicating: “Would you like to have brain stimulation to enhance your physical and mental performance: (example: such as improving concentration focus or to read a book? Or to perform other mental work?)”.
If the result analysis is in Group 2: User's interface SW sends a real-time image of the subject to display 205 with a message indicating: “You find yourself in the following emotional state: (example: Confused/Sad/Smiling/Attentive/Depressive/Unquiet)”.
User's interface SW may also send a real-time image of the subject to display 205 along with a message indicating: “Would you like to have brain stimulation for producing enhanced physical and mental performance: (example: such as for relaxation or to have a feeling of well-being)?”.
If both the questions are defined within Group 1 or Group 2: the consensual answer of the subject “Yes” or “No” will start or not start the Phase D processes.
Phase D—Transcranial Stimulation
In Phase D, it is assumed that a device (herein called “a transcranial stimulation device”) is installed on or within to a seat near to the head of the subject and is capable of performing some type of transcranial stimulation to the seated subject that results in providing that subject with an enhanced physical sense of well-being and mental performance, such as relaxation or enhanced mental focus and concentration.
The provision of transcranial stimulation is consensual, so if the subject does not agree by responding via the user interface 205 then the process is terminated and the User's interface SW sends a real-time image of the subject to appear in the display device 205 with an additional message indicating: “Example: Operation completed”.
If the subject does agree to receive a transcranial stimulation, then the User's interface SW sends a real-time image of the subject to appear in the display device 205 with a message indicating: “Please put/position the transcranial stimulation device near your head”.
Once the transcranial stimulation device is positioned at or near the subject's head, the system performs transcranial stimulation following the orientations defined by the results of the Diagnostic SW.
Once the process is finalized, the User's interface SW sends a real-time image of the subject to the display device 205 with a message indicating: “Example: Operation completed, Please take off the brain device”.
While the invention has been described in connection with what is presently considered to be the most practical and preferred embodiments, it is to be understood that the invention is not to be limited to the disclosed embodiments, but on the contrary, is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.
Filing Document | Filing Date | Country | Kind |
---|---|---|---|
PCT/BR2016/000090 | 9/12/2016 | WO | 00 |
Publishing Document | Publishing Date | Country | Kind |
---|---|---|---|
WO2018/045438 | 3/15/2018 | WO | A |
Number | Name | Date | Kind |
---|---|---|---|
20090306741 | Hogle et al. | Dec 2009 | A1 |
20100134302 | Ahn et al. | Jun 2010 | A1 |
20120283502 | Mishelevich et al. | Nov 2012 | A1 |
20140330350 | Kokos | Nov 2014 | A1 |
20150174418 | Tyler et al. | Jun 2015 | A1 |
20160220163 | Yamada et al. | Aug 2016 | A1 |
20160256691 | Cecchi et al. | Sep 2016 | A1 |
20170291037 | Tamiya | Oct 2017 | A1 |
20170323485 | Samec | Nov 2017 | A1 |
Number | Date | Country |
---|---|---|
2013192582 | Dec 2013 | WO |
Entry |
---|
International Search Report for PCT/BR2016/000090 dated May 19, 2017, 5 pages. |
Written Opinion of the ISA for PCT/BR2016/000090 dated May 19, 2017, 7 pages. |
“Transcranial direct-current stimulation”, Wikipedia, last edited on Jan. 15, 2021, https://en.wikipedia.org/wiki/Transcranial_direct-current_stimulation (14 pages). |
Brunoni et al., “Transcranial Direct Current Stimulation in Neuropsychiatric Disorders: Clinical Principles and Management”, pp. 293-298 (Springer 2016) (7 pages). |
“Transcranial magnetic stimulation”, Wikipedia, lasted edited on Jan. 20, 2021, https://en.wikipedia.org/wiki/Transcranial_magnetic_stimulation (17 pages). |
Nitsche et al., “Transcranial direct current stimulation: State of the art 2008”, pp. 203-223, vol. 1, No. 3 (Brain Stimulation 2008) (18 pages). |
Thync, Neurostimulation For Everyone (Thync Global, Inc., 2021), www.thync.com (6 pages). |
“Transcranial pulsed ultrasound”, Wikipedia, last edited on Dec. 22, 2020, https://en.wikipedia.org/wiki/Transcranial_pulsed_ultrasound (3 pages). |
Black et al., “Recognizing Facial Expressions in Image Sequences Using Local Parameterized Models of Image Motion”, pp. 23-48, No. 25, No. 1 (International Journal of Computer Vision 1997) (26 pages). |
Cohn et al., “Feature-Point Tracking by Optical Flow Discriminates Subtle Differences in Facial Expression”, pp. 396-401 (Proc. Jnt'l Conf. Automatic Face and Gesture Recognition 1998) (6 pages). |
Schut et al., “Biometrics for Emotion Detection (BED1): Exploring the combination of Speech and ECG”, Workshop an Bio-inspired Human-machine interfaces and healthcare applications—B interfaces (Spain. 2010) (8 pages). |
Monajati et al., “Emotions States Recognition Based on Physiological Parameters by Employing of Fuzzy-Adaptive Resonance Theory”, pp. 166-175, vol. 2 (International Journal of Intelligence Science, 2012) (12 pages). |
Sharma et al., “Emotion estimation of physiological signals by using low power embedded system”, Conference on Advances in Communication and Control Systems 2013 (CAC2S 2013) (4 pages). |
Fitzgerald et al, Repetitive Transcranial Stimulation Treatment For Depressive Disorders: A Practical Guide (Springer 2013) (textbook; no copy submitted). |
Holtzheimer et al, A Clinical Guide to Transcranial Magnetic Stimulation (Oxford University Press 2014) (textbook; no copy submitted). |
Reti, Brain Stimulation: Methodologies and Interventions (Wiley-Blackwell 2015) (textbook; no copy submitted). |
M. Pantic et al, Expert System for Automatic Analysis of Facial Expressions, vol. 18, No. 11, pp. 881-905, (Image and Vision Computing 2000) (abstract). |
Edwards et al, “Face Recognition Using Active Appearance Model”, vol. 2, pp. 581-695 (Proc. European Conference on Computer Vision 1998) (abstract). |
G.J. Edwards, T.F. Cootes, C.J. Taylor, “Active Appearance Models”, vol. 2, pp. 484-498 (Proc. European Conf. Computer Vision 1998) (abstract). |
Application Note 276: “Emotional State Recognition via Physiological Measurement and processing.”, www.biopac.com (2014) (abstract). |
Mikuckas et al, “Emotion Recognition in Human Computer interaction systems”, vol. 20, No. 10, Elektronika Ir Elektrotechnika, ISSN 1392-1215 (2014) (abstract). |
Pantic et al, “Automatic Analysis of Facial Expressions: The State of the Art”, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 22, No. 12 (Dec. 2000). |
Number | Date | Country | |
---|---|---|---|
20190231243 A1 | Aug 2019 | US |