Patent Grant
11756671
The present invention relates to cognitive behavioral therapies for overcoming obsessive compulsive disorders through the guided exposure to anxiety triggers administered using a smartphone app.
Cognitive behavioral therapy (CBT) is a form of psychotherapy based on both cognitive and behavioral principles designed to modify a patient's irrational thinking and behavior. CBT is used to treat behavioral conditions that cannot be controlled through rational thought, but rather result from earlier environmental conditioning. Such treatable conditions include adult anxiety disorders such as obsessive compulsive disorder (OCD).
The basis for CBT treatments is the controlled exposure of the patient to the situation or object that causes the anxiety. The therapist encourages the patient directly to confront the feared situation or object. By exposing the patient to the anxiety trigger, the prior environmental conditioning can be undone, and the patient's undesired subconscious adverse response to the situation or object can be unlearned. Conventionally, the exposure therapy of a CBT program is administered in face-to-face sessions between the therapist and the patient in which the patient performs an exposure assignment. If the exposure assignment is too difficult, the therapist must recognize the patient's inability to proceed with the treatment and suggest an easier assignment.
The success of the CBT program depends on the ability of the therapist to assess the patient's engagement in the exposure therapy and to encourage the patient to complete each successively more difficult assignment. However, the cost of a human therapist to accompany the patient in every step of the exposure treatment is prohibitively expensive in many cases. CBT programs can be more cost effectively administered through an interactive computer interface instead of requiring a human therapist to be physically present. However, even having a human therapist being remotely present for the exposure treatment is expensive.
A system is sought that can replace the human therapist in a CBT program, but yet that can monitor the patient's progress, provide encouragement to the patient, and suggest easier or more difficult exposure assignments at the appropriate times.
A method for administering an exposure treatment of a cognitive behavioral therapy (CBT) is performed using a mobile app and a server application. The CBT involves the controlled exposure of a patient to an object that causes the patient to experience anxiety. A user state is detected of the patient while the patient is currently undergoing a first step of the CBT. The user state is based on the patient's behavior and physiological condition during the first step of the CBT as detected by sensors of a smartphone used by the patient. A situational state is detected of the patient's surroundings while the patient is undergoing the first step of the CBT. The situational state is detected by sensors of the smartphone used by the patient. The mobile app controls the detection of the user state and the situational state. The mobile app determines whether the patient has performed the first step in a manner that achieves progress in the CBT. Whether the patient has achieved progress in the CBT is determined based on the stress level and the struggle level of the patient during the first step. The stress level of the patient during the first step is determined based on touch interaction data and motion data detected by sensors on the smartphone during the first step.
A verbal user prompt is generated based on the detected user state and the detected situational state during the first step. The verbal user prompt is output by a loudspeaker of the smartphone. A next step of the CBT is configured based on the detected user state and the detected situational state during the first step. The next step of the CBT is also configured using machine learning based on the past task completions by the patient and other users so as to minimize how many steps are required for the patient to complete the CBT.
The content and character of the verbal user prompt is generated using machine learning based on past task completions by the patient and other users in order to make it more likely that the patient will complete the next step of the CBT. The machine learning generates the content of the verbal user prompt so as to include a level of reassurance most likely to motivate the patient to proceed with the next step of the cognitive behavioral therapy. The character of the verbal user prompt is defined by parameters such as the tone of the voice, the pitch of the voice, and the cadence of the verbal user prompt. The machine learning that generates the verbal user prompt is performed by a deep neural network.
In another embodiment, a system for administering an exercise task of an exposure treatment includes a smartphone, a central server and a database storage system. Data regarding past therapy steps by the patient and other users is stored in the database storage system.
Instructions of a mobile application are stored in the device memory of the smartphone. Instructions of a server application are stored in the server memory of the central server. The instructions of the mobile application when executed cause the smartphone to detect a user state and a situational state of a patient currently undergoing a first step of a CBT, determine whether the patient has achieved progress in performing the first step, and output a user prompt based on the user state and situational state detected during the first step. The user state is based on the patient's behavior and physiological condition during the first step as detected by sensors of the smartphone. The situational state of the patient's surroundings while the patient is undergoing the first step is also detected by the sensors of the smartphone.
The instructions of the server application when executed cause the central server to generate the user prompt using machine learning based on past therapy steps by the patient and other users of the mobile application so as to have a content and character adapted to influence the patient to complete the next step of the CBT. The instructions of the server application also configure the next step of the CBT based on the user state and situational state detected during the first step.
The mobile application determines whether the patient has achieved progress in the CBT based on the stress level of the patient during the first step. The stress level of the patient during the first step is determined based on touch interaction data and motion data detected by the sensors of the smartphone during the first step. The touch interaction data is sensed by a touchscreen of the smartphone, and the motion data is sensed by an accelerometer of the smartphone.
Further details and embodiments and methods and techniques are described in the detailed description below. This summary does not purport to define the invention. The invention is defined by the claims.
The accompanying drawings, where like numerals indicate like components, illustrate embodiments of the invention.
Reference will now be made in detail to some embodiments of the invention, examples of which are illustrated in the accompanying drawings.
The cognitive behavioral therapy cannot begin until patient 5 installs app 3 on his or her smartphone 1.
In step 203, app 3 queries user 5 for this personal information using a series of visual prompts provided to user 5 on the touchscreen of smartphone 1. User 5 responds by making appropriate selections on the touchscreen of the smartphone and entering information text. In step 204, app 3 sends the collected information to the central server 2 via wireless cellular telephone communication. As seen in the flowchart of
The remaining steps of the novel method of
In step 209 of
Examples of physical quantities and occurrences detected in order to determine “user state” include: the hand of user 5 shaking as detected by the accelerometer of smartphone 1, the detected amount of time user 5 spends performing a prompted act, an audio response uttered by user 5 as detected by the smartphone microphone and detected by voice recognition performed by smartphone 1. User 5 may, for example, be detected to have stated, “okay, I have done it,” or “I can't do this.”
Examples of physical quantities and occurrences are detected and sensed to make a determination as to the “situational state” include detected proximity to objects, scene mapping, detected intensity of light, GPS coordinates, relative detected movement from a previously determined position of the smartphone, detection of other individuals speaking or making noises, and detection of background noises and sounds.
In decision step 215, if app 3 determines that user 5 has made progress in performing the assigned act of an exercise task, then collected data is stored in step 216. This information is temporarily stored on smartphone 1. In step 217, app 3 uses the task model to generate feedback to provide to user 5 and/or to determine how to proceed in prompting user 5 to continue performing the assigned act of the exercise task. If, for example, user 5 is struggling to perform the assigned act that user 5 has been prompted to perform, and user 5 has not completed the act, the feedback may determine another audio message to be output to user 5 via the smartphone speaker. Such a message may, for example, be “you can do it.” The feedback determines the content of the message as well as the voice, tone, pitch and cadence. In this way, the conversation between user 5 and smartphone app 3 can be modified and tailored for the particular user so that app 3 is more comforting, reassuring and soft. Alternatively, app 3 can be more firm, strong and confident in its communication with user 5. App 3 performs steps 212-217 of the method repeatedly as user 5 carries out the various discrete assigned acts of the exercise task. In the example of the exercise task “touching toilet flush handle”, user 5 is prompted in step 213 by audio message to use the user's right hand to take a photograph of user 5 touching the toilet flush handle with the user's left hand. For example, app 3 causes a verbal user prompt in the form of an audio message “snap the picture—you can do it” to be output from the loudspeaker of smartphone 1. App 3 places smartphone 1 in the camera mode so that the pressing of the camera shutter icon by user 5 causes the image on the screen to be captured as digital image data.
Returning to decision step 215, if app 3 determines that user 5 has not made progress in carrying out the assigned act of the exercise task, then in step 218 app 3 stores the latest task completion information, user state information, and situational state information. This information is temporarily stored on smartphone 1. In step 219, if app 3 determines that the exercise task has not been completed, then in step 222, app 3 sends the collected task experience data to the central server system 2. One way that app 3 can determine that the task has not been completed is if the user is prompted in step 213 to take a photograph of the exercise completion act, and app 3 detects no such image having been taken. Then user 5 may be provided with an audio prompt to verbally indicate that user 5 did not complete the assigned act of the exercise task. Speech recognition functionality of smartphone 1 (either performed entirely locally on the smartphone, or performed partially on central server system 2) is used for this speech recognition purpose. In step 219, if app 3 determines that the exercise completion act has been completed, then app 3 determines in step 220 whether the detected user state has a particular desired characteristic. In the case of the assigned act being a physically intensive exercise, the detected user state might be the physical fatigue of user 5 and whether the exercise completion act took more than a predetermined amount of time to complete. In the case of the exercise task being “touching the toilet flush handle”, the detected user state might be a detected stress level of user 5. In step 220, app 3 determines whether the detected user stress level in carrying out the task was below a predetermined threshold of user stress. If the determined user stress is still above the maximum desired level, then app 3 prompts user 5 to perform the same exercise task again until the user's detected stress level drops below the desired threshold. Once app 3 determines in step 220 that the user state has the particular desired characteristics, in step 221 app 3 sets the current assigned act to be the next exercise task in the task hierarchy. For example, the user state would have the desired characteristics in step 220 if user 5 performed the “touching the toilet flush handle” task, and the user's detected stress level in doing so was below the maximum desired stress level. Then app 3 would proceed in step 221 to the next task of the task hierarchy. In step 222, app 3 sends the collected task experience data 10 to central server system 2.
In step 112 of
In step 303, the data from the sensor signals is used to calculate “user state” parameters of user 5 during the task, such as stress level, struggle level, heart rate, breathing rate, and whether user 5 is seeking reassurance or is following rituals. Heart rate and breathing rate can be determined using accelerometer motion data. The user's breathing movement and beating heart are sensed by the accelerometer. App 3 uses the touch interaction data and the motion data to determine the current stress level of user 5. The user's struggle level with the current task is determined based on the motion data, the heart rate, the breathing rate and the audio input from the microphone, as well as the amount of time user 5 has spent performing the assigned task. For example, the audio input could be the voice recognition of the user's statement, “I can't do this.”
In step 304, the data from the sensor signals is used to calculate “situational state” parameters of user 5 during the task. For the exemplary exercise task of touching a toilet flush handle, these “situational state” parameters include: distance of user 5 to toilet, distance of user's hand to toilet flush handle, angle between user's face and toilet flush handle, number of objects between user 5 and toilet, number of people in room, light intensity in the room, and whether user 5 washed his or her hands. For example, the audio input is used to determine the number of people in the room by detecting other individuals speaking or making noises. The video camera input is used to determine the distance of the user's hand to the toilet flush handle.
In step 305, app 3 determines the task completion state parameters based on the other parameters and on the data from the sensor signals. Examples of task completion state parameters are the time the user's hand was touching the toilet flush handle, the time elapsed after touching the handle before user 5 washed his or her hands, and the percentage of task steps completed.
In step 306, app 3 generates a real-time verbal user prompt based on the user's current stress level and struggle level, on the intensity of the current step of the exposure treatment task, and on the user's progress achieved in the current step. The flowchart of
In decision step 307, app 3 determines the progress user 5 has achieved in performing the current step of the exposure treatment task. In this example, app 3 determines whether the time user 5 spent touching the toilet flush handle exceeded a minimum time threshold, for example, one second in the first task step. App 3 determines the user's progress in performing the task based on the video camera input and the other sensor signals. If user 5 achieved the minimum time threshold, the decision flow proceeds to decision step 308; otherwise the decision flow proceeds to decision step 309.
In decision step 308, app 3 determines whether user 5 abstained from washing his or her hands after touching the toilet handle for a minimum time threshold, for example, thirty second in the first task step. If user 5 achieved the minimum time threshold, the decision flow proceeds to step 310; otherwise the decision flow proceeds to decision step 311. If user 5 refrained from washing his hands for the minimum time threshold associated with the first task step, then the first task step is completed in step 310, and a prompt is generated in step 306. After user 5 completes the first task step in step 310, the user is given an audio prompt such as, “Well done, you have completed the task. You are now ready to take on a more complicated version of the task.”
Depending on the user stress level and struggle level and the intensity level of the first task step, app 3 generates a user prompt that instructs user 5 to proceed to a subsequent task step that has a greater intensity level. The intensity of the subsequent task step is selected to achieve the fastest completion of the exposure therapy attainable by user 5 based on both input from machine learning on the knowledge base 317 and mobile app intelligence analyzing the user and situational states.
When app 3 detects the user state parameters, the situational state parameters, and the task completion state parameters in steps 303-305, those parameters are transmitted to database storage system 6 and stored in knowledge base 317. In addition, knowledge base 317 includes parameters collected from cognitive behavioral therapies undertaken by other users. In step 318, machine learning is performed on knowledge base 317 in order to identify the characteristics of the subsequent task step that best match the capabilities of user 5 to complete the exposure therapy in the shortest time. In one embodiment, the machine learning is performed using a deep neural network. For example, the machine learning in step 318 might determine that user 5 is able to skip an intensity level so as to complete the overall exposure therapy in a shorter time. In step 319, machine learning and mobile app intelligence configure the characteristics of the subsequent task step. Examples of task configuration characteristics include task intensity, minimum time for the user's hand to touch the toilet flush handle, minimum time for the user not to wash hands after touching toilet handle, struggle level threshold to decrease intensity of next task step, struggle level threshold to stop task, maximum permitted stress level to consider task completed, and total number of task repetition steps.
The user prompt generated in step 306 after the task step is completed in step 310 includes both positive feedback regarding the current task step as well as encouragement and reassurance regarding the next task step, which is described to user 5 based on the task configuration performed in step 319.
Returning to the decision step 308, if user 5 is unable to refrain for the minimum time threshold from washing his or her hands, the decision flow proceeds to step 311, where app 3 determines whether user 5 immediately washed his or her hands after touching the toilet handle. If at least some time elapsed after touching the toilet handle and before washing hands, the decision flow proceeds to step 312; otherwise the decision flow proceeds to decision step 309. If user 5 waited some time between touching the handle and washing hands, then in step 312 the configuration of the next task step is maintained with the current configuration. A prompt is generated in step 306 encouraging user 5 to attempt the assigned task again with the current configuration. The tone of the prompt voice, the pitch of the prompt voice, and the cadence of the prompt message are determined using machine learning on the knowledge base 317 and mobile app intelligence analyzing the user state parameters and situational state parameters.
If in decision step 311, no time elapsed after touching the toilet handle and washing hands, the decision flow proceeds to step 309. The decision flow also proceeds to step 309 from step 307 if user 5 did not touch the toilet flush handle for at least the minimum time threshold. In decision step 309, app 3 determines whether the struggle level of user 5 during the first task step did not exceed the maximum struggle level threshold for decreasing the task intensity of the next task step. If the struggle level of user 5 during the first task step is less than (within) the maximum allowed struggle level threshold for decreasing the task intensity, then the decision flow proceeds to step 312, and the configuration of the next task step is maintained with the current configuration and intensity. If, however, the struggle level of user 5 during the first task step exceeds the maximum allowed struggle level threshold for decreasing the task intensity, then the decision flow proceeds to step 313.
In decision step 313, app 3 determines whether the struggle level of user 5 during the first task step both exceeds the maximum struggle level threshold for decreasing the task intensity and is less than the maximum struggle level threshold for stopping the exercise task. If the struggle level of user 5 during the first task step is between the struggle level threshold for decreasing the task intensity and the struggle level threshold for stopping the exercise, then the decision flow proceeds to step 314; otherwise the task is stopped in step 315.
In decision step 314, app 3 determines whether the task intensity level of the current task step is set at the lowest task intensity level. If the task intensity level of the current task step is already set at the lowest task intensity level, then the task is stopped in step 315. If, however, the task intensity level of the current task step is not set at the lowest task intensity level, the decision flow proceeds to step 316, in which the configuration of the next task step is modified to run with a lower task intensity level.
In addition, a prompt is generated in step 306 after both step 315 and step 316. If user 5 has stopped trying to completed the task in step 315, then the voice, tone, pitch, cadence and content of the audio prompt will be composed to motivate the user not to give up on the exposure therapy altogether. The audio prompt would be the most comforting, reassuring and soft. An example of the prompt content is, “Let's pause for now and continue another time. You showed improvement and are closer to completing the task.” If user 5 will be prompted to repeat the task at a lower intensity level in step 316, an example of the prompt content would be, “Continue, focus on the task and don't look away. Feel the discomfort, but keep going for five more seconds.” The tone of the prompt after step 316 would be somewhat firmer and confident than the tone of the prompt after step 315.
Machine Learning: As described above, the central server 2 receives many different task experience data sets from numerous smartphones used by many different users including smartphone 1, and performs machine learning using that data to optimize parameter values of a task model. In one example, the task model is a decision tree for a particular exercise. A parameter of that decision tree may, for example, have a threshold value. By machine learning, this threshold value is adjusted. After adjustment, the entire task model complete with all its parameter values (including the adjusted threshold value) is returned to the smartphone 1. The next time the user 5 uses smartphone 1, the new adjusted threshold value will be used in the decision tree to determine what next step in the decision tree to perform or proceed to. Each task experience data set includes task completion state information.
In order to determine the values of parameters of the decision tree (A), the machine learning method estimates the likelihood of successful task completion given the contextual information about the user and determined the parameter values as a function of the likelihood. This contextual information includes the user profile, the subjective task difficulty (STD) introduced by the user and information from previous “task experiences” (TE) the user had (e.g. number of times the user tried the exercise before, percentage of completion, minimum stress level in the tasks completed, stress level at the end of the last task trial, etc.). The contextual information is arranged as a numerical vector (S). The machine learning method (M) estimates the likelihood given S and a policy (p) as a function of the estimated likelihood can be used to make the decision. In one example, a contextual bandit method is used to determine the parameter values: Each time (t) that the central server 2 needs to select the parameter values for a user (u), the contextual information is computed as a function of the previous task experiences St,u=f(TE1,u, . . . , TEt-1,u). Given St,u∈Rn, the central server 2 determines the decision parameter values for the decision tree At,u∈Rm following a certain policy μ(St,u) (vector of probabilities of choosing each of the decision parameter values given St,u). Once the user finishes the task, the new TEt,u is received by the central server 2, which contains the result Rt,u=1 if the task was successfully completed or Rt,u=0 otherwise. The central server 2 uses the collected tuples {(S1,u, A1,u, R1,u), . . . , (St-1,u, At-1,u, Rt-1,u)}u of all users u to determine the result Rt-1,u as a function of the contextual information St-1,u and the decision parameters At-1,u:Rt-1,u=gθ(St-1,u, At-1,u). This function gθ is a machine learning model, e.g. a logistic regression or a feedforward neural network parameterized by the parameter vector θ. The machine learning model is trained in all the collected tuples using the vector St-1,u At-1,u as features and Rt-1,u as target for all users u, and time t to estimate the optimal parameter vector θ*. Then, when the parameter values of the decision tree need to be determined, the central server 2 uses the trained model (get) to estimate the result Rt,u given St,u for each possible value of the parameters of the decision tree. The central server 2 determines the updated model and parameter values according to a policy μ. In one example, the policy p is an E-greedy policy (epsilon-greedy reinforcement learning policy) that selects the decision tree parameter values with the highest estimated Rt,u argmaxAt,u gθ*(St,u, At,u) with probability (1−ε) and any other decision tree parameter value randomly with probability ε. Once determined, the updated model including its parameter values is sent to the smartphone 1.
Examples of methods usable to determine user state include: 1) An accelerometer-based method such as the stress detection method described in U.S. patent application Ser. No. 17/227,308, by Joao Guerreiro, entitled “Determining A Stress Level Of A Smartphone User Based On The User's Touch Interactions And Motion Sensor Data”, filed Apr. 10, 2021 (the entire subject matter of which is incorporated herein by reference). 2) A camera-based method such as is described in “Instant Stress: Detection of Perceived Mental Stress Through Smartphone Photoplethysmography And Thermal Imaging”, by Youngjun Cho et al., JMIR Mental Health, Vol. 6, No. 4 (2019), or as explained in “VitaMon: Measuring Heart Rate Variability Using Smartphone Front Camera”, by Sinh Huynh, et al., SenSys 2019: Proceedings of the 17th Conference On Embedded Networked Sensor Systems, New York, Nov. 10-13, 2019, pages 1-14. Examples of methods usable to determine situational state include: 1) “Learning To Detect Human-Object Interactions”, by Yu-Wei Chao, et al., IEEE Winter Conference on Applications Of Computer Vision (WACV), pages 381-389 (2018). 2) “PPDM: Parallel Point Detection And Matching For Real-Time Human-Object Interaction Detection”, Yue Liao, et al., Proceedings of the IEEE/CVF Conference On Computer Vision and Pattern Recognition (CVPR), pages 482-490 (2020).
Although certain specific embodiments are described above for instructional purposes, the teachings of this patent document have general applicability and are not limited to the specific embodiments described above. For example, the same methodology for encouraging a user to continue with an exposure therapy task can be used to motivate a user to continue with physical exercise training, such as weight training with ever more repetitions and/or weight. Accordingly, various modifications, adaptations, and combinations of various features of the described embodiments can be practiced without departing from the scope of the invention as set forth in the claims.
| Number | Name | Date | Kind |
|---|---|---|---|
| 20160140864 | Forman et al. | May 2016 | A1 |
| 20170319123 | Voss et al. | Nov 2017 | A1 |
| 20190246968 | Matic et al. | Aug 2019 | A1 |
| 20210391083 | Moturu | Dec 2021 | A1 |
| 20210401337 | Flickinger | Dec 2021 | A1 |
| 20220300787 | Wall | Sep 2022 | A1 |
| Number | Date | Country |
|---|---|---|
| WO 20200059789 | Sep 2019 | WO |
| WO 20200223600 | May 2020 | WO |
| WO 20200254127 | Jun 2020 | WO |
| Entry |
|---|
| Extended European Search Report dated Mar. 11, 2022, from the European Patent Office in the related foreign application EP21199455.3 citing the references A-F above (8 pages). |
| Number | Date | Country | |
|---|---|---|---|
| 20230005595 A1 | Jan 2023 | US |
