Patent Application
20180228430
The present invention is of a system, method and apparatus for rehabilitation with tracking, and in particular, to such a system, method and apparatus for rehabilitation with computational feedback, based upon tracking the movement of the user.
A stroke is a cerebrovascular accident that happens when the blood flow to a portion of the brain is disrupted, resulting in brain cell death. The consequences can be physical as well as cognitive, and can lead to a decrease in movement function and a loss of independence. This disorder is a major cause of long-term physical disabilities and handicaps in Western countries, mostly in the older age range of the population. Thus, as the worldwide population is aging, this disorder is one of the main concerns for the future of health care due to budgetary constraints limiting the intensity and length of the conventional rehabilitative treatment consisting of physical and occupational therapy (C. Bosecker et al., “Kinematic robot-based evaluation scales and clinical counterparts to measure upper limb motor performance in patients with chronic stroke,” Neurorehabilitation and Neural Repair, 2010).
Stroke survivors often experience hemiparesis, i.e., weakness on one side of the body, which can in turn lead to upper limb impairment and thus to a decrease in their independence and quality of life (I. Aprile et al., “Kinematic analysis of the upper limb motor strategies in stroke patients as a tool towards advanced neurorehabilitation strategies: A preliminary study,” BioMed Research International, 2014).
Stroke patients often use compensatory strategies to fill in for their lack of mobility, by using, for example, trunk recruitment, fixation of specific body segments or pathological synergies, e.g., reduction in shoulder movement can be compensated by gross flexion of the elbow, wrist moment and gravity (see, for example, M. C. Cirstea and M. F. Levin, “Compensatory strategies for reaching in stroke,” Brain, 2000; W. Liu et al., “Compensatory arm reaching strategies after stroke: Induced position analysis,” Journal of Rehabilitation Research and Development, 2013). M. C. Cirstea and M. F. Levin showed that the use of compensatory movement correlated well with the level of deficit and degree of spasticity of the patients. However, they also suggested that the use of inappropriate compensatory movements might have a negative impact on the rehabilitative process. Indeed, such strategies favor distorted positions of the joints, thus leading to a shortening of the muscles, which is an obstacle to recovery.
The motor deficits that may appear in the upper limb post-stroke are characterized by movements that show segmented patterns, meaning that they are composed of a concatenation of sub-movements (B. Rohrer et al., “Movement smoothness changes during stroke recovery,” The Journal of Neuroscience, 2002; L. van Dokkum et al., “The contribution of kinematics in the assessment of upper limb motor recovery early after stroke,” Neurorehabilitation and Neural Repair, 2013). This lack of smoothness together with slowness leads to a decrease in motivation for the use of the paretic arm (M. F. Levin et al., “Virtual reality environments to enhance upper limb functional recovery in patients with hemiparesis.” Studies in Health Technology and Informatics, 2009). Indeed, it causes clumsiness and a low level of precision, which make it difficult for the patients to perform Activities of Daily Living (ADL).
The difficulty of rehabilitation is increased, because these programs often have a low compliance if they are carried out without the presence of a therapist, due to a lack of motivation and slow evolution of the patient's progress. This is a problem because of the constant increase of cases and the lack of time for the therapist to fully be involved in everyone's therapy to the extent required.
The present invention, in at least some embodiments, is of a system, method and apparatus for rehabilitation with computational feedback, based upon tracking the movement of the user. Such a system, method and apparatus may be performed with or without the presence of therapist, increasing the therapeutic opportunities for the patient.
According to at least some embodiments, there is provided a method with a computationally directed set of movements, the set of movements being directed by a computational system that comprises providing visual direction to the patient and tracking the movements of the patient, the computational system comprising a display, at least one tracking sensor and a plurality of machine instructions for controlling the display to provide the visual direction and for receiving sensor data from the tracking sensor to track the movements of the patient, the method comprising: displaying a virtual object to the patient; indicating a movement to be performed with the virtual object; tracking said movement; and adjusting said displaying and said indicating according to said tracking; wherein said displaying, indicating and tracking apply a higher degree of therapeutic intensity as compared to a standard of care rehabilitative measure.
Optionally the computational system comprises: a depth camera and an RGB camera for obtaining tracking data, a tracking engine for tracking the movements of the patient, and a data analysis layer for analyzing the tracked movements and for adjusting said displaying and said indicating according to said tracking.
Optionally the computational system comprises a MindMotion™ PRO system.
Optionally said standard of care rehabilitative measure comprises GRASP.
Optionally said standard of care rehabilitative measure comprises at least one of reduced joint pain and improved motor function.
Optionally the method further comprises providing an improvement from baseline in upper extremity motor function measured by the Fugl-Meyer Assessment for Upper Extremity (FMA-UE) and/or its subscales as compared to said standard of care rehabilitative measure.
Optionally the method further comprises providing an improvement from baseline in upper extremity motor ability measured by the streamlined Wolf Motor Function Test (sWMFT) score as compared to said standard of care rehabilitative measure.
Optionally the method further comprises providing an improvement from baseline in self-care ability measured by the Barthel index (BI) as compared to said standard of care rehabilitative measure.
Optionally the method further comprises providing an improvement from baseline in functional independence measured by the Modified Ranking Scale (MRS) and/or associated disability-adjusted life year (DALY) as compared to said standard of care rehabilitative measure.
Optionally the method further comprises providing an improvement from baseline in the general health status as measured by the Stroke Impact scale (SIS) as compared to said standard of care rehabilitative measure.
Optionally the method further comprises providing an improvement from baseline in the severity of stroke symptoms as measured by the NIH stroke scale (NIHSS) as compared to said standard of care rehabilitative measure.
Optionally the method further comprises providing an improvement from baseline in arm function in daily activities as measured by the Motor Activity Log (MAL) as compared to said standard of care rehabilitative measure.
Optionally the method further comprises providing an improvement in motivation measured by the Intrinsic Motivation Index (IMI) as compared to said standard of care rehabilitative measure.
Optionally the method further comprises providing reduced therapist time spent administrating rehabilitation exercises as compared to said standard of care rehabilitative measure.
Optionally the method further comprises providing an improvement from baseline in upper extremity muscle strength measured by the Medical research Council Scale (MRC) as compared to said standard of care rehabilitative measure.
Optionally said muscle strength comprises one or more of strength for shoulder elevation, elbow flexion/extension, forearm pronation/supination and wrist extension/flexion.
Optionally said higher degree of therapeutic intensity comprises increasing an amount of time the patient spends during each therapeutic session, or increasing a number of exercises that the patient performs during said session, within a specific time frame, or both.
Optionally the method further comprises providing an increased rehabilitation dose as measured by the duration of the rehabilitation session without planned rest periods as compared to said standard of care rehabilitative measure.
Optionally the method further comprises performing the method during an acute period following a neurological trauma.
Optionally said neurological trauma comprises at least one of stroke and head injury.
Optionally the virtual object is displayed to the patient in an AR (augmented reality) or VR (virtual reality) environment.
Optionally the method further comprises determining a location of the virtual object in the AR or VR environment to avoid trunk involvement in a movement by the patient.
Optionally the method further comprises performing a calibration to determine a maximum reach of the patient and performing said determining said location according to a maximum of a range of 80-95% distance of said maximum reach of the patient.
Optionally said maximum distance is 95% of said maximum reach of the patient.
Optionally the method further comprises measuring an extent of trunk involvement according to movement of shoulders of the patient.
Optionally the method further comprises measuring EEG signals of the patient during said tracking said movement by the patient.
Optionally the method further comprises providing feedback to the patient according to said EEG signals.
Optionally the method further comprises providing feedback to the patient through a visual display of a mirror avatar.
Optionally the computational system comprises: a plurality of inertial sensors attached to the patient and an inertial sensor receiver for obtaining tracking data, a tracking engine for tracking the movements of the patient, and a data analysis layer for analyzing the tracked movements and for adjusting said displaying and said indicating according to said tracking.
According to at least some embodiments, there is provided a method for rehabilitating a patient with a computationally directed set of movements, the set of movements being directed by a computational system that comprises providing visual direction to the patient and tracking the movements of the patient, the method comprising: assessing an ability of the patient to perform a movement with a physical object; displaying a virtual object to the patient according to said assessing; indicating a movement to be performed by the patient with the virtual object, determined according to said assessing; tracking said movement; and adjusting said displaying and said indicating according to said tracking.
Optionally the computational system comprises: a depth camera and an RGB camera for obtaining tracking data, a tracking engine for tracking the movements of the patient, and a data analysis layer for analyzing the tracked movements and for adjusting said displaying and said indicating according to said tracking.
Optionally the virtual object is displayed to the patient in an AR (augmented reality) or VR (virtual reality) environment.
Optionally the computational system comprises a MindMotion™ PRO system.
Optionally the method further comprises receiving tracking data using an inertial sensor receiver of the computational system, the tracking data generated by a plurality of inertial sensors attached to the patient; analyzing data representing the tracked movements, using a data analysis layer of the computational system; and adjusting the displaying and the indicating according to the tracking using the data analysis layer of the computational system; wherein movements of the user are tracked using a tracking engine of the computational system.
According to at least some embodiments, there is provided a method for computationally directing a patient therapy, comprising: displaying a virtual object, using a display of the computational system; indicating a movement, using the display of the computational system, to be performed with the virtual object; tracking said movement, using a tracking sensor; receiving, using a data analysis layer, sensor data from the tracking sensor, the data analysis layer comprising a plurality of machine instructions; configuring one or more display instructions according to the tracking; and sending one or more display instructions to the display, the one or more display instructions to provide a visual direction on the display; wherein the indicating the movement and the configuring the one or more display instructions, indicating and tracking apply a higher degree of therapeutic intensity as compared to a standard of care rehabilitative measure.
Optionally the method further comprises assigning a visual indicator to the virtual object, the visual indicator communicating to the user to correct a trajectory.
According to at least some embodiments, there is provided a method with a computationally directed set of movements, the set of movements being directed by a computational system that comprises providing visual direction to the patient and tracking the movements of the patient, the computational system comprising a display, at least one tracking sensor and a plurality of machine instructions for controlling the display to provide the visual direction and for receiving sensor data from the tracking sensor to track the movements of the patient, the method comprising: displaying a virtual object; indicating a movement to be performed with the virtual object; tracking said movement; and adjusting said displaying and said indicating according to said tracking; wherein said displaying, indicating and tracking using a higher degree of therapeutic intensity as compared to a standard of care rehabilitative measure.
Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. The materials, methods, and examples provided herein are illustrative only and not intended to be limiting.
Implementation of the method and system of the present invention involves performing or completing certain selected tasks or steps manually, automatically, or a combination thereof. Moreover, according to actual instrumentation and equipment of preferred embodiments of the method and system of the present invention, several selected steps could be implemented by hardware or by software on any operating system of any firmware or a combination thereof. For example, as hardware, selected steps of the invention could be implemented as a chip or a circuit. As software, selected steps of the invention could be implemented as a plurality of software instructions being executed by a computer using any suitable operating system. In any case, selected steps of the method and system of the invention could be described as being performed by a data processor, such as a computing platform for executing a plurality of instructions.
Although the present invention is described with regard to a “computer” on a “computer network,” it should be noted that optionally any device featuring a data processor and the ability to execute one or more instructions may be described as a computer or as a computational device, including but not limited to any type of personal computer (PC), a server, a cellular telephone, an IP telephone, a smart phone, a PDA (personal digital assistant), a thin client, a mobile communication device, a smart watch, head mounted display or other wearable that is able to communicate externally, a virtual or cloud based processor, or a pager. Any two or more of such devices in communication with each other may optionally comprise a “computer network.”
The invention is herein described, by way of example only, with reference to the accompanying drawings. With specific reference now to the drawings in detail, it is stressed that the particulars shown are by way of example and for purposes of illustrative discussion of the preferred embodiments of the present invention only and are presented in order to provide what is believed to be the most useful and readily understood description of the principles and conceptual aspects of the invention. In this regard, no attempt is made to show structural details of the invention in more detail than is necessary for a fundamental understanding of the invention, the description taken with the drawings making apparent to those skilled in the art how the several forms of the invention may be embodied in practice.
Embodiments provides VR-based exercises for upper-limb neurorehabilitation after brain injuries. An implementation of the MindMotion™ PRO platform with immersive virtual reality (VR) is one exemplary embodiment. This platform is a mobile unit that includes a computing unit, a camera with stereo and depth sensors, and embedded 3D image processing system that captures motion by tracking the movement of six colored markers positioned on the joints, and two inertial hand sensors that add precision to the orientation of the subject's arm. Skilled artisans can appreciate that other embodiments need not be mobile, can include other or fewer components, and can include components configured in different ways. The MindMotion™ PRO platform is but one exemplary embodiment and should not be viewed as limiting to other embodiments. The colored markers are preferably active markers that emit a signal, such as LED lights for example, and more preferably emit different signals, such as different colored lights. Skilled artisans can appreciate that types of signals other than LEDs can be used. However, optionally no such markers are used in the form of devices attached to the subject, and the markers are instead data points that are detected to determine the location of particular joints in the data obtained, as described for example in PCT Application No. PCT/US18/17292, filed on Feb. 7, 2018, owned in common with the present application, which is hereby incorporated by reference as if fully set forth herein.
Optionally no inertial sensors are used; alternatively and optionally, the camera could be replaced by the inertial sensors. Optionally all markers are replaced by the inertial sensors.
Positions (3D cartesian coordinates) and orientations (quaternions) of the joints are mapped in real-time onto an avatar following the participant's movement. Motion data are recorded at a suitable sampling frequency, such as 30 Hz for example, and without limitation, and are stored in the computation unit for further analysis. Additionally, there are two screens for one for the exercises and one for the monitoring, respectively for the patient and the therapist, and a battery unit.
As described in greater detail below, optionally EEG signals (and/or optionally other biosignals) can be measured from the patient (not shown).
Sensor data preferably relates to the physical actions of a user (not shown), which are accessible to the sensors. For example, camera 102 may optionally collect video data of one or more movements of the user, while depth sensor 104 may provide data to determine the three-dimensional location of the user in space according to a distance of a part of the user from depth sensor 104. Depth sensor 104 preferably provides TOF (time of flight) data regarding the position of the user. The combination of sensor data from depth sensor 104 with video data from camera 102 allows a three-dimensional map of the user in the environment to be determined. As described in greater detail below, such a map enables the physical actions of the user to be accurately determined, for example, with regard to gestures made by the user.
To assist in the tracking process, optionally one or more markers 118 are placed on the body of a user. As used herein, the term “tracking” relates to tracking the movements of the user, whether through markers 118 or any other landmark or point(s) to follow. Markers 118 optionally feature a characteristic that can be detected by one or more of the sensors. Markers 118 are preferably detectable by camera 102, for example as optical markers. While such optical markers may be passive or active, preferably markers 118 are active optical markers, for example featuring an LED light. More preferably each of markers 118, or alternatively each pair of markers 118, comprises an LED light of a specific color which is then placed on a specific location of the body of the user. The different colors of the LED lights, placed at a specific location, convey a significant amount of information to the system through camera 102; as described in greater detail below, such information can be used to make the tracking process efficient and accurate. Alternatively, as described above, no such markers 118 are used and instead data points relating to specific joints are detected. Such markers, or the above described camera or other sensors for tracking the movements of the user, or a combination thereof may be described as a tracking device.
A computational device 130 can receive sensor data from camera 102 and depth sensor 104. Sensor data from other sensors can be received also. Any method steps performed herein may optionally be performed by such a computational device. Further, all modules and interfaces shown herein are assumed to incorporate, or to be operated by, a computational device, even if not shown. Optionally preprocessing is performed on the signal data from the sensors. Computational device 130 features a plurality of machine instructions controlling the display to provide the visual direction and for receiving sensor data from the tracking sensor to track the movements of the user.
Preprocessed signal data from the sensors can be passed to a data analysis layer 110, which preferably performs data analysis on the sensor data for consumption by an application layer 116. Application layer 116 can provide any type of interaction with a user. Preferably, such analysis includes tracking analysis, performed by a tracking engine 112. Tracking engine 112 preferably tracks the position of a user's body and preferably of one or more body parts of a user, including but not limited to one or more of arms, legs, hands, feet, head and so forth. Tracking engine 112 optionally decomposes data representing physical actions made by a user to data representing a series of gestures. A “gesture” in this case may optionally include an action taken by a plurality of body parts of a user, such as taking a step while swinging an arm, lifting an arm while bending forward, moving both arms and so forth. Such decomposition and gesture recognition could optionally be done separately.
Tracking data, generated by tracking engine 112, representing the tracking of a user's body and/or body parts, optionally decomposed to data representing a series of gestures, can then be received by application layer 116, which can translate the data representing physical actions of a user into data representing some type of reaction, analyzes this reaction data to determine one or more action parameters, or both. For example, and without limitation, a physical action taken by the user to lift an arm is a gesture and is captured by cameras and sensors (102, 104) among other possible sensors which generate sensor data. The sensor data could be translated by application layer 116 to data for generating an image of an arm lifting a virtual object. Alternatively, or additionally, such data representing a physical action could be analyzed by application layer 116 to determine the user's range of motion or ability to perform the action. Application layer 116 could for example provide a game for the user to perform as described herein.
Optionally, application layer 116 could create a mirror avatar to provide feedback, which would mirror the user's motions and provide a visual display of such motions. Such a mirror avatar is illustrated and described below in connection with
Data analysis layer 110 also preferably includes a system calibration module 114. As described in greater detail below, system calibration module 114 can calibrate the system in regard to the position of the user, in order for the system to be able to track the user effectively. System calibration module 114 may optionally perform calibration of the camera 102, depth sensor 104, as well as any other sensors, in regard to the requirements of the operation of application layer 116; however, preferably device abstraction layer 108 performs any sensor specific calibration. Optionally, the sensors may be packaged in a device, such as the Kinect, which performs its own sensor specific calibration.
In preferred embodiments, after each repetition of an exercise, a score appears informing the player about his performance during the task. Additionally, in some preferred embodiments, if a task is not completed within 5 s, a timeout warning appears and the exercise resumes. For example, in the reaching exercise of the embodiment illustrated in
Game events, or triggers, can be determined by the super-position of collision volumes of the hand and the elements of the game (for example, start-pad 210, path 206, target 202, and the like). Events can include: contact with the start-pad, contact with the target, mistake feedback on, success feedback on, timeout feedback on, and the like. The beginning of movement can be defined as when the collision volumes of the hand and start-pad do not superimpose anymore, and the end of movement is set when the tip of the participant's fingers reach the target.
The systems and games as described herein may be implemented in an overall plan for rehabilitation, and are currently being tested in a clinical trial, as described with regard to Example 1 below.
An additional plan for implementing such systems and games for rehabilitation that leads to an increase in an ability to perform ADL (activities of daily living) is described with regard to Example 2 below.
The Move-Rehab clinical trial protocol may be found at https://clinicaltrials.gov/ct2/show/NCT02688413, and is described below briefly for clarity only. The trial is intended to show that exercises according to methods which embody the invention or which use systems in accordance with embodiments of the invention, including exercises embodied in MindMotion™ PRO exercises, can deliver a higher degree of therapeutic intensity as compared to standard of care rehabilitative measures that are currently considered to be intensive. The standard of care protocol that is being used for the comparison is called GRASP (Graded Repetitive Arm Supplementary Program). GRASP is an arm and hand exercise program for stroke patients. Therapeutic intensity is a combination of the amount of time that a patient can spend during each therapeutic session, plus the number of exercises that the patient can do during each session, within a specific time frame. Two ways to increase therapeutic intensity are by increasing the amount of time spent or by increasing the number of exercises performed during a particular period of time.
In addition, the clinical trial is intended to determine whether the standard of care in accordance with embodiments of the invention, including the standard of care delivered by the MindMotion™ PRO platform, is more effective than the standard of care, which is again GRASP, in comparison to the below list of changes from baseline in comparison to the standard of care.
Further discussion of the clinical trial protocol reference MindMotion™ PRO. It is to be understood that such references are but one preferred embodiment and do not operate to limit the invention.
Purpose
Randomized controlled multi-centered study using MindMotion™ PRO, an immersive virtual reality-based system for upper limb motor rehabilitation in early post-stroke patients. The study aims to evaluate the ability of MindMotion™ PRO technology to increase the rehabilitation dose. Effectiveness will be evaluated by validated rehabilitation performance scales. Cost-effectiveness will be assessed by the resource utilization. Table 1 shows a summary.
Rehabilitation dose as measured by the duration of the rehabilitation session without planned rest periods [Time Frame: 4 weeks]
Number of exercises performed [Time Frame: 4 weeks]
Change from Baseline in upper extremity motor function measured by the Fugl-Meyer Assessment for Upper Extremity (FMA-UE) and its subscales [Time Frame: baseline, 4 weeks, 16 weeks]
Change from Baseline in upper extremity motor ability measured by the streamlined Wolf Motor Function Test (sWMFT) score [Time Frame: baseline, 4 weeks, 16 weeks]
Change from Baseline in self-care ability measured by the Barthel index (BI) [Time Frame: baseline, 4 weeks, 16 weeks]
Change from Baseline in functional independence measured by the Modified Ranking Scale (MRS) and associated disability-adjusted life year (DALY) [Time Frame: baseline, 4 weeks, 16 weeks]
Change from Baseline in the general health status as measured by the Stroke Impact scale (SIS) [Time Frame: baseline, 4 weeks, 16 weeks]
Change from Baseline in the severity of stroke symptoms as measured by the NIH stroke scale (NIHSS) [Time Frame: baseline, 4 weeks, 16 weeks]
Change from Baseline in arm function in daily activities as measured by the Motor Activity Log (MAL) [Time Frame: baseline, 4 weeks, 16 weeks]
Motivation measured by the Intrinsic Motivation Index (IMI) [Time Frame: 1 week and 4 weeks]
Resource utilization: time spent administrating rehabilitation exercises [Time Frame: 4 weeks] therapist (physiotherapist or other medical staff) time spent administrating rehabilitation exercises
Change from Baseline in upper extremity muscle strength measured by the Medical research Council Scale (MRC) [Time Frame: Baseline, 1 week, 2 weeks, 3 weeks, 4 weeks and 16 weeks] muscle strength for shoulder elevation, elbow flexion/extension, forearm pronation/supination and wrist extension/flexion.
Table 2 shows an effect on the arms.
The goal of the study is to show that the MindMotion™ PRO platform (or other embodiments) is a tool that allows a patient to increase the amount of rehabilitation therapy performed. The study measures the rehabilitation dose, as measured by the duration of the rehabilitation session and the number of exercises performed by the patient. The study hypothesis is that patients in the MindMotion™ PRO group spend more time performing rehabilitation exercises than in the Self-Directed Prescribed exercises group and, thus, receive more effective rehabilitation. The effectiveness of the MindMotion™ PRO versus Self-Directed Prescribed Exercises are also measured, based on the change in rehabilitation performance measures. The cost-effectiveness is measured by the resource utilization, as defined by the time spent by the therapist providing the rehabilitation session.
Ages Eligible for Study: 18 Years and older (Adult, Senior)
It is highly desirable to improve ADL for rehabilitation of patients, particularly for stroke victims. As noted in the Background, rehabilitation for such patients can be quite difficult, due to lack of necessary therapists and therapist resources to fully impact all patients.
The method described herein, which is described as being implemented with MindMotion™ PRO but which may, in fact, be implemented with other systems and methods in accordance with embodiments of the invention as described herein, relates to a specific rehabilitation process that leads to an increase in the ability of the patient to perform ADL.
Part of the consideration relates to outcomes of specific motions that could lead to an improvement in ADL performance by the patient. These outcomes are listed below.
Tables 3 and 4 list desired outcomes that could be measured and improved by a system as described herein, upon interacting with the patient. RoM as used herein relates to range of motion.
For each activity, such as a game, the start hand position=hand placed on the table so that the wrist joint is aligned with the table edge. the arm should be positioned with elbow flexion: 90°, shoulder flexion: 0°, shoulder abduction: 0°.
The game would induce the patient to reach for a virtual object at the positions shown which would lead to an improvement in the patient's ability to perform ADL.
Defining the Workspace:
In the acute phase, for example following a neurological trauma (including but not limited to a stroke or traumatic head injury), it is desired to avoid compensation as the brain is highly plastic so there is a better chance of true recovery. The workspace is defined below with regard to the goals to be reached.
X-Y direction: reach exercise should be the maximal arm extension without compensation. See if in grab exercise there is trunk compensation, if yes find the distance at which it starts. Find distance at which compensation starts in place exercise.
Use the maximal displacement out of the first 3 exercises of Table 3 to define the maximal Y distance. Use the maximal and minimal X values from the 1 reach exercise.
Z direction: using exercises 5-7 of Table 3 find maximal height reached.
Specific examples of such games may optionally be performed by interactions with the MindMotion™ PRO system or, again, other embodiments, as described in greater detail below, in Table 5. All exercises may be set with a 25 s time-out for example.
To test the above games for their ability to improve ADL performance, a specific set of movements were selected as described with regard to Table 6.
A long straight-line trajectory would emphasize the smoothness parameter as the longer the trajectory, the higher the possible NMUs. A far reach would cause high elbow ROM or/and trunk displacement. A mid-sagittal plane reach would avoid having to distinguish between natural trunk rotation and compensatory trunk rotation. Shoulder elevation can be emphasized by reaching for objects on high platforms.
The first sequence of movements was chosen to find the maximal elbow flexion. The subjects were therefore asked to reach as far away as possible by sliding their hand along the table in five different directions while keeping their back against the chair. In order to then find the maximal possible reach, and to find at what point trunk compensation starts, the second movement chosen was to take a cup and place it at different distances on the table along the mid-sagittal plane. These movements belong to the function domain of the ICF. The remaining movement sequences were based on activities of daily living.
An assessment of the movement capabilities of the patient is first performed, for example as described above. From such an assessment, the farthest distance in the Y (away from the patient, Ymax) and X (to the side of the patient, Xmax and Xmin) directions can be decided, for object placement. The placement of the objects between will be placed along the ellipse made by the maximal positions according to the following equation:
During game time, a certain percentage of the movements would be made to reach objects at the assessment limit, set as x %, and (100-x) % farther than the limit, such as for example 80% at the limit and 20% beyond the limit. If there was a sequence of 10 movements for one exercise for example, 8 of the movements would be to the limit points (blue in
One popular specific task for rehabilitation for ADL is a drinking task which comprises reaching, grasping, transporting, drinking, and placing a water filled cup. This task uses a cup filled with water, therefore an adapted version of this task avoiding spillage was used. The subjects were asked to pretend to drink from an empty weighted cup. This task may be impossible for severely impaired patients, and therefore a more general task, which includes reaching, grasping, transporting, and placing a cup was also chosen. A few other references, as well as the ones for the drinking task, also include these aspects. Many ADLs include this series of aspects such as drinking, eating, teeth brushing, shampooing etc. and mostly include bringing an object towards oneself. For this reason, a task to grab a cup and bring it towards oneself was chosen. This task was adapted to have easy and difficult versions by having the objects placed in different positions (contralateral, ipsilateral and midsagittal) and at four different heights.
According to Fitts' law (described in P. M. Fitts, “The information capacity of the human motor system in controlling the amplitude of movement,” Journal of Experimental Psychology, vol. 47, pp. 381-391, June 1954), the farther the movement distance, the shorter the time, and the higher the precision, the more difficult a movement is. Therefore, reaching for a small object far away is more difficult than reaching for a large object that is close. Even though the MindMotion™ PRO system as used in the experiment does not track finger movements, a more difficult fine motor skill task was included as a movement sequence to see if the proximal joints perform differently than with easier tasks. The chosen task was to take a spoon and put it in the cup, and then take a sugar cube to put in the cup. The last part of the sequence includes bringing the cup back towards oneself to see if this movement is performed at a slower pace than earlier. This was included to verify for fatigue.
The task of grasping a real cup, as an ADL, was tracked with the MindMotion™ PRO technology, to determine the motor performance. This information is being used to adapt the MindMotion™ PRO game activities to specifically connect the missing or reduced capabilities to the games that the patient performs. Based on the connection between the patient's deficits in an ADL activity and the MindMotion™ PRO, the MindMotion™ PRO activities are adapted for the specific patient, to overcome these deficits.
The following concepts are tested: correlation between the performance of a healthy subject in an ADL and the motions that can be decomposed to MindMotion™ PRO activities. Further, it is validated that the MindMotion™ PRO activities actually demand the same movements that need to be improved.
The accuracy of the MindMotion™ PRO was validated through experimental testing, which demonstrated that the MindMotion™ PRO can track and measure movements with a high degree of accuracy.
Methods
A draw-wire encoder (Phidgets encoder ENC4104_0) with a 0.08 mm displacement resolution was used. A joining piece was 3D-printed to couple the marker to the encoder, and another one to screw to the encoder to be able to clamp the device to the table. Various tests were then performed, in terms of various movements that were performed:
1) at a natural pace and at the fastest pace the tester could move.
2) parallel to the camera close, far, and central on the testing table and perpendicular to the camera right, left and central on the testing table
3) with a camera tilted downwards at two different angles
4) along the axes of the left arm movement assessment schema, as this represents the region of the movements subjects perform during the assessment.
5) with a new and old camera calibration
The linearity of the movement was guaranteed by clamping a plastic board to the table and sliding the maker/cable joining piece along the edge of it. For each test the movement was repeated five or ten times in each direction.
The analysis of the data included finding the difference of the maximal measured displacement of the two systems, and the percentage error as followed:
Therefore, a positive difference indicates an overestimation of the maximal displacement and a negative difference an underestimation of the maximal displacement. Moreover, the Bland-Altman graph of each movement was plotted (see J. M. Bland and D. Altman, “Statistical methods for assessing agreement between two methods of clinical measurement,” The Lancet, vol. 327, no. 8476, pp. 307-310, 1986, available at: http://dx.doi.org/10.1016/S0140-6736(86)90837-8). This corresponds to plotting the average of both displacement values against the difference between them at each time point.
In order to plot these, the movement graphs (x-axis: time, y-axis: displacement) had to be temporally aligned. This was done by finding the moment when the displacement value was 50% of the averaged maximal displacement, and aligning the movement plots at this point. Moreover the temporal rate had to be the same in order to compare the displacement of the two systems at each time point. Since the marker data was recorded with a rate of 30 Hz, and the encoder recorded data did not have a steady rate (as it only recorded data at a change of displacement, maximum 140 Hz for fast movements), the encoder data was interpolated linearly and replotted with the same time points as the marker data.
A static validation was also implemented. This validation included filming the six markers attached to a board at specific distances from each other and verifying that the measured distance between markers remain the same no matter the recording day, the marker board position and orientation, and the camera orientation. The six markers were attached in the same disposition as they would be on a patient, that is the two shoulder markers aligned, the two elbow markers below those and then wrist markers below again, and taking into account left and right. Moreover, a static validation for the measurement of height was performed. A marker was placed on the table and on platforms of different heights used for the upper limb assessment. The platforms were 3D printed at heights of 11, 23 and 38 cm. The platforms with the markers were placed on the same table position one after the other. They were then filmed on different days, to verify that the heights measured by the camera system remain the same.
The first test was performed with regard to speed of movement: Each movement along the arrows of the assessment schema (X, Y, D2, D4 in
Location of movement in the camera's field of view: Each movement was performed five times for each camera orientation. Again, the errors were very low or negligible.
With the camera tilted downwards at two different angles: changing the downward orientation of the camera had little or no effect.
With a new and old camera calibration: The movements were processed once with the calibrated camera data from 13 months previous to the test, and with the calibrated camera data from a new calibration. Similar data was obtained on both occasions, indicating the stability of the calibration.
Along the arrows of the assessment scale: For the displacement validation performed along the arrows of the assessment scale, each movement was plotted on a displacement against time graph to verify that the temporal alignment was correct. Overall, the results show high repeatability.
Static validation: The static validation to verify the displacement between the markers was performed on multiple occasions. The deviation between measurements was very low or negligible.
The performance of the MindMotion™ PRO system was tested with 10 healthy subjects. The tests demonstrated that MindMotion™ PRO could accurately measure the movements of the subjects. In addition, various adjustments to the games and system operation were also determined and are described herein. An exemplary set of measurements for determining trunk displacement with the MindMotion™ PRO system is also described.
Methods
As noted above, a workspace was defined so as to assist the subjects in performing the motions and also measurement of these motions.
The chosen task was to take a spoon and put it in the cup, and then take a sugar cube to put in the cup. The last part of the sequence includes bringing the cup back towards oneself to see if this movement is performed at a slower pace than earlier. The weighted plastic cup weighed 150 g to be similar to other research groups. The cup was 5.3 cm and 7.3 cm in diameter at the bottom and top respectively, and 11.7 cm high. For the fine motor task, a 19 cm long metal spoon was used, and a sugar cube of dimensions 3.5×2.3×1.2 cm was used.
The movements of 10 subjects were measured. The subjects were seated so that their feet were firmly on the floor. The chair was positioned so that the subjects' wrists were aligned with the edge of the table when they had 0. shoulder flexion and adduction. The camera was aligned to the mid-sagittal plane of the subject and approximately a meter away, as shown in
The end-effector workspace was calculated by finding the maximal position of the wrist marker in three perpendicular directions during the reaching exercise without trunk movement. The results were used to adapt the virtual object grabbing height, and the target positions of the MindMotion™ PRO VR grasp exercise. The start pad was positioned at the same distance away from the body as the green target from the assessment had been (
Results
The measurements by the MindMotion™ PRO system were consistent with actual movements of the subjects' hands and of objects grasped therein, showing that the system correctly measurements movement and displacement in space (data not shown). In addition, the measurements of movements made in a virtual environment were consistent with results obtained by other researchers with actual physical objects, as shown in
For the measurements in
System Operation Adjustments
Trunk Involvement—
As noted above, various adjustments to the operation of the MindMotion™ PRO system or other embodiments may be made for various therapeutic effects. In particular, the games may be adjusted to avoid involvement of the trunk, for example during an early time period after brain injury, such as after stroke. Such adjustments may include setting a maximum reach to avoid trunk involvement, such as for example up to 95% of the maximum reach displayed during calibration, up to 90%, up to 85%, up to 80% or any number in between.
Determining Trunk Involvement—
In order to prevent trunk involvement or trunk motions from occurring, it is necessary to at least detect trunk involvement. Such detection may optionally be performed by measuring trunk displacement as the mid-point between the two shoulder markers. As noted above, by “markers” it is optionally meant any type of detection of specific points associated with the joints, whether with active markers or through other types of detection. Preferably, specific markers associated with the shoulders are used for the determination of trunk involvement; optionally the game or virtual environment interactions may then be adjusted to reduce or remove trunk involvement, for example by adjusting the location of the virtual object.
The performance of the MindMotion™ PRO system was tested with a subject who had suffered a stroke. A 60-year-old right-handed (Edinburgh test >95) male ischemic stroke (left paramedian pontine) subject with hemiparesis on the right side (at Day 8: NIHSS=6, Fugl-Meyer Assessment=26/66) was recruited. In addition, 13 age-matched right-handed (Edinburgh test >80) adults (58 9 years; 7 males and 6 females) were also recruited.
All participants performed planar reaching exercises that implied active shoulder and arm movements using the MindMotion™ PRO system that provided embodied two embodied visual feedback: (i) direct mode (arm movements are translated to the ipsilateral an avatar on the computer screen); and (ii) mirror mode (arm movements are translated to the contralateral side of the avatar). These activities may be described as VR mediated mirror visual feedback (VR-MVF). The 16 electrode Electroencephalogram of the participants was recorded from frontal, central and parietal areas, while performing the movements.
The results showed that VR-MVF is likely to influence cortical sub-threshold activity during movement preparation and execution, and results in more balanced activity as observed in MRCPs (Movement-Related Cortical Potentials). Without wishing to be limited by a single hypothesis, VR-MVF may increase cortical excitability around the penumbra of the lesion in a selected group of stroke patients and hence could be used as a therapeutic intervention.
Methods
The previously described stroke survivor was hospitalized on Day 0, assessments were performed on Day 8 (Pre), Day 13 (Post) and after follow-up. The intervention was performed for 3 consecutive days, Day 9, 10 and 11 using MindMotion™ PRO system for neurorehabilitation as previously described with centered-out reaching tasks directly at the bedside at the acute neurorehabilitation center. The participant also received approximately full body physiotherapy approximately 60 min per day.
Experimental Setup
Participants were asked to execute upper limb centered out reaching movements in a 2-dimensional plane (i.e. over a physical table) to five locations randomly presented using the MindMotion™ PRO system as previously described for delivering game-like rehabilitation exercises. The system, shown in
In a typical centered-out reach exercise, when ready, the patient places hand on Start pad, and after a random period of time (max 2 s) a target (35 cm distance) is shown in one of the five locations (each 32.5 cm apart) as shown in
Data Collection
Electroencephalogram
From all participants full-band electroencephalogram (fbEEG) data was recorded in 10-20 international system using 16-electrodes (FC3, FCz, FC4, C5, C3, C1, Cz, C2, C4, CP3, CP1, CP2, CP4 and Pz) spanning frontal, central and parietal areas, while performing the centered out task, using g.USBamp (g.tec medical engineering GmbH, Austria) at the sampling rate of 512 Hz. Two additional electrodes were used as ground (AFz) and reference (right earlobe).
Outcome Measures
The stroke participant's impairment was measured using National Institute of Healthy Stroke Scale (NIHSS) (L. Zeltzer, Stroke engine: Assessments: National Institutes of Health Stroke Scale (NIHSS)), Fugl-Meyer upper limb scale (FMUL) (A. R. Fugl-Meyer, L. Jaasko, I. Leyman, S. Olsson, S. Steglind, The poststroke hemiplegic patient. 1. a method for evaluation of physical performance, Scand J Rehabil Med. 7 (1975) 13-31) in motor, sensation and passive joint motion and Frenchay arm test (FAT) (D. T. Wade, R. Langton-Hewer, V. A. Wood, C. E. Skilbeck, H. M. Ismail, The hemiplegic arm after stroke: measurement and recovery, Journal of Neurology, Neurosurgery, and Psychiatry 46 (1983) 521-524) at Day 8, Day 13 and at the follow up day. Additionally, the patient's self-reporting on various aspects of motivation and engagement was recorded.
Data Processing
Electroencephalogram
Firstly, the DC offset of the EEG recordings was reduced by removing the first sample from each electrode data. Then, the data was downsampled to 64 Hz (using zero-phase low-pass 3rd order Butterworth filter fc=24.6 Hz). The data was then band-pass filtered in the range 0.1 1.5 Hz that correspond the spectral content of movement related slow cortical potentials (SCPs) in EEG. The data was then re-referenced to the average activity of C5 and C6.
Grand-Averages
Epochs were extracted from the full recordings using a −2 s to −5 s time window around movement onset at 0 s (
The potential recorded from each electrode, for each epoch, was then corrected for baseline activity, identified at −1.5 s. The trials were rejected if the movement onset was detected too late or multiple movement onsets were detected. Additionally, artifact data that may have resulted due to movement or other external causes that exceeded +/−120 microV at any electrode were rejected from the further analysis. The remaining epochs are then averaged across all healthy participants separately for each condition and electrode. The stroke participant EEG epochs data were treated in a similar way. Due to the reduced number of trials available from the stroke participant in the Direct Right (i.e. paretic arm movements) after the artifact rejection, further analysis was limited to Direct Left and Mirror Left conditions.
Topographic plots of the average activity at each channel during a short time window selected using the time point that corresponded to the maximal difference between the conditions observed using student t-test over C2 electrode data across conditions for pooled healthy participant data and stroke participant data separately.
Results
Healthy Participants
The grand average traces for the pooled data of healthy participants is presented in
A window of activity of MRCP for topographic maps was chosen based on maximal differences observed between Mirror Right and Direct Right conditions (t-test, pi0.01). Topographic maps of MRCPs for the all the conditions is shown in
One way ANOVA performed at the group level for electrodes separately revealed pi0.01 at electrode sites FC3 (F=7.98), FCz (F=18.7), FC4 (F=33.4), C3 (F=12.6), Cz (F=21.6), C2 (F=29.9), C4 (F=41.7), CPz (F=31.5), CP2 (F=31.3) and CP4 (F=32.4). The pair wise Wilcoxon rank sum test between Direct Right and Direct Left showed significant differences (pi0.01) at electrodes FCz, FC4, Cz, C2, C4, CPz, CP2 and CP4. Similarly, significant differences between Direct Right and Mirror Right was observed at electrodes C1, CP1, CP2 and CP4.
Stroke Participant
The grand average MRCP data of the stroke participant is presented in
A window of activity of MRCP for topographic maps was chosen based on maximal differences observed between Mirror Right and Direct Right conditions (t-test, pi0.01). Topographic maps of MRCPs for the all the conditions are shown in
Table 5 shows the various improvements in the condition of the stroke participant according to various measures. The outcome measures of the stroke participant were measured at Pre and Post VR-sessions and at Follow-up.
The NIHSS score reduced from 6 to 3 post VR-intervention, which corresponds to a mild impairment and hence discharged from the acute neurorehabilitation center. The FAT score did not change from 3 Pre to Post but improved at the Follow-up. Furthermore, the motor impairment of the right arm as measured with FMA-UE in motor function, sensation, passive joint motion and joint pain showed clinically important improvement (=6 points; with main contributions from the synergies of shoulder, elbow and forearm) observed in motor function with at Post compared to Pre VR-intervention (=26 points), which further improved.
The stroke participant showed significant improvement in various measures as described herein. The addition of the EEG cap permits additional feedback to be provided to such participants, such that the method of treatment may be adjusted according to the EEG measurements.
Furthermore, the stroke participant performed a high number of activities with the MindMotion™ PRO system: during the 3 day (Day 9-11) intervention the patient performed 160 Direct Paretic and 150 Mirror non-paretic repetitions, which is generally considered as high intensity during the acute hospitalization period. Pre (Day 8) and Post (Day 13) VR therapy intervention showed an increase in FMA-UE score of 6 points which is above minimal clinically important difference in FMA-UE score. The score further increased at the follow-up. An improvement in the joint pain of 3 points at the post-therapy and further improvement at the follow-up was also observed. By answering a routine questionnaire, the stroke participant reported a high level of concentration, enjoyment and relaxation although increased level of fatigue while performing the VR exercises. Interestingly, he also reported the willingness to continue performing the VR exercises at home and demanded more challenging exercises.
While the invention has been described with respect to a limited number of embodiments, it will be appreciated that many variations, modifications and other applications of the invention may be made, including different combinations of various embodiments and sub-embodiments, even if not specifically described herein.
| Number | Date | Country | |
|---|---|---|---|
| 62457192 | Feb 2017 | US | |
| 62467872 | Mar 2017 | US |
