Patent Application
20230025266
Annually, about 800,000 individuals in the US and 17 million worldwide suffer a stroke. Most are left with residual neurological deficits that persistently impair function, making stroke the leading cause for serious disability in developed countries. Meanwhile, prevalence rates in low- and middle-income countries will shortly surpass those of developed nations. Among the most common and disabling deficits is arm hemiparesis as it has been reported as the dominate functional limitation in as much as 80% of patients with acute stroke. Even after patients have left the acute care setting, 50% to 95% of individuals have continued impairment of the upper extremities (UE).
Current widespread care and therapy models focus on unilateral training of the paretic arm and when not successful, resort to learning compensatory techniques with their non-paretic (NP) arm. As a result, the current standard of care is often limited to managing (not resolving) the problem, and relies on the patient learning compensatory skills to make up for their deficit, which may result in further learned disuse
What can be important to the stroke survivors is their ability to carry on their daily tasks. Many core activities of daily living (ADL), like dressing, bathing, toileting, and mobility functions like carrying objects, require the use of both arms. Furthermore, aging adults that are more likely to suffer stroke have an increased reliance on both arms to perform their ADL. If most tasks are performed bilaterally in those most likely to suffer a stroke, aging adults, then training should be task specific and thus, bilateral. The focus on task-specificity and bilateral training is supported by distinct neuromotor control mechanisms at play in unilateral versus bilateral UE skills
A bilateral arm trainer with rhythmic auditory cuing (BATRAC) apparatus has been shown to improve UE motor function in mild/moderately affected patients well beyond the 3-month period (chronic hemiparesis) after 6 weeks of repetitive non-progressive training. In a randomized study, functional imaging (fMRI) has shown that new foci of contralesional as well as ipsilesional neural activation for paretic arm movement occurs after BATRAC training, but not after dose-matched therapeutic exercises [4].
BATRAC was used for “no-assist” training only, with no instrumentation for measuring patient performance and only auditory cueing to provide patient stimulus. It is here recognized that adding instrumentation to the BATRAC, to sense patient performance or provide video feedback or both, offers feedback to patients during use to enhance functional outcomes, while also allowing care providers (such as physical therapists, PTs) to track patient progress objectively. Furthermore, instrumenting with assist mechanisms will produce repetitive training without human intervention. Even further, configuring bilateral interaction in both performance and assist is likely to lead to better outcomes. Thus, techniques are provided for arm training that enable and are suitable for automated objective performance assessment or automated repeated or bilateral training or some combination.
In a first set of embodiments, a therapeutic device includes a handle, a sensor assembly, and a display. The handle is configured to move along a path. The sensor assembly is configured to provide a measured time series comprising a plurality of handle location measurements at a corresponding plurality of measured time instances. The display is configured to present an image based on the measured time series.
In some embodiments of the first set, the therapeutic device includes a handle drive assembly secured to the handle and configured to apply position-correcting force to the handle when the measured time series indicates an actual position of the handle which deviates from a target trajectory for the handle by a predetermined tolerance. The handle drive assembly is also configured to apply no position-correcting force to the handle so long as the actual position of the handle is within the predetermined tolerance. In some of these embodiments, the handle drive assembly is configured to be backdriven by the handle only when the position of the handle is within the predetermined tolerance.
In some embodiments with a handle drive assembly, the handle drive assembly includes a position-controlled DC motor secured to the handle via a linkage. In some of these embodiments, the linkage comprises a lead screw secured to the DC motor and a linear actuator configured to convert rotation of the lead screw into translation of the handle.
In some embodiments of the first set, the device includes two handles, one handle for each hand of a user. In some of these embodiments, the device includes two handle drive assemblies, one handle drive assembly for each handle.
In some embodiments of the first set, a system includes the therapeutic device and a processor. The processor is configured to receive from the sensor assembly the measured time series. The processor is also configured to determine an objective metric for the subject based at least in part on deviations of the measured time series from a target trajectory for the subject. Furthermore, the processor is configured to present on the display an image based on the objective metric.
In a second set of embodiments, an automatic method for providing arm training includes receiving first data that indicates a target trajectory, for movement along a path, of a handle configured to be held at a distal end of an arm of a subject. The method also includes, operating a sensor configured to provide a measured time series comprising a plurality of handle location measurements at a corresponding plurality of measured time instances. Still further, the method includes determining an objective metric for the subject based at least in part on deviations of the measured time series from the target trajectory. Even further, the method includes presenting on a display an image based on the objective metric.
In some embodiments of the second set, the first data further indicates a second target trajectory for movement of a different second handle configured to be held at a distal end of a different second arm of the subject. The sensor is further configured to provide a different second measured time series comprising a plurality of second handle location measurements at a second corresponding plurality of measured time instances. The step of determining the objective metric further includes determining the objective metric based at least in part on second deviations of the second measured time series from the second target trajectory.
In some embodiments of the second set, the first data also indicates a tolerance slot bracketing the target trajectory. The step of determining the objective metric includes determining the objective metric based at least in part on deviations of the measured time series from the tolerance slot bracketing the target trajectory. In some of these embodiments, the method also includes operating a drive mechanically connected to the handle to reduce deviations of the measured time series outside the tolerance slot bracketing the trajectory. In some of these embodiments, the step of determining the objective metric includes determining the objective metric based at least in part on said operating the drive.
In some embodiments of the second set, the method includes modifying the target trajectory based at least in part on the objective metric. In some embodiments of the second set, the target trajectory is a curve of minimum jerk for a path of specified range and a cycle of specified period. In some of these embodiments, the method still further includes modifying, based at least in part on the objective metric, at least one of the specified range or the specified period.
In other sets of embodiments, a non-transitory computer-readable medium, or an apparatus, is configured to perform one or more steps of the above method.
Still other aspects, features, and advantages are readily apparent from the following detailed description, simply by illustrating a number of particular embodiments and implementations, including the best mode contemplated for carrying out the invention. Other embodiments are also capable of other and different features and advantages, and its several details can be modified in various obvious respects, all without departing from the spirit and scope of the invention. Accordingly, the drawings and description are to be regarded as illustrative in nature, and not as restrictive.
Embodiments are illustrated by way of example, and not by way of limitation, in the figures of the accompanying drawings in which like reference numerals refer to similar elements and in which:
A method and apparatus are described for automated arm training. In the following description, for the purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the present invention. It will be apparent, however, to one skilled in the art that the present invention may be practiced without these specific details. In other instances, well-known structures and devices are shown in block diagram form in order to avoid unnecessarily obscuring the present invention.
Some embodiments of the invention are described below in the context of bilateral synchronized assisted repeated movements of a subject's arms, for human subject such as a stroke survivor. However, the invention is not limited to this context. In other embodiments automated single arm training is provided, or arm extension/retraction training is combined with training of arm rotation (wrist, elbow or shoulder, or some combination) for injured or diseased or normal animals, such as apes, or humans. Other embodiments include resistance training (using a position rather than slot controller) for strength related impairments as well as active assistance training (using a position rather than slot controller) for passive stretching exercises in the context of muscle conditioning.
The path assembly 120 includes a mechanical path, such as rail 122 and a stop 124, for each arm of the subject. While the path defined by rail 122 depicted in
When the handle is moved by a subject, the movement of the handle along the path and back to its starting part is called a cycle. A first, or extension, half-cycle is said to begin when the handle 132 is closest to the subject who is propped against the chest support and to end when the handle is farthest from the subject who is still propped against the chest support. A second, or contraction, half-cycle is said to begin when the handle 132 is farthest from the subject who is propped again the chest support and ends when the handle is closest to the subject who is still propped against the chest support. The distance to the handle position at the beginning of the first half cycle and the distance to the handle position at the end of the first half cycle is the cycle range, the distance between those two positions is the cycle extent. The time it takes to complete a half cycle is the half cycle duration. The time it takes to complete both half cycles is the cycle period. A trajectory is a trace of the time dependent position of a handle over one cycle. A target trajectory is a trace of the time dependent position of a handle over one cycle that is desired for a particular patient for therapeutic results; and can be defined by a curve type, a cycle period (or two equal or unequal half-cycle durations) and a cycle extent (or range). In some embodiments, the curve type is a smooth curve, such as a minimum jerk curve. In some embodiments, the curve type is a curve fit to the trajectories of many healthy persons serving as subjects for the system 100, which may be normalized by extent and period.
In some embodiments, the rail 122 is a bar or beam around which the stops 124 and at least a shoe portion of the handle 132, or both, fits. In some embodiments, the rail is an elongated groove or recess in a plank into which the stop 124 or the shoe of the handle 132, or both, fits. In general, the handle 132 is constrained to follow a certain mechanical path such as defined by rail 122. In some embodiments, the system 100 is used for only one arm of the subject and includes only one rail 122 and stop 124. For example, in some embodiments, a rail 122 for an arm not included in training is removed from a modular arrangement of the system 100.
The handle drive assembly 130 includes at least a sensor 131 and the handle 132 that is configured to interface with the path such as rail 122, as described above. Each handle 132 is also configured to be grasped by a hand or prosthetic at a distal end of a corresponding arm of the subject. In various embodiments, the handle 132 has an angled grip that is configured by form or adjustment for grasping by the hand or prosthetic of the subject to cause the arm of the subject to assume a preferred orientation.
The sensor 131 is configured to determine the position of the handle 132 along the path, such as along rail 122. In some embodiments, one sensor 131 is configured to detect the positions of both handles 132 on their corresponding paths, such as rails 122. In some embodiments, there is a separate dedicated sensor 131 for each handle 132, as depicted in
In some embodiments, which provide an automated assist to a subject, the system 100 includes an impedance drive 134 connected by a different or the same mechanical linkage 138 to the handle 132. The impedance drive 134 includes a motor 135 and an actuator 136; and is under the control of computer system 150 via communication links 151 and provides feedback to the computer system 150 by the same or different communications links 151. The motor 135 provides movement; and the actuator 136 transfers movement from the motor to a force applied along the mechanical linkage 138 to the handle 132. The actuator 136 also serves to transfer force supplied by the subject to a signal that is transferred to computer system 150, e.g., for back driving the impedance drive 134. In various embodiments, each impedance drive 134 has low intrinsic end-point impedance, low inertia (0.6 to 1.5 kilograms), low friction (0.5 to 1.1 Newtons), and is capable of producing a predetermined range of forces (0 to 45 Newtons) and impedances (0 to 2 Newtons/millimeter). In an example embodiment, there may be two 16-bit virtual absolute encoders for position and velocity measurements (higher torques can be produced for limited periods of time). There may also be DC-tachometers with a sensitivity of 1.8 V/rad/sec for redundant velocity sensing. A six degrees-of-freedom force sensor may be mounted on the robot end-effector (e.g. handles 132).
In various embodiments, the handle drive assembly 130 is configured to be imperceptible, or barely-perceptible, to the subject when the actual position of handle 132 is within tolerance of a target trajectory for movement of the handle by the subject. Simple Impedance Control (SIC) renders a device a passive physical system, hence making it immune to contact instability. SIC also explicitly modulates the end-effector mechanical impedance rather than position (or force), which for the patient, translates to “robot feel”. Coupled with highly back-drivable actuators (gearless, linear screw drives rendering negligible stiction), a robot can be made to “get out of the way” when desired, and the patient does not expend effort to learn robot dynamics, which is the case for many commercial robotic arm trainers. SIC, along with a damping controller, is utilized as a one dimensional (1-D) moving box or “slot” controller, in which a virtual slot is created around each point of a virtual reference, called herein a target trajectory. In some embodiment the virtual reference is a minimum jerk trajectory that has been experimentally verified.
The target trajectory can be for one arm, or for both arms moving together (in phase), or for both arms moving in opposition (180 degrees out of phase), or for any other coordinated movement of two arms (out of phase by different amounts). Such imperceptible, or barely-perceptible force is accomplished by allowing the subject to be able to backdrive the impedance drive 134. When the impedance drive 134 provides reduced or no assistance, the subject applies force to exert the control on the handle 132; and, this may accelerate therapeutic progress. In some embodiments, the impedance-drive 134 is configured to apply viscous cushioning when the actual position of the handle 132 is within the tolerance of the target trajectory. This offers an advantage in possibly inducing fine motor skill recovery in the subject.
In various embodiments, when the subject's movements are within the slot (e.g., the subject is backdriving the motor-actuator assembly), there are no assistive or resistive (viscous) forces applied. In contrast, when the subject's movements are outside the slot, the Simple Impedance Controller (SIC) provides safe and gentle “push” (if the subject is “behind” the lower edge of slot) or “pull” (if the subject is “ahead” of the upper edge of slot) to the handle 132/subject. Safety is assured by using SIC (consisting of a virtual, programmable stiffness and programmable damping that define the commanded controller impedance) since it renders the coupled system passive at the port of interaction; gentle behavior is assured by using backdrivable hardware. In other words, a force field is created around the dynamic slot. The magnitude and phase of this force field are dependent on the controller parameters. The slot (or moving box) control strategy is versatile in that, arbitrary control laws within and/or outside the slot can be programmed. For example, in some embodiments one may choose to have the device provide viscous damping if the subject's movements are outside the slot, and no assistance inside the slot. Such a strategy would convert the “assist as-needed” functionality to a “resist as-needed” strategy for strengthening applications.
A virtual tolerance slot brackets the virtual target trajectory. The top and bottom edges of the slot move continuously such that they pseudo-collapse at the end of the movement. At any given time, if the actual (hand) position is outside the slot, assistance is provided in accordance with SIC dynamics—the magnitude depends on both the controller gains and the positional and velocity error, while the directionality (“push” or “pull”) depends only on the latter. Inside the slot however (“on task”), the hand experiences no positional assistance; instead, it experiences viscous cushioning in accordance with the dynamics of a damping controller. This strategy provides two critical benefits: 1. Spatial Autonomy, wherein the patient receives (or not) assistance if the movement is conformant with the needs of the task (or not); and 2. Temporal Autonomy, wherein due to the continually diminishing height of the slot, the amount of spatial autonomy reduces over time in order to “hold” the gains made over the course of the movement. Mechanistic studies utilizing the MIT-Manus in chronic stroke show that recovery in arm motor control occurs via the blending of sub-movements, eventually resulting in a continuous blended movement well modeled by a minimum jerk trajectory. Thus, when the actual position of handle 132 is outside of the tolerance of the target trajectory, the impedance drive 134 is configured to exert corrective force on the handle 132, thus “assisting” the subject. For example, the arm training module 156 is further configured as a simple impedance controller (SIC) with a programmable reference position (target trajectory), proportional gain (controllable torsional stiffness), and a derivative gain (controllable torsional damping in parallel with the stiffness).
Thus, in some embodiments, the system 100 includes a handle drive assembly 130 secured to the handle 132 and configured to apply position-correcting force to the handle 132 when the measured time series indicates an actual position of the handle which deviates from target trajectory for the handle by a predetermined tolerance, and is configured to apply no position-correcting force to the handle 132 so long as the actual position of the handle 132 is within the predetermined tolerance. In addition, as described above, in some embodiments, the handle drive assembly 130 is configured to be backdriven by the handle 132 only when the position of the handle 132 is within the predetermined tolerance. Further, in some embodiments, as described above, the handle drive assembly 130 is configured to apply viscous cushioning to the handle so long as the actual position of the handle 132 is within the predetermined tolerance.
The hardware components, including frame 110, path assembly 120, and handle drive assembly 130, together constitute a therapeutic device, as that term is used herein. In a modular version of the therapeutic device, the handle 132, impedance drive 134 or sensor 131, or some combination, is configured to be separated from the mechanical path, such as rail 122, or the path assembly from the frame 110, or some combination, for separate storage, shipment, repair or replacement, or some combination of reasons.
The computer system 150 includes a display 154, a data structure 152, and an arm training module 156 configured to perform one or more steps of a method described in more detail below with reference to a flow diagram depicted in
In various embodiments, the arm training module 156 is configured to automatically receive first data from a caregiver, either directly through a graphical user interface (GUI) or other input device, or indirectly from data structure 152, at least about a target trajectory for a particular subject. Furthermore, the based on the movements detected by sensors 131 the training module is configured to produce one or more objective metrics of subject performance relative to the target trajectory; and, optionally record the data or objective metric or both in data structure 152. In some embodiments, the arm training module 156 is also configured to control the impedance drives 134, if any, based on subject performance. In addition, the arm training module 156 is also configured to provide feedback or motivation or both to the subject through display 154, and to provide at least one objective metric to a caregiver for the subject, either via the display 154 or by a message transmitted to a device used by the caregiver, and optionally to update the target trajectory based on subject performance. The caregiver is then enabled to treat the subject based on the at least one objective metric. In some embodiments, the treatment includes prescribing a number of sessions on the system 100 for the subject and one or more adjustments to the target trajectory for one or more such sessions. Thus, the system includes a processor configured to receive from the sensor assembly the measured time series; determine an objective metric for the subject based at least in part on deviations of the measured time series from a target trajectory for the subject; and, present on the display an image based on the objective metric.
For example, in some embodiments, in addition to auditory cueing, visual cueing can be equally or more effective, especially in providing performance-based patient feedback for motivation and goal-setting, which are fundamental to motor learning. Visual tasks e.g., video games, are integrated with rehabilitation techniques to also tackle patient nonadherence. Physiological data suggests that game play can induce neuro-plastic reorganization that leads to long-term retention and transfer of skill. Studies have been conducted to evaluate the efficacy of incorporating a visual task to aid neurorehabilitation, robot-assisted or otherwise. There is evidence suggesting that key factors in game design, including choice, reward, and goals, lead to increased engagement. For example, early clinical trials with an ankle training robot, Anklebot, consisted of subjects playing a “racer” video game (described in more detail below with reference to
Each subject record 162 includes a subject identifier (ID) field 164, a subject status field 166, a subject goal field 168, and a session record 170 (solid outline) among zero or more other session records for other sessions by this subject indicated by ellipsis inside subject record 162 and outside session record 170. The subject ID field 164 holds data that indicates a unique identifier for the subject, such as government issued ID like a Social Security Number (SSN), a name and birthdate, a unique or random number generated by a clinic or by system 100 itself, and used as a key in a secured database for storage of private or encrypted information. The subject status field 166 holds data that indicates current status of the subject with respect to the use of the system 100, such as a current trajectory for one or both arms or a current or average score for an objective metric achieved over one or more current or previous sessions, or both. The subject goal field 168 holds data that indicates a trajectory or score or both for the subject to complete therapy, such as a target trajectory with its corresponding curve type, range and durations.
Each session record 170 includes a session identifier (ID) field 171, a target spatial range field 172, a target cycle period field 173, a bilateral flag field 174, a target number of together cycles field 175, and a target number of opposed cycles field 176. Each session record 170 also includes a cycle record 180 (dashed outline) among zero or more other cycle records for other cycles for this session, indicated by ellipsis inside session record 170 and outside cycle record 180. The session ID field 171 holds data that indicates a unique identifier for the session, at least for the current subject, such as a session sequence number. The target spatial range field 172 holds data that indicates the cycle range or extent or curve type or some combination for a target trajectory for the session; and, the target cycle period field 173 holds data that indicates a period or first half cycle duration or second half cycle duration or some combination for the target trajectory for the session. In some embodiments, either of field 172 or field 173 also holds data that indicates not only the target trajectory values, but a tolerance slot in space or time for differences in the actual trajectory of a handle that do not invoke a scoring penalty or robotic assist or both.
The bilateral flag holds data that indicates whether the training is for one arm (e.g., flag bit value equal to 1) or both arms (e.g., flag bit value equal to 0). In some embodiments, the bilateral flag is two bits, e.g., the first bit indicating whether the left arm is included (0→no, 1→yes) and the second bit indicating whether the right arm is included (0→no, 1→yes). The target number of together cycles field 175 holds data that indicates how many cycles are in the session (either planned or completed). If the bilateral flag indicates both arms are undergoing training, then the number indicated in field 175 is the number of cycles in which the arms are moving together (in phase). The target number of opposed cycles field 176 holds data that indicates how many cycles in the session (either planned or completed) have the arms moving in opposite directions if the bilateral flag indicates both arms are undergoing training. In some embodiments, the out-of-phase phase difference is understood to be 180 degrees as arms move opposite to each other. In other embodiments, the field 176 also holds data that specifies a phase difference and the number of cycles, for each of one or more different out-of-phase trajectories. If the bilateral flag indicates only one arm is undergoing training then, field 176 may be omitted.
Each cycle record 180 holds data collected by sensor 131 during one cycle of the session, which is used to determine various metrics or target trajectories or some combination. The cycle ID field 171 holds data that indicates a unique identifier for the cycle or half-cycle, at least for the current session, such as a half cycle sequence number. The left slot parameters field 182 holds data that indicates an envelope of trajectories for the subject's left arm that do not invoke a scoring penalty or robotic assist or both. For example, the field 182 holds data that indicates a distance ahead of, or behind, the target trajectory at a particular instant of the current cycle that will not invoke a scoring penalty or robotic assist or both. In some embodiments, the field 182 also includes current values for the target trajectory that may be different from the session target trajectory indicated in session fields 172 or 173 or both. Similarly, the right slot parameters field 183 holds data that indicates an envelope of trajectories for the subject's right arm that do not invoke a scoring penalty or robotic assist or both. In some embodiments, at one or more cycles, differences from session target and slot trajectories are not allowed; and, then fields 182 or 183 or both can be omitted.
Left half-cycle start time field 184 holds data, such as a timestamp, that indicates an instant when the left handle begins to be moved by the subject from the nearest or farthest point (for either the extension or contraction half-cycles, respectively). Left half-cycle start position field 185 holds data, such as a time of flight or number of rotations or distance, that indicates a location where the left handle begins to be moved by the subject from the nearest or farthest point (for either the extension or contraction half-cycles, respectively). Left half-cycle end time field 186 holds data, such as a timestamp, that indicates an instant when the left handle moved by the subject reaches the farthest or nearest point (for either the extension or contraction half-cycles, respectively). Left half-cycle end position field 187 holds data, such as a time of flight or number of rotations or distance, that indicates a location where the left handle moved by the subject reaches the farthest or nearest point (for either the extension or contraction half-cycles, respectively). The left slot excursion field 188 holds data that indicated occurrences of handle position outside the slot tolerances of the target trajectory, such as the number of positive excursions (too far ahead) or negative excursions (too far behind) or number of total excursions or average positive or negative excursion distance or time or variance thereof or some combination. In some embodiment, field 188 holds all the left handle measurements in the half-cycle as a time series.
Similarly, right half-cycle start time field 194 holds data, such as a timestamp, that indicates an instant when the right handle begins to be moved by the subject from the nearest or farthest point (for either the extension or contraction half-cycles, respectively, if arm motions are in phase or the opposite if arm motions are out of phase). Right half-cycle start position field 195 holds data, such as a time of flight or number of rotations or distance, that indicates a location where the right handle begins to be moved by the subject from the nearest or farthest point (for either the extension or contraction half-cycles, respectively, if in phase or the opposite if out of phase). Right half-cycle end time field 196 holds data, such as a timestamp, that indicates an instant when the right handle moved by the subject reaches the farthest or nearest point (for either the extension or contraction half-cycles, respectively, if in phase or the opposite if out of phase). Right half-cycle end position field 197 holds data, such as a time of flight or number of rotations or distance, that indicates a location where the right handle moved by the subject reaches the farthest or nearest point (for either the extension or contraction half-cycles, respectively, if in phase or the opposite if out of phase). The right slot excursion field 198 holds data that indicates occurrences of handle position outside the slot tolerances of the target trajectory, such as the number of positive excursions (too far ahead) or negative excursions (too far behind) or number of total excursions or average positive or negative excursion distance or time or variance thereof or some combination. In some embodiment, field 198 holds all the right handle measurements in the half cycle as a time series.
Timestamp temporal resolution on the order of one millisecond to 100 milliseconds is beneficial. Position resolution on the order of 0.1 millimeter to 10 millimeters is beneficial. If the left arm is not undergoing training, then any or all fields 182, 184, 185, 186, 187, 188 can be omitted. If the right arm is not undergoing training, then any or all fields 183, 194, 195, 196, 197, 198 can be omitted.
Although processes, equipment, and data structures are depicted in
In step 201, an operator, also called a user herein, such as a caregiver for a subject such as a human patient, is prompted to configure system 100 for the particular subject. For example, a graphical user interface is presented at display 154 with active area configured to allow the operator to enter data that indicates the subject and a goal for that subject, such as a target trajectory defined by a trajectory curve type, a cycle range, and a cycle period for one or both arms. For example, the data can entered using a pointing device 1916 or other input device 1913 depicted in
In some embodiments, the operator is also prompted to manually position any stops 124 for any paths such as rails 122, for one or both arms. The operator may also be prompted to position the subject against the chest support 112 and position the subject's arms so that a hand or prosthetic at a distal end of each arm to be trained grasps a corresponding handle 132. In some embodiments, positioning the subject's arm may involve strapping or otherwise attaching a handle to the distal end of the arm.
Indicating the target trajectory goal in field 168 includes defining a tolerance slot for the training, such that if the subject performance is within the tolerance slot, no scoring penalty or, if available, no robotic assistance will be called upon or applied.
The dot-dashed lines indicate the tolerance limits, called herein a tolerance slot, for the subject to proceed near the target trajectory without penalty or robotic assist. At a given time sample, the tolerance slot in the illustrated embodiment can be viewed as any position lying between a position one centimeter ahead of the target trajectory at that time sample and one centimeter behind the target trajectory at that time sample. Equivalently, at a given position, the tolerance slot in the illustrated embodiment can be viewed as any time occurring between a time 5 time samples before of the target trajectory at that position and a time 5 time samples after the target trajectory at that position. In other embodiments other tolerances are used. In some embodiments, the tolerances are small, such as about equal to the measurement error of the spatial sensor and timestamp clock; and almost any deviation will be an excursion outside the slot. In some embodiments, the range tolerance slot reduces to zero at the half-cycle range ends. Thus, in some embodiments, the first data further indicates a tolerance slot bracketing the target trajectory. The points plotted in
Returning to
In step 205, it is determined whether the session is complete. If so, control passes to step 239 described below. If not, such as when the session is beginning, control passes to step 207.
In step 207 data is received from the sensor during a cycle of the session, and the time and position of each handle is recorded, e.g., in fields 188 for the left handle and 198 for the right handle for every sample time in the half-cycle. Thus, the method includes operating a sensor configured to provide a measured time series comprising a plurality of handle location measurements at a corresponding plurality of measured time instances. In some embodiments, only excursions outside the tolerance slot are recorded, e.g., as the number of excursions or the number in each direction or the number and sum and sum of squares of excursions in each direction so that statistics of excursions can be computed. For example the number of excursions, the average excursion, the number of positive excursions, the average positive excursion, the number of negative excursions, the average negative excursion, the excursion variance can all be computed and any used as an objective metric. In addition, a weighted sum of any or all of these individual measures can provide a single-valued score that serves as the objective metric. Even if there are no excursions, the maximum range and shortest cycle period to accomplish excursion free motions can each be an objective measure or contribute to a single-valued score. Thus, the method includes determining an objective metric for the subject based at least in part on deviations of the measured time series from the target trajectory. In some embodiments, the sensor is further configured to provide a different second measured time series comprising a plurality of second handle location measurements at a second corresponding plurality of measured time instances. In these embodiments, determining the objective metric further comprises determining the objective metric based at least in part on second deviations of the second measured time series from the second target trajectory.
In step 209, the deviations from the target trajectory, such as the excursions outside the tolerance slot, along with the relationship to the target trajectory, are presented on display 154. Thus, in some embodiments, determining an objective metric involves determining the objective metric based at least in part on deviations of the measured time series from the tolerance slot bracketing the target trajectory. In some embodiments, the presentation is literally the target trajectory and the measured positions, such as depicted in
In
In
In
In
Returning to
If it is determined in step 211 that the handle is not within the tolerance slot, or that the handle is in the tolerance slot but there is no force applied by the subject, then control passes to step 221. In step 221, it is determined if the handle is outside and behind the tolerance slot. If so, then in step 223 the motor and actuator are engaged to assist the subject by accelerating the handle and to record the assist or evaluate the work done by the motor or both. Any method may be used to record this motor operation as an excursion, such as by incrementing a motor count by one, or measuring the work done by the motor, such as the power drawn by the motor, or counting the revolutions of the motor. This recorded motor operation contributes to an objective metric and or a single score in some embodiments. For example, using a novel metric to quantify step-by-step human versus motor (“robot”) contributions during assisted walking, it was shown that a robust ankle motor learning profile for reduced robot reliance was achieved concomitant with human autonomy for volitional swing clearance, collectively contributing to reduced reliance on assistive devices (85%) and increased daily free-living ambulatory activity (>20%), even six weeks after training ceases in adults with chronic stroke disability. Such a “co-robotic” metric is included in some embodiments. The co-robotic metric is applicable for any human-robot collaborative applications as long as a slot control strategy is employed. Basically, the metric counts the time spent inside the slot as a percent of total time for a complete (flexion or extension) movement. The rationale is that since the slot control will only assist the subject to reach the edges of the slot (if the subject's hand movements are outside the slot), the only way the subject can be inside the slot is through their own volitional effort. Hence, for most nondisabled individuals the co-robotic metric (also called Human Autonomy Index, or HAI) will be “high” (ideally 100%) while for impaired individuals the HAI will be “low” and will increase as recovery progresses. Control then passes to step 231, described below, to determine whether the current cycle is complete.
If it is determined in step 221 that the handle is not outside and behind the tolerance slot, then control passes to step 225. In step 225, it is determined if the handle is outside and ahead of the tolerance slot. If not, then control passes to step 231, described below, to determine whether the current cycle is complete. If so, then in step 227 the motor and actuator are engaged to assist the subject by decelerating the handle and to record the assist and work done by the motor. Any method may be used to record this motor operation as an excursion, such as by incrementing a motor count by one, or measuring the work done by the motor, such as the power drawn by the motor, or counting the revolutions of the motor or otherwise updating a “co-robotic” metric. This recorded motor operation contributes to an objective metric and or a single-valued score in some embodiments. Thus, by virtue of steps 223 and 227, the method includes operating a drive mechanically connected to the handle to reduce deviations of the measured time series outside the tolerance slot bracketing the trajectory; and, determining the objective metric involves determining the objective metric based at least in part on operating the drive. Control then passes to step 231, described below, to determine whether the current cycle is complete.
Returning to
If it is determined in step 231 that the cycle is complete, then control passes to step 233. In step 233, the cycle stats are recorded, e.g., in fields 184 through 198 of cycle record 180. In some embodiments, step 233 also accumulates scores or other objective metric for the last several cycles, such as the single score at least since the target trajectory was last updated in step 203.
In step 235, it is determined if there are sufficient numbers of new cycles (e.g., in a number range from 10 to 100 cycles) to warrant an update to one or more target trajectories. If not, control passes back to step 207 and following for the next cycle. However, if there are sufficient number of new cycles, such as 10 cycles or 20 cycles or 50 cycles or 100 cycles, then control passes to step 203 to update a target trajectory based on the performance over these multiple new cycles. In some embodiments, no update is done until all the cycles in one session are completed. Thus, in some embodiments, the method includes modifying the target trajectory based at least in part on the objective metric. In some embodiments this is done by modifying, based at least in part on the objective metric, at least one of the specified range or the specified duration for the target trajectory.
For example, in step 203, if the score or objective metric is consistently poor, as determined from the accumulated metric over several cycles, then the range for the target trajectory is advantageously reduced or the period is advantageously increased or the tolerance is advantageously increased, or some combination, for the next cycle or number of cycles or the next session. In some embodiments, the change in any parameter's value is a percentage change based on one or more objective metrics or the single score. Conversely, if the score or objective metric is consistently good or close to maximum, as determined from the accumulated metric over several cycles, then the range for the target trajectory is advantageously increased or the period is advantageously decreased or the tolerance is advantageously decreased, or some combination, for the next cycle or number of cycles or next session. In some embodiments the change in any parameter values is a percentage change for the difference between the current target trajectory and the subject goal target trajectory stored in field 168 for the subject. In some embodiments, the percentage is based on one or more objective metrics or the single-valued score. The update is recorded in one or more fields of data structure 152. For example, in some embodiments, the updated range or period or tolerance is recorded in fields 182 or 183 for the next cycle, or fields 172 and 173 for the remainder of the session, or the field 166 for the next session for this subject.
If it is determined in step 205 that the session is complete, then in step 239 the final score for the session and any visual game reward is presented on the display 154 in video or auditory form or both. In some embodiments, step 239 includes sending a message to the caregiver at a terminal associated with that caregiver, the final score or any changes to the current target trajectory for the subject, or some combination. As a result, in some embodiments, the caregiver will prescribe further treatment for the subject. The process then ends.
An advantage of various embodiments is greater patient throughput, because PTs can initially determine the patient impairment-specific robotic settings (such as curve type, cycle range and cycle period) then let the robot “guide” the training. Since adaptive robotics makes semi-automated training possible, a single PT can provide therapy for multiple patients at the same time; compared to the 1:1 PT to patient conventional training setup. Another advantage of various embodiments is improved outcomes, by focusing on task specific training with adaptive robotic algorithms. This allows a neoplastic response allowing for neural recovery and improved reaching functions beyond usual care. Another advantage of various embodiments is providing objective robotic measures, enabling PTs to systematically guide therapy and quantitatively document patient improvements. Currently, functional recovery can only be measured either in expensive biomechanics labs or by therapists using instruments and testing protocols with limited and subjective output, such as goniometers. In contrast, various embodiments provide quantitative measures of function at the point-of-care. By adding objective measures to current clinical outcomes, clinics have adequate data to justify insurance extensions and treat patients beyond the usual cap. Another advantage of various embodiments is giving clinics that do regularly treat stroke the tools needed to effectively address neurologically impaired function in former patients and creating a new tool for clinics that do not regularly treat neurologically impaired patients a simple cost-effective method to draw on a new patient population. Also, by documenting the use of rehabilitation robotics, such facilities can obtain Commission on Accreditation of Rehabilitation Facilities (CARF International) credit for exemplar stroke care. Patients, providers, and payers recognize CARF accreditation as a demonstration of internationally accepted superior standards.
The impedance-controlled assembly 306 includes a sensor assembly 310 configured to track a position of the handle 302 along the path 304; and a handle drive assembly 312 secured to the handle 302. The impedance-controlled assembly 306 is configured to apply a position-correcting force to the handle 302 when an actual position 320 of the handle 302 deviates from an expected position 322 of the handle 302 by a tolerance 324. The tolerance may be predetermined and/or adjustable. The position-correcting force is configured to change a speed of the handle 302 and move the handle 302 toward the expected position 322. The position-correction force is also configured to apply no force to the handle 302 so long as the actual position 320 of the handle 302 is within the tolerance 324. As shown in
In an example embodiment, the impedance-controlled assembly 306 is configured to move the expected position 322 over time to form a target trajectory along the path. The subject would then move the handle 302 over time in an effort to keep the actual position 320 of the handle 302 as close as possible to the expected position 322. As above, if the actual position 320 is within the tolerance 324 of the expected position 322, the impedance-controlled assembly 306 will exert no position-correcting force. If the actual position 320 is ahead or behind the expected position 322, the impedance-controlled assembly 306 will resist the subject's efforts and may be further configured to push the handle 302 toward the expected position 322 via a position-correcting force.
In a therapeutic environment, the subject's ability to remain within the tolerance 324 should improve over time. The magnitude of the tolerance can be relatively large to start and can decrease as the subject's ability increases. The subject's ability to stay within the tolerance 324 can be tracked and the improvement quantified. As noted above, the actual position 320, the expected position 322, and the tolerance 324 can be visually represented on the display 154. Further, these can be incorporated into a video game where the object is for the subject viewing the display 154 to keep the visually represented actual position within the visually represented tolerance of the visually represented expected position.
In an example embodiment, the impedance-controlled assembly 306 is configured to be imperceptible, or barely-perceptible to the subject when the actual position 320 is within the tolerance 324. This involves allowing the subject to be able to backdrive the impedance-controlled assembly 306. In contrast to when the actual position 320 is outside of the tolerance 324 and the impedance-controlled assembly 306 exerts corrective force on the handle 302, when the actual position 320 is within the tolerance 324, it is expected that the subject is to be in control of the position of the handle 302. Were this not the case, when inside the tolerance 324, a subject would still receive assistance from the impedance-controlled assembly 306. This may slow therapeutic progress. When the impedance-controlled assembly 306 provides reduced or no assistance, the subject will be forced to exert the control on the handle 302 and this may accelerate therapeutic progress.
Optionally, instead of providing no force, in an example embodiment, the impedance-controlled assembly 306 is configured to apply viscous cushioning when the actual position 320 is within the tolerance 324. Viscous cushioning cushions motion of the handle 302 but does not change a direction of movement of the handle 302. A magnitude of the viscous cushioning may be adjustable.
Using an impedance control scheme makes it possible for the impedance-controlled assembly 306 to sense and attempt to conform to the subject's will when the handle 302 is within the tolerance 324. However, the amount of conformity, e.g. the amount of backdrivability, and associated reduction in assistance, can be increased by focusing on mechanical aspects of the impedance-controlled assembly 306 as is detailed below.
The therapeutic device 300 further includes a motor housing assembly 330, a shoulder assembly 332, a bearing support assembly 334, an actuated slide mount assembly 336 that includes the handle 302, a linkage such as an actuated shaft 338 (e.g. a lead screw) which actuates the actuated slide mount assembly 336, a guide path assembly 340, and a back cap assembly 342. The example embodiments of each of these assemblies are not meant to be limiting. In a broad sense, the therapeutic device can function as intended with a handle that can be moved along a path (whether or not the movement along the path is guided by a rail or the like), and a sensor system that can track a position of the handle. For robot assist, the device also includes an effector (not necessarily impedance controlled) that can effect movement of the handle as disclosed above. The embodiments below represent various reductions to practice deemed respectively appropriate to reach certain performance, economy, and adaptability levels. However, the embodiments are not necessarily required for the therapeutic device to perform at difference performance, economic, and adaptability levels, and hence are not limiting. For example, the impedance-controlled assembly could be a fully articulated robotic arm and the movement of the handle could be fully unconstrained. In such an embodiment, the path could be any path imaginable and could be straight, curved, and any combination of both. In such an embodiment, the sensor system could be a type that is able to locate the handle anywhere in a field, such as an optical sensor or the like. The reductions chosen below represent what is understood by the inventors to be an efficient blend of function, cost, and modularity that is most likely to result in therapeutic benefit to the greatest number of those in need.
In an example embodiment, the linear actuator 1100 is a low stiction (<1 Newton-meter full torque range) linear actuator. An example suitable linear actuator 1100 is a Roh'Lix® linear actuator manufactured by Zero-Max of Plymouth, Minn. The Roh'Lix® linear actuator includes six rolling element ball bearings that contact the actuated shaft 338 at an angle. Thrust adjustment spring screw assemblies 1112 permit adjustment of an amount of spring force between two block halves 1114, 1116. The spring force determines an amount of thrust the Roh'Lix® linear actuator can accommodate before the Roh'Lix® linear actuator begins to slip on the actuated shaft 338. This creates an overload protection for the therapeutic device 300.
The slide mount frame 1102 includes handle receptacles (not visible) that receive handle protrusions 1120 and pins 1122 that readily releasably secure the handle protrusions 1120, and likewise the handle 302, in the slide mount frame 1102. The handle 302 may be configured to support the subject's hand in pure pronation, to support the hand 45° supinated from fully-prone, or to support the hand or to support the hand 90° supinated from fully-prone. The therapeutic device 300 may therefore include three or more handles 302, each having a different configuration. Alternately, the handle may be configured to rotate about one or more axis to accommodate a range of motion of the subject's hand during use. The pillow blocks 1104 allow the actuated slide mount assembly 336 to slide along rails of the guide path assembly 340 with minimal friction by receiving the rails within pillow block through holes 1130.
The sensor 1106 is part of the sensor assembly 310 and is configured to provide information about a position of the actuated slide mount assembly 336 relative to the path 304. The sensor 1106 is mounted to the slide mount frame 1102 and may be a relative or absolute encoder. In an example embodiment, the sensor 1106 is an absolute encoder configured to cooperate with an encoder track associated with the path 304.
As noted above, the amount of conformity, e.g. the amount of backdrivability, and associated reduction in assistance, can be increased by focusing on mechanical aspects of the impedance-controlled assembly 306 e.g. to reduce stiction. One mechanical aspect is an inertia of the actuated slide mount assembly 336. Namely, the lower the inertia, the more responsive the actuated slide mount assembly 336 is to lighter inputs by the subject's hand. In an example embodiment, a suitable inertia results when the actuated slide mount assembly 336 has a mass of 0.6 to 1.5 kilograms. A suitable friction for the actuated slide mount assembly 336 is 0.5 to 1.1 Newtons. In an example embodiment, the impedance-controlled assembly 306 generates from zero to 45 Newtons of force on the actuated slide mount assembly 336. Acceptable impedance is from zero to 2 Newtons/millimeter.
Another mechanical aspect can be in the mechanical connection within and between the impedance-controlled assembly 306 and the actuated slide mount assembly 336. For example, the mechanical connection between the impedance-controlled assembly 306 and the actuated slide mount assembly 336 could be traditional gears with intermeshing teeth. However, these assemblies have inherent backlash, greater friction, and resist being back-driven. The subject then encounters and overcomes these characteristics in order to backdrive the device. Hence, in those circumstances, the device is working against the subject specifically when the device is intended to be invisible to the subject. The Roh'Lix® linear actuator significantly reduces backlash, friction, and resistance to backdriving, and this enables the therapeutic device 300 to respond much more readily to the subtle inputs of a subject rehabilitating a paretic arm.
Thus, the illustrated embodiment is configured to provide an impedance-controlled, modular arm trainer interfaced with a “smart” visual task, further including the benefits of actuation-control, instrumentation, and visual feedback for use as a lowest-cost in-class training device for stroke rehabilitation.
A sequence of binary digits constitutes digital data that is used to represent a number or code for a character. A bus 1910 includes many parallel conductors of information so that information is transferred quickly among devices coupled to the bus 1910. One or more processors 1902 for processing information are coupled with the bus 1910. A processor 1902 performs a set of operations on information. The set of operations include bringing information in from the bus 1910 and placing information on the bus 1910. The set of operations also typically include comparing two or more units of information, shifting positions of units of information, and combining two or more units of information, such as by addition or multiplication. A sequence of operations to be executed by the processor 1902 constitutes computer instructions.
Computer system 1900 also includes a memory 1904 coupled to bus 1910. The memory 1904, such as a random access memory (RAM) or other dynamic storage device, stores information including computer instructions. Dynamic memory allows information stored therein to be changed by the computer system 1900. RAM allows a unit of information stored at a location called a memory address to be stored and retrieved independently of information at neighboring addresses. The memory 1904 is also used by the processor 1902 to store temporary values during execution of computer instructions. The computer system 1900 also includes a read only memory (ROM) 1906 or other static storage device coupled to the bus 1910 for storing static information, including instructions, that is not changed by the computer system 1900. Also coupled to bus 1910 is a non-volatile (persistent) storage device 1908, such as a magnetic disk or optical disk, for storing information, including instructions, that persists even when the computer system 1900 is turned off or otherwise loses power.
Information, including instructions, is provided to the bus 1910 for use by the processor from an external input device 1912, such as a keyboard containing alphanumeric keys operated by a human user, or a sensor. A sensor detects conditions in its vicinity and transforms those detections into signals compatible with the signals used to represent information in computer system 1900. Other external devices coupled to bus 1910, used primarily for interacting with humans, include a display device 1914, such as a cathode ray tube (CRT) or a liquid crystal display (LCD), for presenting images, and a pointing device 1916, such as a mouse or a trackball or cursor direction keys, for controlling a position of a small cursor image presented on the display 1914 and issuing commands associated with graphical elements presented on the display 1914.
In the illustrated embodiment, special purpose hardware, such as an application specific integrated circuit (IC) 1920, is coupled to bus 1910. The special purpose hardware is configured to perform operations not performed by processor 1902 quickly enough for special purposes. Examples of application specific ICs include graphics accelerator cards for generating images for display 1914, cryptographic boards for encrypting and decrypting messages sent over a network, speech recognition, and interfaces to special external devices, such as robotic arms and medical scanning equipment that repeatedly perform some complex sequence of operations that are more efficiently implemented in hardware.
Computer system 1900 also includes one or more instances of a communications interface 1970 coupled to bus 1910. Communication interface 1970 provides a two-way communication coupling to a variety of external devices that operate with their own processors, such as printers, scanners and external disks. In general the coupling is with a network link 1978 that is connected to a local network 1980 to which a variety of external devices with their own processors are connected. For example, communication interface 1970 may be a parallel port or a serial port or a universal serial bus (USB) port on a personal computer. In some embodiments, communications interface 1970 is an integrated services digital network (ISDN) card or a digital subscriber line (DSL) card or a telephone modem that provides an information communication connection to a corresponding type of telephone line. In some embodiments, a communication interface 1970 is a cable modem that converts signals on bus 1910 into signals for a communication connection over a coaxial cable or into optical signals for a communication connection over a fiber optic cable. As another example, communications interface 1970 may be a local area network (LAN) card to provide a data communication connection to a compatible LAN, such as Ethernet. Wireless links may also be implemented. Carrier waves, such as acoustic waves and electromagnetic waves, including radio, optical and infrared waves travel through space without wires or cables. Signals include man-made variations in amplitude, frequency, phase, polarization or other physical properties of carrier waves. For wireless links, the communications interface 1970 sends and receives electrical, acoustic or electromagnetic signals, including infrared and optical signals, that carry information streams, such as digital data.
The term computer-readable medium is used herein to refer to any medium that participates in providing information to processor 1902, including instructions for execution. Such a medium may take many forms, including, but not limited to, non-volatile media, volatile media and transmission media. Non-volatile media include, for example, optical or magnetic disks, such as storage device 1908. Volatile media include, for example, dynamic memory 1904. Transmission media include, for example, coaxial cables, copper wire, fiber optic cables, and waves that travel through space without wires or cables, such as acoustic waves and electromagnetic waves, including radio, optical and infrared waves. The term computer-readable storage medium is used herein to refer to any medium that participates in providing information to processor 1902, except for transmission media.
Common forms of computer-readable media include, for example, a floppy disk, a flexible disk, a hard disk, a magnetic tape, or any other magnetic medium, a compact disk ROM (CD-ROM), a digital video disk (DVD) or any other optical medium, punch cards, paper tape, or any other physical medium with patterns of holes, a RAM, a programmable ROM (PROM), an erasable PROM (EPROM), a FLASH-EPROM, or any other memory chip or cartridge, a carrier wave, or any other medium from which a computer can read. The term non-transitory computer-readable storage medium is used herein to refer to any medium that participates in providing information to processor 1902, except for carrier waves and other signals.
Logic encoded in one or more tangible media includes one or both of processor instructions on a computer-readable storage media and special purpose hardware, such as ASIC 1920.
Network link 1978 typically provides information communication through one or more networks to other devices that use or process the information. For example, network link 1978 may provide a connection through local network 1980 to a host computer 1982 or to equipment 1984 operated by an Internet Service Provider (ISP). ISP equipment 1984 in turn provides data communication services through the public, world-wide packet-switching communication network of networks now commonly referred to as the Internet 1990. A computer called a server 1992 connected to the Internet provides a service in response to information received over the Internet. For example, server 1992 provides information representing video data for presentation at display 1914.
The invention is related to the use of computer system 1900 for implementing the techniques described herein. According to one embodiment of the invention, those techniques are performed by computer system 1900 in response to processor 1902 executing one or more sequences of one or more instructions contained in memory 1904. Such instructions, also called software and program code, may be read into memory 1904 from another computer-readable medium such as storage device 1908. Execution of the sequences of instructions contained in memory 1904 causes processor 1902 to perform the method steps described herein. In alternative embodiments, hardware, such as application specific integrated circuit 1920, may be used in place of or in combination with software to implement the invention. Thus, embodiments of the invention are not limited to any specific combination of hardware and software.
The signals transmitted over network link 1978 and other networks through communications interface 1970, carry information to and from computer system 1900. Computer system 1900 can send and receive information, including program code, through the networks 1980, 1990 among others, through network link 1978 and communications interface 1970. In an example using the Internet 1990, a server 1992 transmits program code for a particular application, requested by a message sent from computer 1900, through Internet 1990, ISP equipment 1984, local network 1980 and communications interface 1970. The received code may be executed by processor 1902 as it is received, or may be stored in storage device 1908 or other non-volatile storage for later execution, or both. In this manner, computer system 1900 may obtain application program code in the form of a signal on a carrier wave.
Various forms of computer readable media may be involved in carrying one or more sequence of instructions or data or both to processor 1902 for execution. For example, instructions and data may initially be carried on a magnetic disk of a remote computer such as host 1982. The remote computer loads the instructions and data into its dynamic memory and sends the instructions and data over a telephone line using a modem. A modem local to the computer system 1900 receives the instructions and data on a telephone line and uses an infra-red transmitter to convert the instructions and data to a signal on an infra-red a carrier wave serving as the network link 1978. An infrared detector serving as communications interface 1970 receives the instructions and data carried in the infrared signal and places information representing the instructions and data onto bus 1910. Bus 1910 carries the information to memory 1904 from which processor 1902 retrieves and executes the instructions using some of the data sent with the instructions. The instructions and data received in memory 1904 may optionally be stored on storage device 1908, either before or after execution by the processor 1902.
In one embodiment, the chip set 2000 includes a communication mechanism such as a bus 2001 for passing information among the components of the chip set 2000. A processor 2003 has connectivity to the bus 2001 to execute instructions and process information stored in, for example, a memory 2005. The processor 2003 may include one or more processing cores with each core configured to perform independently. A multi-core processor enables multiprocessing within a single physical package. Examples of a multi-core processor include two, four, eight, or greater numbers of processing cores. Alternatively or in addition, the processor 2003 may include one or more microprocessors configured in tandem via the bus 2001 to enable independent execution of instructions, pipelining, and multithreading. The processor 2003 may also be accompanied with one or more specialized components to perform certain processing functions and tasks such as one or more digital signal processors (DSP) 2007, or one or more application-specific integrated circuits (ASIC) 2009. A DSP 2007 typically is configured to process real-world signals (e.g., sound) in real time independently of the processor 2003. Similarly, an ASIC 2009 can be configured to performed specialized functions not easily performed by a general purposed processor. Other specialized components to aid in performing the inventive functions described herein include one or more field programmable gate arrays (FPGA) (not shown), one or more controllers (not shown), or one or more other special-purpose computer chips.
The processor 2003 and accompanying components have connectivity to the memory 2005 via the bus 2001. The memory 2005 includes both dynamic memory (e.g., RAM, magnetic disk, writable optical disk, etc.) and static memory (e.g., ROM, CD-ROM, etc.) for storing executable instructions that when executed perform one or more steps of a method described herein. The memory 2005 also stores the data associated with or generated by the execution of one or more steps of the methods described herein.
Pertinent internal components of the telephone include a Main Control Unit (MCU) 2103, a Digital Signal Processor (DSP) 2105, and a receiver/transmitter unit including a microphone gain control unit and a speaker gain control unit. A main display unit 2107 provides a display to the user in support of various applications and mobile terminal functions that perform or support the steps as described herein. The display 2107 includes display circuitry configured to display at least a portion of a user interface of the mobile terminal (e.g., mobile telephone). Additionally, the display 2107 and display circuitry are configured to facilitate user control of at least some functions of the mobile terminal. An audio function circuitry 2109 includes a microphone 2111 and microphone amplifier that amplifies the speech signal output from the microphone 2111. The amplified speech signal output from the microphone 2111 is fed to a coder/decoder (CODEC) 2113.
A radio section 2115 amplifies power and converts frequency in order to communicate with a base station, which is included in a mobile communication system, via antenna 2117. The power amplifier (PA) 2119 and the transmitter/modulation circuitry are operationally responsive to the MCU 2103, with an output from the PA 2119 coupled to the duplexer 2121 or circulator or antenna switch, as known in the art. The PA 2119 also couples to a battery interface and power control unit 2120.
In use, a user of mobile terminal 2101 speaks into the microphone 2111 and his or her voice along with any detected background noise is converted into an analog voltage. The analog voltage is then converted into a digital signal through the Analog to Digital Converter (ADC) 2123. The control unit 2103 routes the digital signal into the DSP 2105 for processing therein, such as speech encoding, channel encoding, encrypting, and interleaving. In one embodiment, the processed voice signals are encoded, by units not separately shown, using a cellular transmission protocol such as enhanced data rates for global evolution (EDGE), general packet radio service (GPRS), global system for mobile communications (GSM), Internet protocol multimedia subsystem (IMS), universal mobile telecommunications system (UMTS), etc., as well as any other suitable wireless medium, e.g., microwave access (WiMAX), Long Term Evolution (LTE) networks, code division multiple access (CDMA), wideband code division multiple access (WCDMA), wireless fidelity (WiFi), satellite, and the like, or any combination thereof.
The encoded signals are then routed to an equalizer 2125 for compensation of any frequency-dependent impairments that occur during transmission though the air such as phase and amplitude distortion. After equalizing the bit stream, the modulator 2127 combines the signal with a RF signal generated in the RF interface 2129. The modulator 2127 generates a sine wave by way of frequency or phase modulation. In order to prepare the signal for transmission, an up-converter 2131 combines the sine wave output from the modulator 2127 with another sine wave generated by a synthesizer 2133 to achieve the desired frequency of transmission. The signal is then sent through a PA 2119 to increase the signal to an appropriate power level. In practical systems, the PA 2119 acts as a variable gain amplifier whose gain is controlled by the DSP 2105 from information received from a network base station. The signal is then filtered within the duplexer 2121 and optionally sent to an antenna coupler 2135 to match impedances to provide maximum power transfer. Finally, the signal is transmitted via antenna 2117 to a local base station. An automatic gain control (AGC) can be supplied to control the gain of the final stages of the receiver. The signals may be forwarded from there to a remote telephone which may be another cellular telephone, any other mobile phone or a land-line connected to a Public Switched Telephone Network (PSTN), or other telephony networks.
Voice signals transmitted to the mobile terminal 2101 are received via antenna 2117 and immediately amplified by a low noise amplifier (LNA) 2137. A down-converter 2139 lowers the carrier frequency while the demodulator 2141 strips away the RF leaving only a digital bit stream. The signal then goes through the equalizer 2125 and is processed by the DSP 2105. A Digital to Analog Converter (DAC) 2143 converts the signal and the resulting output is transmitted to the user through the speaker 2145, all under control of a Main Control Unit (MCU) 2103 which can be implemented as a Central Processing Unit (CPU) (not shown).
The MCU 2103 receives various signals including input signals from the keyboard 2147. The keyboard 2147 and/or the MCU 2103 in combination with other user input components (e.g., the microphone 2111) comprise a user interface circuitry for managing user input. The MCU 2103 runs a user interface software to facilitate user control of at least some functions of the mobile terminal 2101 as described herein. The MCU 2103 also delivers a display command and a switch command to the display 2107 and to the speech output switching controller, respectively. Further, the MCU 2103 exchanges information with the DSP 2105 and can access an optionally incorporated SIM card 2149 and a memory 2151. In addition, the MCU 2103 executes various control functions required of the terminal. The DSP 2105 may, depending upon the implementation, perform any of a variety of conventional digital processing functions on the voice signals. Additionally, DSP 2105 determines the background noise level of the local environment from the signals detected by microphone 2111 and sets the gain of microphone 2111 to a level selected to compensate for the natural tendency of the user of the mobile terminal 2101.
The CODEC 2113 includes the ADC 2123 and DAC 2143. The memory 2151 stores various data including call incoming tone data and is capable of storing other data including music data received via, e.g., the global Internet. The software module could reside in RAM memory, flash memory, registers, or any other form of writable storage medium known in the art. The memory device 2151 may be, but not limited to, a single memory, CD, DVD, ROM, RAM, EEPROM, optical storage, magnetic disk storage, flash memory storage, or any other non-volatile storage medium capable of storing digital data.
An optionally incorporated SIM card 2149 carries, for instance, important information, such as the cellular phone number, the carrier supplying service, subscription details, and security information. The SIM card 2149 serves primarily to identify the mobile terminal 2101 on a radio network. The card 2149 also contains a memory for storing a personal telephone number registry, text messages, and user specific mobile terminal settings.
In some embodiments, the mobile terminal 2101 includes a digital camera comprising an array of optical detectors, such as charge coupled device (CCD) array 2165. The output of the array is image data that is transferred to the MCU for further processing or storage in the memory 2151 or both. In the illustrated embodiment, the light impinges on the optical array through a lens 2163, such as a pin-hole lens or a material lens made of an optical grade glass or plastic material. In the illustrated embodiment, the mobile terminal 2101 includes a light source 2161, such as a LED to illuminate a subject for capture by the optical array, e.g., CCD 2165. The light source is powered by the battery interface and power control module 2120 and controlled by the MCU 2103 based on instructions stored or loaded into the MCU 2103.
In the foregoing specification, the invention has been described with reference to specific embodiments thereof. It will, however, be evident that various modifications and changes may be made thereto without departing from the broader spirit and scope of the invention. The specification and drawings are, accordingly, to be regarded in an illustrative rather than a restrictive sense. Throughout this specification and the claims, unless the context requires otherwise, the word “comprise” and its variations, such as “comprises” and “comprising,” will be understood to imply the inclusion of a stated item, element or step or group of items, elements or steps but not the exclusion of any other item, element or step or group of items, elements or steps. Furthermore, the indefinite article “a” or “an” is meant to indicate one or more of the item, element or step modified by the article.
Notwithstanding that the numerical ranges and parameters setting forth the broad scope are approximations, the numerical values set forth in specific non-limiting examples are reported as precisely as possible. Any numerical value, however, inherently contains certain errors necessarily resulting from the standard deviation found in their respective testing measurements at the time of this writing. Furthermore, unless otherwise clear from the context, a numerical value presented herein has an implied precision given by the least significant digit. Thus, a value 1.1 implies a value from 1.05 to 1.15. The term “about” is used to indicate a broader range centered on the given value, and unless otherwise clear from the context implies a broader range around the least significant digit, such as “about 1.1” implies a range from 1.0 to 1.2. If the least significant digit is unclear, then the term “about” implies a factor of two, e.g., “about X” implies a value in the range from 0.5× to 2×, for example, about 100 implies a value in a range from 50 to 200. Moreover, all ranges disclosed herein are to be understood to encompass any and all sub-ranges subsumed therein. For example, a range of “less than 10” for a positive only parameter can include any and all sub-ranges between (and including) the minimum value of zero and the maximum value of 10, that is, any and all sub-ranges having a minimum value of equal to or greater than zero and a maximum value of equal to or less than 10, e.g., 1 to 4.
This application is a National Stage entry and claims priority to PCT/US2020/065605, filed Dec. 17, 2020, which claims benefit of Provisional Application Ser. No. 62/949,128, filed Dec. 17, 2019, the entire contents of which are hereby incorporated by reference as if fully set forth herein, under 35 U.S.C. § 119(e).
| Filing Document | Filing Date | Country | Kind |
|---|---|---|---|
| PCT/US2020/065605 | 12/17/2020 | WO |
| Number | Date | Country | |
|---|---|---|---|
| 62949128 | Dec 2019 | US |
