The invention pertains generally to afferent feedback (also known as sensory feedback) for prosthetic limbs.
Myoelectric prosthetic limbs use EMG signals generated by muscles of the wearer to control actuation of the prosthesis. Typically, the wearer of the prosthetic limb must learn to control operation of the prosthesis by training another, remaining set of muscles. The effort required to do this is demanding, difficult and time consuming. However, a recently discovered technique called target muscle reinnervation may avoid the difficult retraining and using of prosthetic limbs. Nerves leading to the missing limb, which were once used to transmit signals controlling its movement and receiving sensory information, sometimes remain after amputation. These nerves can be used to control a myoelectric prosthetic limb by anastomizing them to muscles no longer being used. When the wearer thinks about moving the missing limb, the reinnvervated nerves actually cause remaining muscles to twitch, which generates the EMG signal.
The process of reinnervating nerves from the missing limb also leads to reinnervation of cutaneous afferents and, possibly, kinesthetic afferents (muscle spindles and golgi tendon organs). Because the same nerve reinnervates both muscles and afferents, collocation is achieved naturally. Targeted reinnervation thus also provides the advantage of collocated muscles and afferents.
There have been attempts to convey sensory information from distal parts of a prosthetic to proximal parts of the human body, but none in connection with targeted reinnervation. Examples of feedback include mechanical conveyance of force, such as shown in U.S. Pat. No. 3,751,733, hydraulic conveyance of force such as shown in U.S. Pat. No. 4,808,187, and electromechanical conveyance of force, such as shown in U.S. Pat. No. 5,888,213. Nevertheless, most prosthetic limbs lack significant afferent feedback. For example, prosthetic hands lack the sense of touch. Wearers of prosthetic hands therefore have no direct way to sense tactile quantities such as grip pressure, surface roughness, surface warmth/coolness, and so on.
The invention concerns generally the feedback of tactile or haptic sensations of various forms from a distal location—for example, on a prosthetic—to locations on the wearer, overcoming one or more of the disadvantages of the prior art. Tactile or haptic sensations include, for example, one or more of temperature, vibration, and shearing and normal forces sensed by the prosthetic.
The transmitted sensory information is preferably presented in the same mode as it was sensed. For example, pressure forces that are measured distally are presented as pressure forces applied to the skin surface of the wearer and sensed vibration is presented as vibration. The sensory information as presented to the wearer may be applied at any sensate surface of the wearer's body. It may be presented inside the socket which attaches the prosthetic to the body. The term socket refers to the attachment system of the prosthetic, regardless of its shape. For instance a vest-like socket might be used to attach a whole-arm prosthetic. The sensory information might also be presented to the wearer at some other location, not in the socket.
One advantage of the invention is that it is well-suited for use with targeted reinnervation. Placing a stimulator (also referred to as a tactor) next to skin reinnervated with nerves from a missing limb, allows the wearer the possibility of experiencing the afferent sensations from a prosthetic as if they originate in the corresponding part of the missing limb itself.
The teachings of various aspects of the invention, in their preferred form, are explained in context of exemplary implementations of detectors (also referred to as receptors) for detecting tactile or haptic sensations and stimulators for transmitting the sensations to the skin of a person. The invention is not, however, limited to the details of these examples. The boundaries of the invention are limited solely by the appended claims.
As described in more detail below, one exemplary implementation of a receptor or detector preferably includes a three-axis force sensor. The force sensor is, in the example, preferably implemented by strain gauges affixed to a flexure. However, other force sensing mechanisms could be substituted. Inclusion of an accelerometer, for example, a multi-axis MEMS accelerometer, offers additional advantages. It is further preferred that the accelerometer is closely coupled to a light (low mass) but rigid or hard structure which serves the purpose of a fingernail or mechanically of a stylus, and aids in the exploration and detection of vibrational signals for a surface being touched or explored with the detector. The receptor or detector also preferably possesses an anthropomorphic shape. For example, if it is to be used in prosthetic hand, it preferably has the shape of a fingertip.
A stimulator, as shown in the exemplary implementations, preferably possesses a flat aspect and an activated tip moving largely perpendicular to the flat aspect when contacting the skin of the wearer of the prosthetic. The flat aspect is advantageous, and thus preferred, because it allows mounting the stimulator low and close to the socket, so that it is more comfortable to the wearer and protrudes little. It may be recessed into the socket. The stimulator preferably moves along at least two axes, one perpendicular to the skin surface, which may be denoted pressure, and one parallel to the skin surface, which may be denoted shear. Another axis of shear might could be added. One advantage of this example is that it can convey vibrations sensed by the detector to the skin. In the illustrated embodiments, two axes of motion are driven by two electric motors through gear trains and a linkage mechanism. Although the illustrated embodiment offers certain advantages, other types of actuators could be substituted.
The stimulator tip may also further adapted to include a heating and/or cooling unit such as a Peltier device. It is also possible to add an actuator, for instance as a voice-coil actuator, to the tip, in order to deliver vibrations of a higher frequency than the motors and transmission can accommodate.
In the exemplary embodiments information from the receptor or detector is transmitted as electrical signals. Sensors in the receptor can include, for example, strain gauges, accelerometers, miniature microphones, and thermometers. There are many varieties of these sensors and many are suited to small scale, low power, robustness, and other conditions of use of a prosthetic. While other modes of transmission of the information from these sensors are possible, including mechanical cables or linkages, hydraulic or pneumatic tubes, RF/wireless, fiber-optic, etc., electrical signals conducted by wiring are presently preferred. The signals may be analog levels, or may be multiplexed or conveyed as a data stream. Electrical conveyance of the signals from the receptor to the stimulator affords the opportunity of signal processing. The signal processing may be used to create mapping between receptor signals and the stimulator actions to make best use of the range of sensations available at the stimulation site. Each type of sensation such as pressure, shear, or temperature may have an independent mapping. In a preferred embodiment the signals are compressed in dynamic range, limited in amplitude, dead-banded, bandpass filtered, and filtered in frequency according to an equalization curve.
In applications involving a socket worn on the chest, for example, the chest skin and muscle of the wearer moves to some extent relative to the socket. One example of a stimulator described below is mounted to a socket so that it can stay in contact with the wearer's chest despite relative motion of the socket and the chest. A flexible coupling allows mounting of larger parts of the stimulator, for example its motors, to the socket, while other parts remain in contact with the wearer's chest at appropriate levels of force.
Although the invention is used to advantage in providing both detection of sensory information and presentation of it to the wearer of a prosthetic, these functions may be separated and used singly, for instance in some cases a synthetic (e.g. computer generated) sensation might be presented to the wearer via the stimulator, without or in addition to sensations originating at the detector. Similarly, the detector might be used alone without the presentation of its outputs by a stimulator. The invention and/or various aspects of it could be implemented in other applications in which it is desired to detect, convey, and display tactile or haptic sensations.
a is a sketch illustrating basic system components of an afferent feedback system for a prosthetic limb.
b is an isometric view illustrating two possible methods of attaching a haptic stimulator to a socket.
c is a schematic diagram of a detector and stimulator.
a is an exploded view of an embodiment of a detector.
b is an exploded view of a second embodiment of a detector.
c is an isometric view of the detector of
a is an isometric view of an embodiment of a two (2) degree of freedom stimulator.
b is an isometric view of a linkage of the stimulator of
a is an isometric view of a second embodiment of a one (1) degree of freedom stimulator that can be mounted remotely.
b is a second embodiment of a linkage mechanism of the tactor embodiment illustrated in
c is an exploded view of the stimulator tip of
a is an isometric view of a second embodiment of a two (2) degree of freedom stimulator.
b is an exploded view of a the tactor illustrated in
In the following description, like numbers refer to like elements.
Referring to
Referring to
Referring now to
Signals from the force detectors and accelerometers are processed by signal processing circuitry 122 to generate output signals for driving preferably at least two motors 136 and 138. The signal processing circuitry acts as a controller for stimulator 123, and preferably receives feedback information from stimulator 123. The circuitry can be implemented in any way desired, and need not be dedicated to just processing signals from the detector. For example, the signal processing circuitry can be implemented using analog circuits, a digital signal processor, or combinations of the two. Furthermore, it can be incorporated in the detector or stimulator, distributed between them or other components.
The motors are indicated as being of stimulator 123, but may be mounted in a structure 140 separate from the stimulator head 142, as illustrated in
a details one exemplary embodiment of a detector. The base of the detector 201 includes a three-axis flexure 202 for a load cell, to which strain gauges (not shown) are adhered. In this embodiment three axes of force measurement are obtained, but other numbers are possible, more or fewer. In this embodiment the flexure is, advantageously, an inherent part of the detector base 201. However, a separately fabricated single or multi axis strain gauge can be substituted. Tip 203 is attached to the endpoint of flexure 202, by a pin 206 which engages a hole 207. Thus forces are transmitted reliably from the environment, via tip 203, to the flexure, and are measured by the strain gauges which are adhered to flexure 202. The strain gauges are not shown in the figure, nor are their wires, which exit via hole 208. Other known ways to measure forces, even multi-axis forces, such as optical measurements of deflection, magnetic measurements of deflection, or force sensitive conductive elastomers, can be used in place of the tips and flexure.
Tip 203 preferably includes other sensor and functional components. In a preferred embodiment the tip has the size and shape of a human fingertip and includes a hard stylus 205 which serves to elicit and transmit vibrations from a surface being explored by a wearer. Accelerometer 204 is attached to stylus 205 in order to best measure said vibrations as well as other accelerations due to contact and motion. Accelerometer 204 is, in the example, a two axis MEMS accelerometer in a preferred embodiment, but may have more or fewer active axes. A piezoelectric accelerometer may also be used, or other devices that are sensitive to vibration such as a magnetic pickup. Pins 209 and 210 serve as additional means of conveying vibration to the accelerometer.
b and 2c illustrate an alternative embodiment of a receptor. Sensing element 211 is a two-axis flexure to which strain gages 212 are bonded. More or fewer axis of force measurement can be obtained by different embodiments of the flexure. The base of the sensing element attaches to a specially modified distal phalanx 213 of the prosthetic finger by means of screws 214. Accelerometer 215 rigidly attaches to sensing beam 211 by means of bracket 216. Sensing beam and accelerometer are encased in the fingertip cap 217 that attaches to the sensing beam 211 by means of screw 218 simultaneously retaining bracket 216 Modified distal phalanx 213 and cap 217 together comprise an anthropomorphic shape of a finger tip 222. A hard, fingernail-like stylus 219 attaches to cap 217 by means of screw 220 Pin 221 may be used as an alternative to the stylus. Electrical signals of the sensing beam and the accelerometer are transmitted to flexible circuit 219 by means of spring loaded pins in the base of the sensing element 211. Not shown is a tail of the flexible circuit that runs the length of the articulated finger to the palm of a prosthetic hand for housing sensor electronics. Although not illustrated, a temperature sensor or a thermal conductivity sensor, or a combination of the two measuring a combined quantity, can be included in an alternative embodiment. Other tactile, thermal, pressure, vibration, acceleration, or other measurement devices or arrays of such devices could be incorporated into the detector of the present invention, as well.
a and 3b detail a preferred implementation of an exemplary stimulator. Base 301 attaches to pad 103 (see
Linkage 306 serves to convert the rotational motion of the outputs of the two gear trains into approximately translational motions of the head of the stimulator 307. Linkage 306 can be described as a 5-bar mechanism with a prismatic constraint that prevents the stimulator tip, which is offset from the linkage pivot point, from tipping uncontrollably. In other words the additional prismatic linkage constraints the stimulator tip to a known orientation. Other numbers of axes of motion, more or less closely translational motion, could be used, and these would require other linkages. In this preferred embodiment there are two axes of approximately translational motion, and these are a motion perpendicular to the skin surface which may be used to transmit a sensation of pressure, and a motion parallel to the skin surface which may be used to transmit a sensation of shear. Pin 308 and another pin not visible limit the range of motion of the linkage in order to prevent excessive excursions.
Vibrations are transmitted by rapid modulation of the motions of the stimulator head 307. Stimulator head 307 preferably also incorporates a one axis force sensor measuring normal pressure of the stimulator head against the skin surface of the wearer. Alternatively, a two-axis or three-axis force sensor, which would measure shear forces as well, could be employed. As a further alternative, if no force sensor is used, contact with the skin could be measured. This could be done by a variety of techniques, including by measuring electrical conductivity. Stimulator head 307 could be integrated with a pad (not shown) for measuring EMG potentials. It could also incorporate a thermal heater and/or cooler (not shown), such as a Peltier device, to convey sensations of temperature to the wearer.
Referring now primarily to
a illustrates, the interposition of a flexible shaft 503 that allows a stimulator to be divided into two parts, a stimulator actuator 501 and a stimulator endpart 505. In this example, the stimulator of
b further details a second embodiment of linkage 406. It is a 4-bar linkage. Arms 512 and 513 are rotatably attached to support structure 405 with parallel but non-coincident axes. Arms 512 and 513 are also rotatably attached to structure 511 with parallel but non-coincident axes. Thus structure 511 translates without rotation as arm 512 is turned by the output of gear train 403.
c further details stimulator head 514. It includes structure 511, previously described, a force sensor 521, a load distribution button 522, and a housing 523 attached by screw 524. The force sensor provided feedback to signal processors or other control circuitry for controlling the stimulator to absorb the amount of force actually being applied.
Referring to
The linkage in the illustrated example is a 6-bar linkage. The outputs of gear trains 602 drive upper links 607 and 610, which are rotatably attached to lower links 608 and 611. Force sensor 613 is rotatably attached to lower links 608 and 611. Excess freedoms of force sensor 613 are removed by a gear pair 609 and 612 which engage with one another. Gears 609 and 612 are rigidly (non-rotatably) attached to lower links 608 and 611. Contact foot 615 is attached to end structure 613 by pin 614.
Inputs 701 and 702 from the fingertip shear and pressure force sensors respectively, are filtered with a low pass filter 704, for example, a second order Butterworth filter with a 50 Hz cutoff. Feedback 703 from the stimulator pressure force sensor is also so filtered. The filtering can be done using analog or digital signals, or in software, using any known techniques. In this embodiment the fingertip shear force measurement 701, filtered as described, is multiplied by a gain 705 and combined with another axis of motion for pressure force display, originating in block 706. The output of block 706 is the difference between the fingertip pressure force measured 702 and the present stimulator pressure force measured 703, the difference constituting an error signal. The error signal is converted by impedance block 706 into a necessary corrective measurement. The two measurements are then converted by a linkage kinematics block 707 to motor rotation commands. The commands are conveyed to PD (proportional differential) controller block 708. PD controller 708 accepts also as inputs signals 709 and 710 indicating motor positions and their derivatives. The motor position signals are first multiplied by gear ratio blocks. The combinations are multiplied by gear ratio blocks 719 and 719b, and by motor constant blocks 720a and 720b, and thus become currents 721a and 721b with which to drive the motors.
Said other signals originate with the fingertip acceleration sensor, which provides a shear signal and a pressure signal. In a preferred embodiment these are processed differently, with fingertip shear acceleration signal 711 passing through a bandpass filter, for example a 2nd order Butterworth filter with bandpass frequencies 50 to 500 Hz, and then through a gain stage 712. Thus the high and low frequency components of motion of the stimulator originate separately in force sensors (for the low frequency components) and accelerometers (for the high frequency components). The resulting signal is then combined with fingertip pressure acceleration signal 713.
The fingertip pressure acceleration signal is, first, low-pass filtered by, for example, a 2nd order Butterworth filter with a cutoff frequency of 500 Hz, and then passed deadband filter 715, which has an adjustable threshold. It is then limited in magnitude by a limit filter 716 with an adjustable amplitude limit and by a contact gain factor 717. The signals are then combined, and converted into the motions needed by the axes of the two motors by linkage Jacobian calculations 718, which is informed by the linkage angles as derived from the motor angles 709 and 710. The output of the linkage Jacobian calculations, representing motor torques, are summed with the output of the PD controller 708. The combined motor torques, in the form of currents, are then passed to the motors as currents, as described previously.