Patent Application
20070038311
This invention pertains to the field of prosthetics. More particularly, this invention relates to a system and method for providing haptic feedback from external prostheses to enhance the functionality of such devices.
Improving the functionality of prostheses, such as artificial upper and lower limb prostheses, is a considerable challenge, especially for high-level amputations where the disability presented by the amputations is the greatest. In the United States during the period from 1988 to 1996 more than 100,000 people lost at least a part of an upper limb (thumb, finger, hand, wrist or transradial, elbow disarticulation, transhumeral, shoulder disarticulation or forequarter amputations) mostly as a result of trauma, dysvascularization or cancer. Lower limb amputations are even more ubiquitous with over 50,000 cases per year in the United States alone. While prosthetic devices can help people perform some daily activities, many upper and lower limb amputees still find that their prostheses have unsatisfactory functionality and do not use them. As a result, many prosthetic users choose not to wear a prosthesis at all.
Conventional prosthetic devices, including body powered and motorized hooks, hands, wrists, elbows, knees, feet, etc. are nevertheless used by many amputees in performing activities of daily living. Such prosthetic devices do not provide the full functionality of a natural limb. For example, conventional prostheses do not allow a user to feel the force or pressure applied by or to the prosthesis. As a result, conventional upper and lower limb prostheses do not give the user the psychological reassurance of sensing touch in the prostheses. Conventional hand prostheses also do not meet the practical needs of allowing a user to sense, without visually observing the prostheses, whether they are gripping an item, let alone whether they are holding it loosely or tightly. Thus, items held in a conventional prosthesis may be dropped because they are not held securely or they may be crushed due to the application of excessive gripping force. Conventional foot prostheses do not allow the user to sense pressure on the foot prostheses as the user walks. This adds to the difficulty of learning to use and then using such devices.
Currently, most powered artificial limbs are controlled using myoelectric signals from an antagonist pair of muscles in the amputated limb. This allows only a single form of motion to be performed at a time and is therefore unduly cumbersome. Furthermore, such devices currently provide no haptic feedback.
Although a limb is lost with an amputation, the control signals to the limb remain in the residual peripheral nerves. In recently developed upper limb prostheses, these control signals are tapped into, using nerve transfers that greatly improve the control and function of the prostheses. See Kuiken T A, Rymer W Z, Childress D S (1995), “The Hyper-reinnervation of Rat Skeletal Muscle,” Brain Res 676, 113-123; Kuiken T A, Stoykov, Popovic M, Lowery M and Taflove A (2001), “Finite Element Modeling of Electromagnetic Signal Propagation in a Phantom Arm,” IEEE Trans Neural Sys and Rehab Engr 9(4), 345-354; Kuiken T A T A, Lowery M M and Stoykov N S, “The Effect of Subcutaneous Fat on Myoelectric Signal Amplitude and Cross-Talk,” Prosthetics and Orthotics International 27, pp 48-54, 2003; Kuiken T A, Dumanian G A, Lipschuzt R D, Miller L A and Stubblefield K A, “Targeted Muscle Reinnervation for Improved Myoelectric Prosthesis Control,” Prosthetics and Orthotics International, 28(3) pp. 245-253, December 2004, the entirety of which are incorporated by reference. It has thus been demonstrated that it is possible to control prostheses using such nerve transfers. This involves denervating expendable regions of muscle in or near an amputated limb and transferring the residual peripheral nerve endings to these muscles. The nerves reinnervate these muscles. Then, the surface electromyograms (EMGs) from the nerve-muscle transfers are used as additional myoelectric control signals for an externally powered prosthesis. While these new prosthetic control techniques represent a very significant advance in the art, even such a highly articulated limb controlled by surface EMGs from the nerve transfers would be substantially improved if haptic feedback could be provided.
With the nerve transfer technique discussed above, the amputee's residual nerves are transferred onto “foreign” regions of muscle and cross-reinnervate these muscles. Using such nerve transfers for amputees takes advantage of the nerves' inherent motor programming so that the nerves simultaneously control physiologically appropriate functions in the prosthesis. The control of the artificial limb has been demonstrated successfully in several patients. They report targeted reinnervation control to be quicker and to have a more natural feel than with their prior conventional myoelectric prostheses. This reduces the conscious effort required by the amputee, making the prosthesis easier to use and more functional.
The nerve transfer control technique discussed above may be used with existing myoelectric technologies. Powered elbows, wrists and terminal devices are commercially available with circuitry allowing up to seven analog inputs (e.g. myoelectric signals) and four on/off input signals that provide the control of up to five motors. The nerve transfer technique enables better control of such complex prosthetic devices but still lacks the haptic feedback necessary for optimal human control.
For the nerve transfer control technique to be successful in amputees, multiple nerves must consistently reinnervate separate regions of muscle. In the past, muscle recovery after nerve transection has been inconsistent and often unsatisfactory. However, in order to address this issue, optimally large nerves containing many times the normal number of motoneurons are grafted onto the muscles thus “hyper-reinnervating” the muscles. Hyper-reinnervating muscle (grafting an excessive number of motoneurons onto a muscle) increases the likelihood that any given muscle fiber will be reinnervated and this improves muscle recovery. A related issue is containment of the reinnervation field. With the nerve transfer technique multiple nerves will be grafted onto different regions of a muscle, each with nerve reinnervating only the intended muscle region. Also, cross-talk is prevented from interfering with prosthesis operation by setting a threshold above background noise and the cross-talk from nearby muscles. The amputee must generate an EMG signal greater than the threshold to operate the prosthesis.
It is therefore an objective of this invention to provide a system and method for providing haptic feedback to an amputee using a prosthesis.
It is another object of the invention to enable a user of a prostheses to regain a sense of touch.
It is a further object of the present invention to provide an amputee using a prosthesis with a sense of force/pressure, temperature, vibration, texture, or sharp/dull edges at various locations on the prosthesis.
It is yet another object of the present invention to reinnervate an area of skin on an amputee's body with nerves that formerly provided sensory feedback from the part of a lost limb replicated by a prosthesis and to supply sensory input from the prosthesis to the reinnervated area.
It is a still further object of the present invention to provide an amputee wishing to use a prosthesis to grip an item with haptic feedback allowing the amputee to sense, control, and adjust the tightness or looseness of the grip.
Yet another object of the invention is to provide lower limb amputees with prostheses that enable the amputee to sense pressure on the prosthesis as he walks.
Still another object of the present invention is the provide an enhancement of systems using nerve transfers as control signals for powered prostheses in which the amputee is also supplied with haptic input from selected areas of the prostheses to areas of reinnervated skin.
These and other objects and advantages are achieved in the practice of the present invention as described below.
The present invention may be used with prostheses for any amputation that would benefit from haptic feedback. For example, it may be used with prostheses for transcarpal and higher upper limb amputations and partial foot and higher lower limb amputations. The sensory information may be any information that is available to nerve endings on the skin including force or pressure, texture, temperature, vibration and sharp/dull and edge sensations so long as appropriate corresponding nerves from the amputated limb can be relocated to a reinnervated skin area and accessed in that area.
The present invention relies on sensory nerve transfers. Motor nerve transfers as described above in the Background of the Invention are used to gain additional motor commands for operating a prosthesis by surgically moving residual limb nerves to muscles. This invention applies a similar concept to skin. Thus, nerve to an area is cut to denervate the skin and new sensory nerve fibers grow into the skin. When this skin is then touched, the person feels like they are being touched in the area that used to innervated by the transferred nerve. For example, the residual arm nerves of a person with a shoulder disarticulation amputation have been transferred to different sections of the chest muscles. The nerves to the skin over these nerve transfers were also cut. Then the sensation nerves to the hand reinnervated the chest skin. When this chest skin is touched, the person feels the touch in their missing hand. They feel light touch, graded pressure, hot, cold, sharp and dull—all as if it were in there missing hand. This reinnervated skin is thus serving as a mechanism to provide sensation to a missing body part. This is referred to in the context of the present invention as “transfer sensation.”
In the practice of the present invention, haptic feedback is provided from a prosthesis to these nerves that formerly served the missing natural limb to enhance the function and control of the prosthesis. According to one embodiment of the invention, a prosthesis is equipped with appropriate sensor(s) and the person using the prosthesis is fitted with one or more transducers that can selectively produce sensory feedback (a “sensory condition”) over one or more locations on the user's skin that have been reinnervated with nerves that formerly provided sensation to the amputated limb (e.g., hand or foot). Now touching this skin provides transfer sensation. When this skin is touched, it feels like the missing hand or foot is being touched. Sensors on the prosthesis will measure the interaction of the prosthesis and an object (e.g., pressure, texture, temperature) and an actuator over the reinnervated skin will apply an appropriate stimulation so the user “feels” what the prosthesis is touching as if it were their own hand or foot.
The sensor, transducer, and coupling therebetween and to the reinnervated skin (using nerves in the reinnervated skin that formerly provided sensation from the part of the lost limb replicated by the prosthesis) together comprise the haptic interface of the invention. For example, the interface may include a pressure sensor located at a tip of a motor-driven movable finger or toe of a prosthesis, where the pressure sensor is capable of detecting and producing an indication of the level of pressure applied to the finger or toe. The person using the prosthetic will be fitted with an actuator that selectively applies a force to the skin reinnervated with the nerves that formerly controlled the tip of the user's finger or the pad of the toe. The actuator is preferably of a type that can exert varying degrees of force on the appropriate location(s) of the reinnervated area corresponding to the pressure indication from the pressure sensor. Preferably the force applied by the actuator will be scaled down since the reinnervated skin typically will be substantially more sensitive or tender than the corresponding area of the natural limb where the pressure would have been sensed but for the events resulting in the amputation and placement of the prosthesis. Also, the pressure is typically better cushioned and more distributed in the natural limb area. The scaling may be linear or non-linear. For example, the pressure may be unscaled or scaled up at lower pressures and scaled down at higher pressures. The scaling where it occurs may be, for example, linear initially and then exponential.
The interface includes the sensor in the prosthetic limb and the actuator over the reinnervated skin and may also include a controller that receives a signal or other indication from the prosthesis sensor corresponding to the level of pressure, texture or temperature detected whereby the actuator exerts either a corresponding force, vibration or temperature or a scaled value on the appropriate location(s) of the skin reinnervated with nerves that formerly controlled the tip of the user's finger. As a result, the amputee senses the force, vibration or temperature through the nerve formerly associated with the lost natural limb in a way corresponding to the way it had been sensed in the now absent natural limb. This feedback makes the prosthetic feel more natural and satisfying and it helps the amputee to control the pressure applied by e.g., a hand prosthesis, or to operate e.g., a leg/foot prosthesis with the benefit of sensing pressure on the toes, ball and heel of the foot. For example, by providing such feedback from portions of some or all of the fingers of a prosthetic hand, the user will have a transfer sensation corresponding to the amount of gripping pressure a prosthetic hand is exerting on an object.
Various embodiments of the present invention provide apparatus and methods for providing an amputee with haptic sensations from a prosthetic device. Thus, the prosthetic sensors can be pressure sensors, vibration sensors, edge detectors or temperature sensors. Non-limiting examples of pressure sensors that may be used include strain gauge sensors and force sensitive resistors (which e.g., may be embedded in the fingertips of the prosthesis). A thermistor may be used as a temperature sensor. An accelerometer may be used to sense vibration.
The prosthetic actuators may be any device capable of responding to the sensor (directly or through a controller) to produce a corresponding or scaled output at the appropriate location(s) of the reinnervated skin area. The actuators will be adapted to provide the appropriate form of sensory feedback to the reinnervated tissue location, e.g., a load actuator will produce a level of force corresponding (scaled down if needed) to the pressure sensed by a pressure sensor and a temperature actuator will heat or cool the appropriate location(s) of the reinnervated skin area to a degree corresponding (or scaled down but proportional) to a temperature detected by a temperature sensor. It is preferred that the actuators include means for monitoring the level of pressure, temperature, etc. applied to insure that the intended level is reached. This monitoring function may be performed by the controller.
Examples of pressure actuators include the devices of
Force sensors, pressure sensors or accelerometers may also be used to sense texture. Thus when an amputee passes a portion of the prosthesis fitted with a pressure sensor over a surface, the rapid variation in pressure as the pressure sensor moves over the surface will be reflected at the reinnervated skin site by the pressure actuator so long as the pressure sensor and actuator are adapted to respond at or near the same frequency as the pressure changes encountered. Texture could also be transmitted by a multiplicity of independent pressure sensors that send pressure data to a multiplicity of different pressure sensitive nerve endings at the reinnervated skin site.
Sharp/dull or edge surface features may be transmitted to the reinnervated skin site by taking pressure readings with sensors with a series of closely packed pins or edges that are oriented generally perpendicular to the prosthesis in the area where the sharp/dull or edge features are to be sensed. These sensors will work with actuators that will produce a corresponding sharp/dull or edge sensation at the reinnervated skin. Other sharp/dull or edge sensors and actuators which are currently known or developed in the future could also be used and are incorporated by reference.
The method of the invention may be performed as follows:
To date, such reinnervation has been accomplished as an adjunct to procedures for reinnervating an area of muscle. For example, nerves from an amputated arm have been transferred to chest muscles or to nerves in the chest so that nerves that formerly controlled a portion of the arm (e.g., wrist, fingers, biceps, etc.) instead are used to produce movement of a muscle in the chest. When the person thinks “close hand” or “curl bicep” for example, the transferred nerves cause specific chest muscles to contract to trigger a detection device. Of course, areas of the body other than the chest could be reinnervated in this way so long as there is suitable healthy remaining tissue for a particular patient and the particular limb that has been lost. A prosthetic device can be adapted to interface with the reinnervated tissue area so that when the wearer “thinks” to perform a particular action of a lost limb (e.g., contract biceps), a portion of the reinnervated tissue moves instead, triggering a detection device that, through a controller, actuates motion of the prosthetic device that corresponds to the desired motion (e.g., retract bicep).
In addition to the motor reinnervation of the muscle, sensory cross-reinnervation can be made to occur in the skin of the chest wall. When, following such muscle reinnervation, the chest wall is touched lightly in different places the patient will experience a transfer sensation of a light touch to different parts of his hand and arm. This sensory transfer takes place over the region where the key median and ulnar nerves along with the sensory nerves in the nerve bundle in the arm are anastomosed such as described in Example 1 below. The same would, of course, apply to nerves in the leg. While the present invention may be most useful to individuals who have undergone such targeted nerve-to-muscle reinnervation, it is also beneficial to individuals who undergo only denervation and reinnervation of an accessible skin area. Thus, targeted sensory reinnervation to produce transfer sensation is achieved by denervating sections of remaining skin in an amputee after which the sensation nerves of the hand or feet are guided to reinnervate this skin. Then, when the target skin is touched, warmed, etc., it will feel like the hand or foot is being touched, warmed, etc. The amputee will have near normal light touch levels, graded pressure, sharp/dull sensation, and thermal sensation—all in the missing limb.
The prosthetic prosthesis preferably will be motor powered and will be operable using EMG from nerve-muscle transfers as discussed above. Other types of prosthetic devices may however be used.
Appropriate pressure or temperature sensors will be located on or in the surface of appropriate portions of the prostheses. For example, in an arm/hand prosthesis the important areas are the tips of the fingers and the palm area. In a foot prosthesis, the important areas are the ball and heel and the pads of the toes.
For pressure sensing, a probe may be pressed lightly against portions of the reinnervated skin and the patient asked to indicate when he feels pressure and where in the former natural limb the pressure seems to originate. These locations will be marked for later use with pressure actuators.
For temperature sensing, a heated or cooled probe will be touched against portions of the reinnervated skin and the patient asked to indicate when he feels hot or cold and where in the former natural limb the hot or cool sensations seem to originate. These locations will be marked for later use with temperature actuators.
The pressure and temperature actuators will be located preferably at the locations identified in the prior step so that the pressure and temperature will be sensed as coming from prosthetic locations corresponding to former natural limb locations.
The sensors and corresponding actuators are either hardwired or they are wirelessly interconnected by using radio frequency (RF) transmitters and receivers. Also, a controller is interposed between the sensor(s) and actuator(s) to provide the power and electronics necessary for powering those components, for providing scaling and amplification as required, for monitoring and maintaining the desired pressure or temperature at the actuator, etc.
Example 1 below focuses on force or pressure sensing which is key to grasping. That is, appropriate force or pressure feedback gives the amputee goal confirmation in his use of a grasping prosthetic and also makes it possible for him to apply graded pressure with the prosthesis.
The following example further illustrates the invention but should not be construed as in any way limiting its scope.
In one embodiment of the present invention, a haptic interface 10 is shown diagramically in
Pressure sensor 16 may be mounted on a motor controlled portion of a prosthetic. The controller is coupled to a actuator positioned adjacent the surface of reinnervated skin containing nerves formerly associated with the portion of a user's lost limb now corresponding to the portion of the prosthetic equipped with the pressure sensor. The controller is coupled to the pressure sensor positioned on the prosthetic device and to an actuator that is capable of applying pressure to the reinnervated skin. The pressure sensor thus sends a signal indicating a pressure magnitude to the controller. In response, the controller actuates the actuator to apply a pressure to an area of skin corresponding to (or scaled relative to) the pressure magnitude detected by the sensor. As a result, the pressure applied to the skin stimulates nerves in the skin to transmit sensations to the brain, thereby providing a pressure transfer sensation of a recognizable magnitude and location.
This is illustrated diagrammatically in
As shown diagrammatically in
As explained above, in another embodiment illustrated in
Alternatively, as shown in
In
As explained above, in alternative embodiments of the invention, sensors for determining temperature may also be provided to the prosthetic device. The controller processes signals from these additional sensors for operating appropriate temperature actuator. Also, a single sensory device may detect multiple sensory conditions, such as pressure and temperature and a single actuator may impart multimodal stimulus to the reinnervated skin. For example, part 40 in these diagrams may include instead of a tactor a Peltier device capable of heating and cooling. Thus a single actuator may provide the transfer sensations of pressure (low frequency force), texture (an additional high frequency vibration) and temperature—all in the same location with the same device.
The following examples describe embodiments of the present invention and should not be construed as limiting its scope in any way.
1. This example describes targeted reinnervation to transfer nerves from a lost limb to denervated pectoralis muscle, achieving sensation of the lost limb on the chest of a subject. To evaluate this sensation as a potential for feedback in accordance with the invention, a high compliance/low inertia series elastic actuator could be used to apply force to the skin surface over the pectoralis muscle. The subject will have good force resolution when an external force is applied using an instrumented terminal device.
Nerve Rewiring
Using targeted reinnervation to transfer nerves from a lost limb to denervated pectoralis major and minor muscles 100 and 102 as shown in
As a result of the surgery, the subject will perceive touch that appears to originate from the prosthetic limb when pressure is applied to the chest at points 122-128, of
A like result can be achieved by denervating an available area of the skin, and reinnervating the denervated skin directly.
Actuator Selection
In order to achieve physiologically appropriate force feedback, an accurate force must be exerted against the chest. Because the chest moves with breathing, this matter becomes more complicated: an accurate force is desired, but the force must track the changing position of the chest. A linear backdrivable Series Elastic Actuator (SEA) can be used to decouple the inertia of the actuator from the force of the actuator. SEA are force controllable actuators with low impedance, high fidelity, and moderate bandwidth. They can be used to convert the accurate position control of traditional DC motors to accurate force control through the use of a spring as shown in
Motor 150 thus generates an accurate position. This position is fed through a compression spring 152 that converts the accurate position into an accurate force. A linear potentiometer 154 that measures the compression and converts it into a force reading, which is then compared to the desired force 156 by a comparator 158. The error between the two signals is sent to a control block 160. The control block multiplies the error signal by a gain (K) and the derivative of the error by another gain (D) and sends this signal to the motor 150 to correct the output force. The forces can be nonlinearly scaled to provide increased resolution for low magnitude forces and/or decreased resolution for high magnitude forces.
Testing of a system as described above with a linear conversion from hand force to chest force and a nonlinear conversion from hand force to chest force will produce increased resolution of low forces helping the subject discriminate low forces more effectively.
The subject's breathing will not disrupt his perception of force. When the subject is given various scale weights, he should be able to subjectively assess the inertia of the actuator.
2. An area of reinnervated skin was tested by applying carefully measured force to the skin. The patient was found to have a light touch threshold (first perception of being touched) of 2 g/mm2, which is very close to normal values for natural limbs. Transfer sensation in terms of sensitivity to graded higher and lower pressures was found to be about one-half of the normal value for natural limbs. The patient had normal hot and cold perception and was able to distinguish between sharper and duller stimuli.
All references, including publications, patent applications, and patents, cited herein are hereby incorporated by reference to the same extent as if each reference were individually and specifically indicated to be incorporated by reference and were set forth in its entirety herein.
The use of the terms “a” and “an” and “the” and similar referents in the context of describing the invention (especially in the context of the following claims) are to be construed to cover both the singular and the plural, unless otherwise indicated herein or clearly contradicted by context. The terms “comprising,” “having,” “including,” and “containing” are to be construed as open-ended terms (i.e., meaning “including, but not limited to,”) unless otherwise noted. Recitation of ranges of values herein are merely intended to serve as a shorthand method of referring individually to each separate value falling within the range, unless otherwise indicated herein, and each separate value is incorporated into the specification as if it were individually recited herein. All methods described herein can be performed in any suitable order unless otherwise indicated herein or otherwise clearly contradicted by context. The use of any and all examples, or exemplary language (e.g., “such as”) provided herein, is intended merely to better illuminate the invention and does not pose a limitation on the scope of the invention unless otherwise claimed. No language in the specification should be construed as indicating any non-claimed element as essential to the practice of the invention.
Preferred embodiments of this invention are described herein, including the best mode known to the inventors for carrying out the invention. Variations of those preferred embodiments may become apparent to those of ordinary skill in the art upon reading the foregoing description. The inventors expect skilled artisans to employ such variations as appropriate, and the inventors intend for the invention to be practiced otherwise than as specifically described herein. Accordingly, this invention includes all modifications and equivalents of the subject matter recited in the claims appended hereto as permitted by applicable law. Moreover, any combination of the above-described elements in all possible variations thereof is encompassed by the invention unless otherwise indicated herein or otherwise clearly contradicted by context.
This application claims the benefit of U.S. Provisional Application No. 60/707,481, filed Aug. 11, 2005, and incorporated by reference in its entirety.
| Number | Date | Country | |
|---|---|---|---|
| 60707481 | Aug 2005 | US |
