SENSOR SYSTEM AND METHOD FOR CONTROL OF PROSTHETIC DEVICES

FIELD

The present disclosure relates to prosthetics. More specifically, this disclosure relates to a sensor assembly and methods of controlling prosthetic systems.

BACKGROUND

Amputees benefit greatly from prosthetic replacements for lost limbs. For instance, prosthetic hands, wrists, feet, legs, or arms restore lost functionality and provide independence to users. Likewise, orthotic devices may be used to assist with limb movement. However, control of a prosthetic or orthotic device (POD) suffers from lack of reliable user data. Improved prosthetic solutions for detecting user inputs for control of PODs are therefore desirable.

Speed control of prosthetics is commonly implemented as a so-called “proportional control”, in which the speed of the prosthetic’s movement (e.g., digit movement, wrist rotation, elbow flexion etc.) is proportional to the strength of a corresponding filtered and scaled electromyography (EMG) signal. Even if a user is trying to achieve constant muscle strength throughout a muscle movement, EMG measurements are typically filtered in a such way that renders the associated EMG signal jittery, and the signal level drops significantly after being strongest at the onset.

Further, unintended and unwanted EMG signals can occur from a variety of noise sources and perturbations, such as relative movement at the skin-sensor interface. While EMG signals are the most common control input in multi-articulating upper-limb prosthetics, these associated problems cause one of the biggest problems faced by prosthetic users: unintended movement. Unintended movement of a prosthetic occurs from unintended sensor input that is interpreted by the prosthetic control algorithm as a valid, movement-inducing control signal. This can result in prosthetic users breaking and dropping items they are holding, significantly reducing the trust they place in their device.

EMG signals are also used in pattern recognition for prosthetics. Pattern recognition uses a learning algorithm to control multiple degrees of freedom of a prosthetic device without “mode switching”. An illustrative example of mode switching is where a user first selects which parts of the device to move (e.g., move thumb and index to create a pinching grip) and subsequently moves only these parts with the control signals. To move other parts of the device (e.g., a powered wrist rotator), the user must then select the corresponding functionality and then control it with the same control signals that were previously used for the pinching movement. Pattern recognition makes this mode switching obsolete.

Pattern recognition in prosthetics is often formulated as a classification problem with supervised learning. This means that sensor data is acquired while prompting a user to perform the intended grips with their phantom limb. For example, a user might be prompted to imagine shaping a fist with their phantom limb, and the data measured during this muscle flexion is labelled as “fist”. The pattern recognition algorithm is then trained to recognize similar data and prompt a prosthetic hand to move to form a fist. For tasks like these, pattern recognition algorithms in prosthetics require multi-dimensional data, with each dimension being made up of a so-called “feature” which may be a measured signal or derived from measured signals.

Prosthetic pattern recognition systems rely on several features that are commonly calculated from the same EMG signal and are therefore inherently correlated. The stronger the correlation of features, the less useful their combined use in a pattern recognition algorithm. Finding uncorrelated features is therefore desirable in order to increase the performance (e.g., classification accuracy) of a pattern recognition algorithm. To reduce correlation and increase the number of features, pattern recognition systems often include more sensors than the conventional two-sensor setup in myoelectric prostheses, often up to eight sensors. However, from a clinical perspective, adding additional sensor sites increases the complexity of sensor placement and might also be constrained by space, as some residual limbs are very short and therefore, not much space is available to place sensors in useful locations.

SUMMARY

The embodiments disclosed herein each have several aspects no single one of which is solely responsible for the disclosure’s desirable attributes. Without limiting the scope of this disclosure, its more prominent features will now be briefly discussed. After considering this discussion, and particularly after reading the section entitled “Detailed Description,” one will understand how the features of the embodiments described herein provide advantages over existing systems, devices and methods for prosthetic control.

The following disclosure describes non-limiting examples of some embodiments. For instance, other embodiments of the disclosed systems and methods may or may not include the features described herein. Moreover, disclosed advantages and benefits may apply only to certain embodiments of the invention and should not be used to limit the disclosure.

One aspect of the present disclosure provides an improved method of controlling a prosthetic device by measuring a user’s muscle movement via a sensor assembly placed within the user’s socket. The sensor assembly may include one or more of a displacement or force sensor, an EMG sensor and an internal measurement unit (IMU). The method may further involve adjusting a speed of movement of a prosthetic device connected to the socket based on an amount of force or muscle displacement derived from measurements of the sensor assembly.

Various embodiments of the various aspects described herein may be implemented. For example, the sensor assembly may comprise the displacement sensor, the displacement sensor may comprise a magnet and a magnetic sensor moveable relative to the magnet, and the sensor assembly may measure a change in a magnetic field strength or a magnetic flux density based on a change in distance and/or orientation between the magnet and the magnetic sensor due to the muscle movement. The amount of force or muscle displacement may be calculated by a processor based on measurements of the amount of force being applied to the sensor assembly or a change in the magnetic field strength or magnetic flux density. The sensor assembly may further be calibrated for a plurality of known distances and/or orientations between the magnet and the magnetic sensor. The amount of muscle displacement may be calculated based on a quadratic relationship between muscle displacement and the magnetic field or magnetic flux. The prosthetic device may be one of: a prosthetic digit, prosthetic hand or partial prosthetic hand, a prosthetic wrist, prosthetic elbow, prosthetic knee, a prosthetic shoulder, or a prosthetic ankle.

The method of controlling a prosthetic device may also include any combination of the following features, among others described herein. The sensor assembly may include the EMG sensor. The sensor assembly may comprise the force sensor, and a pressure measurement may be derived from the force sensor. The method of controlling a prosthetic device may further involve applying a baseline correction to the amount of force or muscle displacement derived from the measurements of the sensor assembly. Such baseline correction may be determined by determining if a muscle is active or inactive utilizing the EMG sensor, and then, if the muscle is active, determining a force or muscle displacement reading and subtracting the displacement reading from the amount of force or muscle displacement derived from measurements of the sensor assembly. The sensor assembly may include an inertial measurement unit (IMU) and an EMG sensor.

In another aspect, the method of controlling movement of a prosthetic device according to the present disclosure may involve detecting muscle activity of a prosthetic device user using a sensor assembly including an EMG and displacement sensor positioned in the user’s socket. The sensor assembly may be capable of determining if a muscle is active or inactive using the EMG sensor and determining a displacement reading if the EMG sensor determines that the muscle is active. The displacement reading may then be subtracted from a muscle displacement amount derived from the displacement sensor to determine an adjusted muscle displacement measurement. The adjusted muscle displacement measurement may then be used to control the movement of the prosthetic device.

In some embodiments, the user’s muscle may be deemed inactive if the measured EMG signal measured by the EMG sensor is below a certain threshold, or the muscle may be deemed active if the EMG signal is at or above a certain threshold. The displacement sensor may comprise a magnet and a magnetic sensor. The magnetic sensor may be configured to measure a change in a magnetic field strength or a magnetic flux density based on a change in distance between a magnetic sensor and a magnet. The sensor assembly may further comprise an inertial measurement unit (IMU). The adjusted muscle displacement measurement may be used to control a speed of the prosthetic device. Further, the adjusted muscle displacement measurement may be used for pattern recognition.

In an additional aspect, a method of controlling movement of a prosthetic device according to the present disclosure may involve detecting a muscle activity measurement derived from a sensor assembly positioned within a socket worn by a user, where the sensor assembly includes an EMG sensor that measures an EMG signal and a force or pressure sensor. Movement of the prosthetic device connected to the socket may then be controlled using a pattern recognition algorithm programmed to use both the measured EMG signal and the muscle activity measurement.

In an additional aspect, the sensor assembly may include an EMG sensor, an IMU, and one or more force sensors. One force sensor may be placed in such a way that securing the sensor assembly in a prosthetic socket results in a measurable signal indicating when there is force against the EMG side of the sensor assembly. When several force sensors are used, the sensors may be placed in such a way that it is possible to measure not only the total force being applied against the sensor assembly, but also how centered or off-centered the force is being applied against the sensor assembly. Further, pressure may, alternatively or in addition to force, be used as described with respect to the force sensor. Thus, any description herein of the use of “force” applies equally to pressure, unless otherwise indicated. In some embodiments, a pressure is detected directly. In some embodiments, a force is detected and a pressure is calculated based on the detected force and an applied area of the force.

While many of the embodiments disclosed herein use a magnetic sensor and a magnet to calculate a displacement reading, it should be noted that further embodiments may include any other type of sensor capable of measuring displacement.

BRIEF DESCRIPTION OF THE DRAWINGS

The foregoing and other features of the present disclosure will become more fully apparent from the following description and appended claims, taken in conjunction with the accompanying drawings. Understanding that these drawings depict only several embodiments in accordance with the disclosure and are not to be considered limiting of its scope, the disclosure will be described with additional specificity and detail through use of the accompanying drawings. In the following detailed description, reference is made to the accompanying drawings, which form a part hereof. In the drawings, similar symbols typically identify similar components, unless context dictates otherwise. The illustrative embodiments described in the detailed description, drawings, and claims are not meant to be limiting. Other embodiments may be utilized, and other changes may be made, without departing from the spirit or scope of the subject matter presented here. It will be readily understood that the aspects of the present disclosure, as generally described herein, and illustrated in the drawing, may be arranged, substituted, combined, and designed in a wide variety of different configurations, all of which are explicitly contemplated and make part of this disclosure.

FIG. 1A is a perspective exploded view of an embodiment of a lower arm prosthetic system that may use the sensor assembly described herein.

FIG. 1B is a side view of another embodiment of a lower arm prosthetic system, including a lower arm stump having four prosthetic digits and a prosthetic thumb attached to the stump, and that includes an embodiment of a sensor assembly.

FIGS. 2A and 2B illustrate side perspective views of an embodiment of a fitting or socket configured to attach to a POD and to a residual limb or a sound limb of a user of the POD and that includes an embodiment of the sensor assembly described herein.

FIG. 3A is a perspective view of an embodiment of a sensor assembly.

FIG. 3B is an exploded view of the sensor assembly of FIG. 3A.

FIG. 4A is a perspective view of another embodiment of a sensor assembly.

FIG. 4B is an exploded view of the sensor assembly of FIG. 4A.

FIG. 4C is a cross-sectional view of the sensor assembly of FIG. 4A.

FIGS. 5A-5D illustrate various configurations of an example sensor assembly showing a lower portion or support of the assembly in various orientations relative to the upper portion or housing.

FIG. 6 is a block diagram showing a schematic of an example prosthetic system having the sensor assembly.

FIG. 7 is a table illustrating various example associations between input signals and hand gestures or hand grips for generating inputs to the sensor assembly.

FIG. 8 illustrates example data graphs of various example input signals detected by the sensor assembly.

FIGS. 9A-9D illustrate example data graphs of various example data received from an example sensor assembly.

FIGS. 10A-10C illustrate a top perspective view, a bottom perspective view, and a side view respectively of an embodiment of a grommet for a sensor assembly.

FIG. 11 illustrates a partial cross sectional side view of an example sensor assembly removably inserted into an example grommet and the grommet attached to a socket of a POD.

FIG. 12A illustrates a perspective view of another embodiment of a sensor assembly removably inserted into another embodiment of a grommet.

FIG. 12B illustrates an exploded view of the sensor assembly and grommet of FIG. 12A.

FIGS. 13A-13D illustrate various views of the grommet of FIGS. 12A and 12B.

FIGS. 14A and 14B illustrate perspective views of the example sensor assembly of FIGS. 12A and 12B with a support located in different positions.

FIG. 15A illustrates a perspective view of the support of FIGS. 12A and 12B.

FIG. 15B illustrates a perspective view of the support of FIG. 15A with an O-ring.

FIG. 16A illustrates a perspective view of an electrode base of the sensor assembly of FIGS. 12A and 12B.

FIG. 16B illustrates a cross-sectional, perspective view of the electrode base of FIG. 16A removably coupled with the support of FIG. 15B.

FIG. 17A illustrates a partial perspective view of a groove of the support of FIG. 15A.

FIG. 17B illustrates a cross-sectional view of the groove of FIG. 17A.

FIG. 17C illustrates a side, close-up view of the support of FIG. 15B with an O-ring.

FIG. 18A illustrates a bottom perspective view of the support of FIGS. 15A and 15B with an electrode printed-circuit-board (PCB) and springs.

FIG. 18B illustrates a bottom perspective view of the support of FIG. 18A without the electrode PCB.

FIG. 18C illustrates a perspective, exploded view showing the sensor assembly and grommet of FIG. 12A with the support separated from the rest of the sensor assembly.

FIG. 19A illustrates an example EMG signal with typical signal processing for proportional control that is measured from a prosthetic user performing three muscle contractions.

FIG. 19B illustrates an example muscle displacement signal simultaneously measured with the EMG signal of FIG. 19A.

FIG. 20A illustrates an example EMG signal with typical signal processing for proportional control that is measured from a prosthetic user attempting to maintain a constant muscle force over a short period of time.

FIG. 20B illustrates a muscle displacement signal simultaneously measured with the EMG signal in FIG. 20A.

FIG. 21A illustrates an example EMG signal measured from a prosthetic user performing three muscle contractions and the threshold EMG signal level.

FIG. 21B illustrates an example graph depicting a raw muscle displacement signal simultaneously measured with the EMG signal of FIG. 21A, a dynamically updated baseline threshold EMG signal level, and the resulting corrected displacement when subtracting the baseline from the raw displacement.

FIG. 22A illustrates an example of dynamic baseline correction in a system with two sensors, where the correction is implemented on each sensor individually.

FIG. 22B illustrates an example of dynamic baseline correction in a system with two sensors, where the correction is implemented in a combined way in that muscle activity from one sensor influences the baseline correction on a different sensor.

FIGS. 23A and 23B show data captured by one sensor while a user performed five different distinct wrist and hand movements to facilitate pattern recognition.

FIG. 24 illustrates an example lower limb prosthetic capable of housing the hybrid sensor system.

DETAILED DESCRIPTION

The following detailed description is directed to certain specific embodiments of the development. In this description, reference is made to the drawings wherein like parts or steps may be designated with like numerals throughout for clarity. Reference in this specification to “one embodiment,” “an embodiment,” or “in some embodiments” means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the invention. The appearances of the phrases “one embodiment,” “an embodiment,” or “in some embodiments” in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments necessarily mutually exclusive of other embodiments. Moreover, various features are described which may be exhibited by some embodiments and not by others. Similarly, various requirements are described which may be requirements for some embodiments but may not be requirements for other embodiments.

Summary of Some Sensor Assemblies for Upper and Lower Limb Prosthetics

The prosthetic systems and methods described herein are described with respect to upper limb prosthetics but may be used with lower limb prosthetics as well. Prosthetic hand and wrist systems and methods described herein provide a more responsive upper limb prosthetic system for amputees. A prosthetic hand or wrist system may include one or multiple sensor assemblies configured to be placed at one or more locations on a natural limb to capture activity from the user. Each sensor assembly may include one or multiple sensors that obtain a variety of different input signal types from the patient, and a controller that may receive, analyze, and translate the input signals into a movement of the prosthetic device, such as an arm rotation, a wrist rotation, and/or a hand/digit movement. The use of multiple sensors within a single sensor assembly and within a small, nonintrusive package to obtain multiple input signals may substantially increase the opportunity of obtaining useful patient input. For example, the controller may apply any one or a combination of various arbitration techniques to select, combine, or ignore input signals when translating the input signals into the prosthetic movement. One or more of the one or more sensors may be moveably held by the assembly such that the sensor(s) adjust their position and/or orientation in response to movement of the user’s muscle, for example to maintain a particular position and/or orientation relative to the user’s muscle.

The present disclosure provides an improved sensor assembly which increases reliability of translating patient input into intended prosthetic movement by allowing for more input data and thus the ability for better classification and accuracy of translation into prosthetic movements. In one aspect, a sensor assembly according to the present disclosure may include a housing and a support. The housing may define a central axis and may be configured to attach with a fitting or socket of a POD. It will be appreciated that the embodiments described throughout this specification may be applied not only to prosthetic devices, but also to orthotics or other wearable devices worn by an amputee or non-amputee user. Accordingly, where the term “prosthetic, “prosthetic device” or similar term is utilized, the embodiments described, e.g., the sensor assembly, may also apply to an orthotic or other wearable device.

The support may be moveably connected with the housing such that the support is configured to move relative to the housing along the central axis. The support may form an enclosure with the housing. The sensor assembly may further include one or more of a plurality of sensors disposed within the enclosure formed by the support and the housing. The plurality of sensors may include one or more sensors that respond to muscle activation, such as a sensor that is sensitive to force, distance/displacement, potential difference (e.g., voltage), vibrations (for example, of muscle fibers), sound/acoustic readings, or another symptom of muscle activity. The plurality of sensors may include an inertial measurement unit (IMU), an electromyography (EMG) sensor, a microphone, a voltage sensor, a displacement or distance sensor, a force or pressure sensor, a force sensitive resistor reading force caused by the displacement of muscle tissue, and/or a magnetic sensor. The sensor assembly may further include a circuit board attached with the support and in electrical communication with the one or more of the plurality of sensors.

Various embodiments of the various aspects described herein may be implemented. For example, the sensor assembly may also include any combination of the following features, among others described herein. An EMG sensor may include multiple EMG pickups carried by the support and configured to contact skin of a user of the POD and to move along with the support. The sensor assembly may further include a magnet. The magnet may be configured to generate a magnetic field strength that may be detected by a magnetic sensor. The magnet may be attached with the housing and the magnetic sensor may be attached with the support and configured to move along with the support. The sensor assembly may include at least one biasing member configured to bias the support in a direction away from the housing and toward the user. The at least one biasing member may include a compression spring. The IMU may be configured to detect a vibromyography (VMG) signal associated with a muscle of the user. The IMU may include one or more of an accelerometer, a gyroscope, or a magnetometer. The gyroscope may be used to determined limb axial rotation. The magnetic sensor may include one or more of a Hall effect sensor, a magnetometer, or the like. The sensor assembly may further include a force sensor.

The sensor assembly may also include any combination of the following features, among others described herein. The sensor assembly may include a tray attached to the support between the support and the housing. The sensor assembly may include a pivot axis perpendicular to the central axis and a sliding axis perpendicular to both the central and pivot axes. The support may be pivotably or slidably connected to the housing such that the support may pivot about the pivot axis relative to the housing or may slide along the sliding axis.

In another aspect, a sensor assembly according to the present disclosure may include a housing and a support. The housing may define a central axis, a pivot axis perpendicular to the central axis, and a sliding axis perpendicular to both the central axis and pivot axis. The housing may be configured to be attached with a fitting or socket or a POD. The support may be moveably connected with the housing such that the support may move relative to the housing. The support may form an enclosure with the housing. The support may be pivotably or slidably connected to the housing such that the support may pivot about the pivot axis or may slide along the sliding axis. The sensor assembly may include one or more of a plurality of sensors disposed within the enclosure. The plurality of sensors may be configured to detect indications of muscle activity. For example, the plurality of sensors may be configured to detect one or more of electrical signals associated with a residual limb or a sound limb of a user of the POD, translation of a muscle of the residual limb or the sound limb of the user, inertial forces generated by movement of the residual limb or the sound limb of the user, force, distance/displacement, potential difference, or vibrations. The sensor assembly may include a circuit board disposed within the enclosure and in electrical communication with the one or more of the plurality of sensors.

The sensor assembly may also include any combination of the following features, among others described herein. The plurality of sensors may include an inertial measurement unit (IMU). The IMU may include at least one of an accelerometer, a gyroscope, and/or a magnetometer. The plurality of sensors may include an electromyography (EMG) sensor. The EMG sensor may include multiple EMG pickups carried by the support and configured to contact the residual limb or the sound limb of the user. The plurality of sensors may include a magnetic sensor. The sensor assembly may further include a magnet. The magnet may be configured to generate a magnetic field strength that may be detected by the magnetic sensor. The plurality of sensors may include a displacement sensor, a distance sensor, a microphone, a voltage sensor, or a force sensor. The sensor assembly may include at least one biasing member configured to bias the support along the sliding axis in a direction away from the housing and toward the user. The at least one biasing member may include a compression spring.

The present disclosure provides an improved apparatus for restoring mobility to an amputee with various advantages. For example, the apparatus may increase reliability of translating patient input into intended prosthetic movement by allowing for more input data and thus the ability for better classification and accuracy of translation into prosthetic movements. An apparatus for restoring mobility to an amputee may include a POD and a fitting or socket. The fitting or socket may be configured to attach to the POD and to a residual limb or a sound limb of a user of the POD. The apparatus may further include a sensor assembly configured to attach with the fitting or socket and configured to contact the residual limb or the sound limb when the fitting or socket is attached to the residual limb or the sound limb. The sensor assembly may include an enclosure formed by a housing and a support. The support may be moveably connected with the housing. The sensor assembly may include one or more of a plurality of sensors carried by the enclosure. The plurality of sensors may include an inertial measurement unit (IMU), an electromyography (EMG) sensor, a magnetic sensor, a microphone, a voltage sensor, or a force sensor. The sensor assembly may include a circuit board in electrical communication with the one or more of the plurality of sensors.

The apparatus and embodiments thereof of any of the preceding paragraphs may also include any combination of the following features described in this paragraph, among others described herein. The support may include a tray. The support may include one or more EMG pickups of the EMG sensor. The support may be pivotably connected to the housing such that the support may pivot relative to the housing. The support may be slidably connected to the housing such that the support may slide relative to the housing. The IMU may include at least one of an accelerometer, a gyroscope, or a magnetometer. The EMG sensor may include multiple EMG pickups carried by the support and configured to contact the residual limb or the sound limb of the user. The apparatus may further include a magnet. The apparatus may further include a grommet configured to attach to the fitting or socket. The sensor assembly can attach to the fitting or socket via the grommet. The apparatus may further include a force sensor.

The apparatus and embodiments thereof of any of the preceding paragraphs may also include any combination of the following features described in this paragraph, among others described herein. The base portion or housing of the sensor assembly can include a cutout and the top portion of the sensor assembly can include a locking device including a detent. The cutout can receive the detent of the locking device and facilitate movement of the top portion relative to the base portion. The top portion of the sensor assembly can include a groove that can receive an O-ring, wherein the O-ring can abut against sidewalls of the base portion and create a seal. The groove can include ridges positioned at corners of the top portion and the ridges may extend vertically within the groove. The ridges can cause the O-ring to stretch around the corners and push the O-ring outwards. This can allow the O-ring to contact inner surface of sidewalls of the base portion and create a seal.

Detailed Description of Embodiments of Prosthetic Systems and Sensor Assemblies

As described at least in part above, the prosthetic hand and wrist system and methods described herein provide a more responsive upper limb prosthetic system for amputees. A prosthetic hand or wrist system may include one or multiple sensor assemblies fitted to an internal side of a fitting or socket worn on a stump of an amputee. The sensor assemblies are located at different sites on the fitting or socket such that when the fitting or socket is worn by the user the sensor assemblies are aligned with various forearm flexor or extensor muscles. In this way, the sensor assemblies may capture information regarding muscle activity or movement. Each sensor assembly may include a plurality of sensors. In some cases, the plurality of sensors can be referred to as muscle activity sensors. A sensor assembly may include one or more of an electromyography (EMG) sensor, a magnetic sensor (e.g., a Hall effect sensor, a magnetometer, etc.), a mechanomyography (MMG) sensor, an inertial measurement unit (IMU), an accelerometer, a gyroscope, a sound sensor (e.g., an acoustic sensor, a microphone), a force sensor, etc. A controller may translate signals received from the sensor assemblies (sometimes referred to as input signals) into a prosthetic movement.

In general, the ability of individuals to apply input to a prosthetic device may be wide and varied. This, in combination with sub-optimal sensor positioning on the individual may make the harvesting of useful patient input difficult. The prosthetic hand and wrist system of the present disclosure addresses these and other concerns by utilizing multiple measurement sites and multiple sensors with different data types at each measurement site. In this way, the disclosed system allows individuals to provide input using multiple different inputs, and a controller may use any one or combination of various arbitration techniques to select, combine, or ignore input signals when translating the input signals into the prosthetic movement.

Furthermore, the present disclosure provides for an improved sensor assembly which increases reliability of translating patient input into intended prosthetic movement by allowing for more input data and thus the ability for better classification and accuracy of translation into prosthetic movements. The sensor assembly of the present disclosure includes multiple sensor types in a single package, thereby providing multiple separate input paths for the individual to act on. This may increase the reliability of a translation from user input to prosthetic movement by allowing multiple dimensions of patient input and thus increasing the opportunity of obtaining useful patient input.

FIG. 1A is a perspective view of an example lower arm prosthetic system 100, with a prosthetic hand 102 shown as detached from a prosthetic wrist 104. For sake of description, various geometric references are used. A “distal” direction and a “proximal” direction are indicated. As shown, the prosthetic wrist 104 is located proximally relative to the prosthetic hand 102, and the prosthetic hand 102 is located distally relatively to the prosthetic wrist 104. The distal and proximal directions refer to, respectively, directions farther from and closer to a user of the lower arm prosthetic system 100 along the length of an arm of the user containing the lower arm prosthetic system 100. A longitudinal axis 108 is an axis of rotation about which the prosthetic wrist 104 and prosthetic hand 102 rotate. The distal and proximal directions may be directions along the longitudinal axis 108, for example when the lower arm prosthetic system 100 is oriented as shown. “Inner,” “inward,” and like terms refer to directions toward the longitudinal axis 108, while “outer,” “outward,” and like terms refer to directions away from the longitudinal axis 108, unless otherwise indicated.

The prosthetic wrist 104 may be attached to the prosthetic hand 102 to cause movement of the prosthetic hand 102 and thereby forming the complete lower arm prosthetic system 100. A rotatable portion of the prosthetic wrist 104 may be coupled with, for example attached to, the prosthetic hand 102. The rotatable portion of the prosthetic wrist 104 rotates, thereby causing rotation of the prosthetic hand 102. The prosthetic wrist 104 and prosthetic hand 102 rotate about the longitudinal axis 108. The prosthetic hand 102 may form a plurality of different grips, for example different palm or digit positions. The lower arm prosthetic system 100 may synchronize the rotation of the prosthetic wrist 104 with the formation of one of the grips with the prosthetic hand 102. These movements may be based on data received and analyzed by a sensor assembly 202, as described herein.

The prosthetic lower arm 106 is a prosthetic for the lower or outer segment of an arm, for example the forearm. The prosthetic lower arm 106 is a hollow tube with an arm-like shape. The prosthetic lower arm 106 may have a variety of other suitable shapes and configurations. The proximal end of the prosthetic lower arm 106 attaches to a user, for example to a stump of an amputee, a fitting, socket, etc. The distal end of the prosthetic lower arm 106 attaches to the prosthetic wrist 104. The prosthetic lower arm 106 may be mechanically or electrically connected to the prosthetic wrist 104. The prosthetic lower arm 106 may include one or more of the sensor assemblies 202 as further described herein, for example, attached to an inner surface thereof.

When attached to the prosthetic wrist 104, the prosthetic hand 102 may rotate about the longitudinal axis 108 as indicated, while the prosthetic lower arm 106 remains rotationally stationary. Thus, the prosthetic hand 102, when attached to the prosthetic wrist 104, may rotate about the longitudinal axis 108 relative to the prosthetic lower arm 106. The rotation of the prosthetic hand 102 is caused by the prosthetic wrist 104.

The lower arm prosthetic system 100 described herein is merely an example of a prosthetic lower arm, prosthetic wrist, or prosthetic hand system that may use the sensor assembly 202. Other prosthetic hands, wrists, or arms may be implemented, for example as described in U.S. Pat. No. 10,369,024, titled SYSTEMS AND METHODS FOR PROSTHETIC WRIST ROTATION and issued on Nov. 02, 2016, the entire contents of which is incorporated by reference herein for all purposes.

FIG. 1B is a side view of another example lower arm prosthetic system 101 including a lower arm portion 112 connected with four prosthetic digits 120 and a prosthetic thumb 130. The digits 120 may be connected to the end of the lower arm portion 112, as shown in FIG. 1B. The lower arm portion 112 may include an embodiment of the prosthetic wrist 104, the prosthetic lower arm 106, the prosthetic hand 102, or any combination thereof. In some cases, the prosthetic digits 120 and the prosthetic thumb 130 may be part of the prosthetic hand 102. The portion 112 may include the sensor assembly 202 attached to an inner surface thereof and contact a residual limb or a sound limb of a user when worn.

The prosthetic 101, prosthetic digits 120 and thumb 130 described herein are merely examples of prosthetics that may use the sensor assemblies of the present disclosure. Other prosthetic and orthotic systems may use the sensor assemblies described herein, for example the prosthetic and orthotic systems as described in PCT Publication No. WO 2021/124060, titled ELECTROMYOGRAPHY AND MOTION BASED CONTROL OF UPPER LIMB PROSTHETICS, published Jun. 24, 2021; PCT Publication No. WO 2021/095014, titled PROSTHETIC DIGIT ACTUATOR, published May 20, 2021; PCT Publication No. WO 2020/234777, titled ACTUATION SYSTEMS FOR PROSTHETIC DIGITS, published Nov. 26, 2020; PCT Publication No. WO 2020/208557, titled PROSTHETIC DIGIT WITH ARTICULATING LINKS, published Oct. 15, 2020; U.S. Patent No. 10,973,660, titled POWERED PROSTHETIC THUMB, issued Apr. 13, 2021; U.S. Patent No. 11,185,426, titled SYSTEMS AND METHODS FOR PROSTHETIC WRIST ROTATION, issued Nov. 30, 2021; U.S. Pat. No. 10,449,063, titled WRIST DEVICE FOR A PROSTHETIC LIMB, issued on Oct. 22, 2019; and U.S. Pat. No. 10,265,197, titled SYSTEMS AND METHODS FOR CONTROLLING A PROSTHETIC HAND, issued Apr. 23, 2019, each of which is hereby incorporated by reference herein in its entirety for all purposes and forms a part of this specification.

FIGS. 2A and 2B illustrate side perspective views of an example fitting 200 having embodiments of sensor assemblies 202A-202D. Any of the fittings described herein may also be referred to as a “socket.” The fitting 200 is configured to attach to a POD and a stump of a user of the POD. The fitting 200 may be formed of plastics, metals, composites, polymers, other suitable materials, or combinations thereof. As shown, the fitting 200 includes four sensor assemblies 202A, 202B, 202C, 202D (individually or collectively referred to as sensor assembly 202 or sensor assemblies 202). However, it will be understood that the number of sensor assemblies of the fitting 200 may vary across embodiments. There may be one, two, three, five, or more sensor assemblies 200, which may be attached to the same fitting 200, or some may be attached to other parts of a user, such as on a residual limb or a sound limb.

Each sensor assembly 202 is at least partially located on an internal side of the fitting 200 such that the sensor assembly 202 contacts a user’s skin when the fitting 200 is worn on a stump. In instances in which the fitting 200 is worn on a trans-radial amputee forearm, the sensor assemblies 202 may be positioned in the fitting 200 such that they are generally aligned with the forearm muscles. For example, in some cases, the sensor assembly 202A contacts an internal side of the arm and is aligned with the forearm flexor muscles and the sensor assemblies 202B, 202C, and 202D contact an opposite, external side of the arm and are aligned with the forearm extensor muscles. The fitting 200 may extend axially and have a cavity defined therein by a sidewall and in which the one or more sensors 200 are located along inner surfaces of the sidewall. A proximal side of the fitting 200 may attach to a user and a distal side of the fitting 200 may attach to a prosthetic device.

FIG. 3A is a perspective view of the sensor assembly 202. FIG. 3B is an exploded view of the sensor assembly 202 of FIG. 3A. As described herein, one or more sensor assemblies 202 are configured to attach to a POD, such as the lower arm prosthetic system 100 of FIGS. 1A or 1B, or a fitting that attaches to a POD, such as the fitting 200 of FIGS. 2A-2B. Further, any features or functions of the sensor assembly 202 may be incorporated into any other embodiments of sensor assemblies described herein, such as those shown in FIGS. 12A-18C, and vice versa.

The sensor assembly 202 includes an upper portion or housing 302, which forms a portion of an outer structure and enclosure of the sensor assembly 202. The housing 302 may be formed of plastics, metals, composites, polymers, other suitable materials, or combinations thereof. The housing 302 has a central portion 360 and a pair of end portions 362 on either end of the central portion 360. The central portion 360 may be rectangular as shown with the long end defining a longitudinal axis 312. The end portions 362 may be rectangular as shown. The housing 302 defines the longitudinal axis 312 extending along a length of the housing 302 and a lateral axis 314 extending along the width of the housing 302. The lateral axis 314 may be perpendicular to the longitudinal axis 312.

The sensor assembly 202 further includes a lower portion or support 304, which may be an overmold, and which forms a portion of the outer structure and enclosure of the sensor assembly 202. The support 304 may be formed of plastics, metals, composites, polymers, other suitable materials, or combinations thereof. The support 304 has a rounded rectangular shape and includes a pair cylindrical wing portions 370 on either end of the support 304.

The support 304 connects with the housing 302 to form the enclosure between the support 304 and the housing 302. The support 304 may be movably connected with the housing 302 such that the support 304 may translate and/or rotate relative to the housing 302. For example, the support 304 may be connected with the housing 302 such that it may move relative to the housing 302 along a central axis 316. The support 304 may be slidably connected to the housing 302 such that the support 304 may slide along the sliding axis or central axis 316. The central axis 316 may be perpendicular to the longitudinal axis 312 of the housing 302 and the lateral axis 314 of the housing 302.

In some embodiments, movement of the support 304 relative to the housing 302 is caused by user movement applying a compressive force toward the sensor assembly and compressing at least one biasing member located between the support 304 and the housing 302. For example, FIG. 3B illustrates four biasing members 372 that bias the support 304 in a direction away from the housing 302 and toward the user. In this example, the biasing members 372 are illustrated as compression springs. However, other biasing members are contemplated, such as flexible members, extension springs, other suitable biasing members, or combinations thereof. The biasing members 372 thus pre-load the support 304 so that it contacts the user. The biasing members 372 then compress in response to outward movement of the user’s limb and/or muscle, and thereby allow relative movement of two portions of the sensor assembly. This relative movement is detected by one or more of the sensors and analyzed for prosthetic control, as further described herein.

In some cases, the support 304 is additionally or alternatively pivotably connected to the housing 302 such that the support 304 may be pivoted or partially rotated about one or more pivot axes. The pivot axis or axes may align with the longitudinal, lateral and/or central axes 312, 314, 316. For example, in some cases, the projections 370 of the support 304 may engage with a complementary engagement cavity of the housing 302. The projections may be cylindrical and extend longitudinally away from the support 304. In some embodiments, the projections 370 act as a hinge and the pivot axis may correspond to an axis that that extends between the centers of the projections 308. In some cases, the pivot axis aligns with the longitudinal axis 312 of the housing 302. The pivot axis may be substantially parallel to the longitudinal axis 312 of the housing 302, substantially perpendicular to the lateral axis 314 of the housing 302, and/or substantially perpendicular to the central axis 316. In some cases, the support 304 may be both slidably and pivotably connected to the housing 302 such that the support 304 may move along the sliding axis and be pivoted about the pivot axis or axes. In some such cases, the location of the pivot axis may be dependent upon the position of the support 304, for example along the lateral axis 314 and/or other axes.

The sensor assembly 202 may further include one or a plurality of sensors. The plurality of sensors may include, but are not limited to, an inertial measurement unit (IMU) 322, an electromyography (EMG) sensor 371, or a displacement or force/pressure sensor shown as a magnetic sensor 378. When embodied as a magnetic sensor, the sensor 378 may be a Hall effect sensor. The magnetic sensor may be a device for detecting and measuring magnetic field strength or magnetic flux densities. In some embodiments, the magnetic sensor 378 may be any of a variety of distance or displacement sensors that senses displacement, such as a change in position and/or orientation, of the user’s muscle relative to a starting position and/or orientation of the muscle. The sensor 378 may be a force or pressure sensor, as further described herein. The sensor assembly 202 may further include a circuit board 324. The circuit board 324 may be in electrical communication with the plurality of sensors to receive sensor data. The sensor assembly 202 may further include a tray 326, as further described.

The IMU 322 may detect velocities, accelerations, angular rates, orientations, and other movements of the limb of the user. The IMU 322 may detect a vibromyography (VMG) signal associated with a muscle of the user. The IMU 322 may be coupled to and in electrical communication with the circuit board 324. The IMU 322 may include at least one of an accelerometer, a gyroscope, or a magnetometer. The IMU 322 may include an accelerometer, a gyroscope, and/or a magnetometer. The IMU 322 may include only an accelerometer and a gyroscope. In some cases, the IMU 322 can be used as a motion sensor or absolute orientation sensor. For example, the IMU 322 can be implemented as a 9-axis IMU (e.g., 3-axis gyroscope; 3-axis accelerometer; 3-axis magnetometer).

The EMG sensor 371 may include one or more EMG pickups 374 carried by the support 320. In this way, the one or more EMG pickups 374 move with the support 320 as the support 320 moves (e.g., moving along the sliding axis and/or pivoting about the pivot axis). The one or more EMG pickups 374 are configured to contact skin of a user. For example, an outer surface of the support 320 and the one or more EMG pickups 374 may be configured to contact the residual limb or the sound limb of the user. The EMG sensor also includes one or more electrical contacts 376 coupled to the circuit board 324 and extending downward therefrom to contact the EMG pickups 374 when assembled. The contacts 376 and EMG pickups 374 may detect EMG signals and communicate the signals to the circuit board 324.

The magnetic sensor 378 may detect or measure one or more magnetic field strengths. In some cases, the magnetic sensor 378 may detect the mechanical response of a muscle of the user during movement of the muscle. For example, muscle movement may be read by magnetic flux density proximity and direction from a magnet 380 to the magnetic sensor 378. The magnet 380 may generate a magnetic field strength and the magnetic sensor 378 may detect the magnetic field strength generated by the magnet 380. The magnet 380 may be attached with the housing 302. The magnetic sensor 378 may be implemented as a Hall effect sensor or magnetometer. The magnetic sensor 378 may be attached with the circuit board 324 or the support 320. In some cases, the magnetic sensor 378 and the magnet 380 are separated by a set distance. In some embodiments, the magnetic sensor 378 may be replaced by an MMG sensor or any type of distance or displacement sensor or detector that detects the distance between the housing 302 and the support 304. For example, a visual sensor such as LIDAR, etc. may be used to detect the variable distance.

The sensor assembly 202 may further include the tray 326. The tray may be positioned between the support 304 and the housing 302. As shown, the tray 326 may be attached to the support 304 and may be moveably attached with the housing 302. For example, ends of the biasing member 372 may be coupled to the tray 326 housing 302. In some cases, the tray 326 is pivotably and/or slidably connected to the housing 302. For example, the tray 326 may move as the support 304 moves. The circuit board 324 may be located underneath the tray 326 as shown, or the circuit board 324 may be in other locations, such as above the tray 326.

FIG. 4A is a perspective view of another embodiment of a sensor assembly 203. FIGS. 4B and 4C show an exploded view and a cross-sectional view, respectively, of the sensor assembly 203. The sensor assembly 203 may have the same or similar features and/or functions as the sensor assembly 202, and vice versa, except as otherwise described. Further, any features or functions of the sensor assembly 203 may be incorporated into any other embodiments of sensor assemblies described herein, such as those shown in FIGS. 12A-18C, and vice versa.

As illustrated in FIGS. 4A-4C, the sensor assembly 203 may include the housing 302, the support 304, the circuit board 324, the tray 326, the biasing member 372, and the plurality of sensors. The plurality of sensors may include an IMU 322, an EMG sensor (EMG pickups 374), and a magnetic or sensor 378. The EMG pickups 374 may extend below the lower plane of the support 304 when assembled. In some embodiments, the electrical connects 376 may extend below this plane. The EMG sensor may thus ensure contact with the skin in this manner.

The sensor assembly 203 also includes an axle 404. The axle 404 may be an elongated structural member extending longitudinally, for example parallel to the longitudinal axis. The axle 404 may be cylindrical, and by solid or hollow. The axle 404may facilitate a pivoting movement of the support 304 relative to the housing 302. For example, the axle 404 may engage with complementary engagement cavities 402 located at longitudinal ends of the housing 302. In some such cases, the axle 404 acts as a hinge such that the support 304 may pivot about a pivot axis 412 that is centered along the axle 404. In some cases, the pivot axis 412 is the longitudinal axis of the housing 302. The pivot axis 412 may be substantially parallel to the longitudinal axis of the housing 302 or substantially perpendicular to a lateral axis of the housing 302.

FIGS. 5A-5D illustrate various configurations of the sensor assembly 202. The various configurations of the sensor assembly 202 described with respect to FIGS. 5A-5D may apply to the other embodiments of the sensor assemblies described herein such as the sensor 203, the sensor assemblies shown in FIGS. 12A-18C, etc. As described herein, the support 304 may be slidably connected to the housing 302 such that the support 304 may be slid, shifted, pushed, or otherwise translated along the central axis 316 (sometimes referred to as the sliding axis). FIGS. 5A and 5B illustrate example orientations of the sensor assembly 202 in which the support 304 has been shifted along the central axis 316. In particular, FIG. 5A illustrates the support 304 shifted along the central axis away from the housing 302 in a first proximal direction 502, and FIG. 5B illustrates the support 304 shifted along the central axis towards the housing 302 in a second distal direction 504. In some cases, the position of the support 304 illustrated in FIG. 5A may be a default resting position of the support 304. For example, at least one biasing member may be disposed between the housing 302 and the support 304 and may bias the support 402 in the first direction 502 (or bias the housing 302 in the second direction 504). The range of motion of the support 304 along the central axis 316 may vary across embodiments. For example, the range of motion of the support 304 along the central axis 316 may be limited by mechanical end stops of the housing 302 or the support 304.

The support 304 may be pivotably connected to the housing 302 such that the support 304 may be pivoted or partially rotated about a pivot axis. FIGS. 5C and 5D illustrate example orientations of the sensor assembly 202 in which the support 304 has been pivoted about the longitudinal axis 312 of the housing 302. In particular, FIG. 5C illustrates the support 304 rotated about the longitudinal axis 312 in a third rotational direction 506, where the third rotational direction 506 corresponds to a counterclockwise direction as oriented in the figure, and FIG. 5D illustrates the support 304 rotated about the longitudinal axis 312 in a fourth rotational direction 508, where the fourth direction 508 corresponds to a clockwise direction as oriented. The support 304 may also pivot about the lateral axis 314 in a similar manner as described with respect to rotation about the longitudinal axis 312.

The sensor assembly 202 thus provides for movement of the support 304 relative to the housing 302 with multiple degrees of freedom. The support 304 may have any or all of the various degrees of freedom described herein. For example, the support 304 may have three degrees of freedom including translation along the longitudinal and central axes 312, 316 as well as rotation about the longitudinal axis 312. In some embodiments, the support 304 may have one, two, three, four, five, or six degrees of freedom. For example, the support 304 may in addition translate along the lateral axis 314 and/or rotate about the lateral and/or central axes 314, 316. The plurality of degrees of freedom of movement of the support 304 relative to the housing 302 enhances tracking of the user’s limb and increases accuracy of the detected electrical and mechanical signals. In some embodiments, the support 304 may only translate axially along a single axis relative to the housing 302, and not rotate about any axis or translate about any other axis.

As shown, when rotated or pivoted about the pivot axis (which, in this case is the longitudinal axis 312), one side of the support 304 moves further away from the housing 302 while the other side of the support 304 moves closer to the housing 302. This movement causes a volume or area of the enclosure between the support 304 and the housing 302 to expand on one side of the sensor assembly 202 (i.e., the side at which the support 304 moves further away from the housing 302) and reduce on the other side of the sensor assembly 202 (i.e., the side at which the support 304 moves closer from the housing 302).

The permitted movement of the support 304 between the various orientations shown in FIGS. 5A-5D allows the support 304 to move congruously with the residual limb or the sound limb to ensure good contact with the residual limb or the sound limb is maintained. In this way, the sensor assembly 202 increases the likelihood that one or more of the plurality of sensors of the sensor assembly 202 may maintain a reliable contact with the skin and thus provide a reliable signal. For example, consistent contact between the sensor assembly 202 and the residual limb or the sound limb may aid in the EMG sensor in maintaining a reliable signal, may ensure the magnetic sensor accurately tracks movement of the muscle, etc.

FIG. 6 illustrates a block diagram of an example prosthetic system 600 including a sensor assembly 602. The sensor assembly 602 may have the same or similar features and/or functions as any of the embodiments of the sensor assemblies described herein, such as the sensors 202, 203, etc., and vice versa. In the illustrated example, the prosthetic system 600 includes an arm 620 attached to a fitting 640. The prosthetic system 600 also includes a prosthetic device 630 attached to the fitting 640. The prosthetic system 600 may be an embodiment of the lower arm prosthetic system 100 of FIGS. 1A or 1B. The fitting 640 may be an embodiment of the fitting 200 of FIGS. 2A or 2B. The prosthetic device 630 may be an embodiment of one or more of the prosthetic hand 102, the prosthetic wrist 104, the prosthetic lower arm 106, the lower arm stump 112, or the prosthetic digits 120 of FIGS. 1A or 1B.

The arm 620 may be a prosthetic arm or may be a natural arm, i.e. a natural or sound human arm, or a stump. The fitting 640 may be attached to a distal end of the arm 620. The prosthetic device 630 may be a prosthetic arm or hand attached to the fitting 640.

The prosthetic system 600 may include a controller 622, a power supply 624, a data store 626, and one or more of the sensor assemblies 602. Although illustrated as being part of the fitting 640, any one or more of the controller 622, power supply 624, data store 626, or sensor assembly 602 may be located in a number of locations including any location in or on the prosthetic device 630, remote from the fitting 640 or the prosthetic device 630 or the like. Furthermore, it will be understood that some of the separate components may be combined in a variety of ways to achieve particular design objectives. For example, in some cases, the fitting 640 may be combined with the prosthetic device 630. As another example, the data store 626 may be combined with controller 622 components to save cost and/or improve performance.

The sensor assembly 602 may be an embodiment of the sensor assembly 202 described herein. The sensor assembly 602 can be configured to sense muscle activity. For example, the sensor assembly 602 can include one or more sensors that are sensitive to force, distance/displacement, potential difference (e.g., voltage), vibrations, sound/acoustic readings, or another symptom of muscle activity. As another example, the sensor assembly 602 may include any one or any combination of an EMG sensor 604, a magnetic sensor 606, an IMU 608, a voltage sensor 610, an acoustic sensor 612, or one or more other sensors 614. Furthermore, the sensor assembly 602 may include a circuit board 616, such as the circuit board 324 of FIG. 3B.

Data from the sensor assembly 602 (e.g., data from the EMG sensor 604, magnetic sensor 606, IMU 608, the voltage sensor 610, the acoustic sensor 612, or the one or more other sensors 614) may be received by the circuit board 614 or the controller 622 and may be used to determine various parameters associated with the prosthetic system 600. For example, the controller 622 may translate input signals (e.g., an EMG signal, a magnetic field strength signal, distance signal, an IMU signal, etc.) received from the sensor assembly 602 into a prosthetic movement. For example, the data store 626 may store an operating profile that includes a plurality of hand gestures or hand grips. The operating profile may associate each of the plurality hand gestures or hand grips with a corresponding input signal, set of input signals, or combination of input signals such that the controller 622 may select a hand gesture or a hand grip from the operating profile based at least in part on the input signals.

FIG. 7 is a table 700 illustrating various example associations between input signals and hand gestures or hand grips. The various associations and control techniques may be used with any of the embodiments of sensor assemblies described herein. The table 700 includes four columns: user movement, muscle contraction, forearm rotation, and classification. The “user movement” column includes images showing a sound hand/arm performing a particular hand gesture or hand grip. The “muscle contraction” column illustrates various input signals associated with each of four measurement sites: Site 1 (fitted to the internal side of the arm aligned with the forearm flexor muscles), and Sites 2-4 (fitted to the external side of the arm aligned with the forearm extensor muscles). The “forearm rotation” column illustrates the forearm movement associated with the particular hand gesture or hand grip. The “classification” column is an identifier of the hand gesture or a hand grip, such as the name to which it is commonly referred.

The first row 702 of the table 700 corresponds to the hand gesture “hand open.” As indicated by the muscle contraction and forearm rotation columns, the controller 622 will translate input signals at Site 1 and no forearm rotation into this hand gesture. The second row 704 of the table 700 corresponds to the hand gesture “hand close.” As indicated by the muscle contraction and forearm rotation columns, the controller 622 will translate input signals at Site 3 and no forearm rotation into this hand gesture. The third row 706 of the table 700 corresponds to the hand gesture “wrist supination.” As indicated by the muscle contraction and forearm rotation columns, the controller 622 will translate input signals across Sites 1-4 and a clockwise forearm rotation into this hand gesture. The third row 706 of the table 700 corresponds to the hand gesture “wrist pronation.” As indicated by the muscle contraction and forearm rotation columns, the controller 622 will translate input signals across Sites 1-4 and a counter-clockwise forearm rotation into this hand gesture.

FIG. 8 illustrates graphs of various example input signals. In particular, FIG. 8 illustrates an EMG graph 802, an MMG graph 894, an accelerometer graph 806, a gyroscope graph 808, and an active mask graph 810.

FIGS. 9A-9D illustrate graphs of various example features extracted from data received from a sensor of an example sensor assembly, such as the sensor assembly 202 or 203, or the sensor assemblies shown in FIGS. 12A-18C. For example, FIG. 9A illustrates example mean absolute values of EMG signals. FIG. 9B illustrates example mean absolute values of MMG signals captured with a Hall Effect sensor in response to magnet movement. FIG. 9C illustrates example mean absolute values of accelerometer readings. FIG. 9D illustrates example peak values from gyroscope readings. Such features and other data can be communicated to a pattern recognition classifier to determine movement data. For example, the movement data can provide an indication or classification of an intended movement. This movement classification can be decoded into motor control signals to drive the prosthetic hand and wrist (e.g., thumb rotation, wrist rotation, hand flexion/extension, etc.).

In some embodiments, the prosthetic system can include one or more grommets for attaching one or more of the sensor assemblies described herein to a POD or a component thereof, such as a sleeve. For example, as described herein, a grommet can be secured to the POD and a sensor assembly can be removably inserted into the grommet. In this way, the sensor assembly can be positioned in a predetermined or fixed location, relative to the POD or the user’s limb. A prosthetic system can include a grommet for each sensor assembly.

FIGS. 10A-10C illustrate respectively a top perspective view, a bottom perspective view, and a side view of an embodiment of a grommet 1000. The grommet 1000 is configured to attach to a POD or component thereof, such as the lower arm prosthetic system 100 of FIGS. 1A or 1B, or a fitting that attaches to a POD, such as the fitting 200 of FIGS. 2A-2B. The grommet 1000 may be formed of plastics, metals, composites, polymers, other suitable materials, or combinations thereof.

The grommet 1000 is configured to removably receive any embodiment of a sensor assembly described herein, such as the sensor assembly 202 of FIGS. 3A-3B, or the sensor assembly 203 of FIGS. 4A-4C, or the sensor assemblies of FIGS. 12A-18C. The grommet 1000 may define an opening 1014, and the grommet 1000 can be configured to receive a sensor assembly at least partially through the opening 1014. The size of the opening 1014 can vary across embodiments. In some cases, the opening 1014 can be sized to fit the sensor assembly. For example, a shape or size of the opening 1014 can match a shape or size of the sensor assembly. In some cases, the opening 1014 can be larger than the sensor assembly. In some cases, the size of the opening 1014 varies across the length or width of the grommet 1000. For example, the opening 1014 may be larger on the upper portion 1001 of the grommet 1000 and smaller at the lower portion 1003 of the grommet 1000. In some cases, such a configuration allows the sensor assembly to be wedged or snuggly fit into the opening 1014 by inserting the sensor assembly into the opening 1014 from the side of the grommet 1000 that corresponds to the upper portion 1001. In some cases, the grommet 1000 may be configured to receive the sensor assembly 202 or 203 when the sensor assembly is without the pair of end portions 362.

The grommet 1000 includes an upper portion 1001, which forms a portion of an outer structure of the grommet 1000. The upper portion 1001 includes a rim 1004 at least partially surrounding an entrance to the opening 1014. The rim 1004 can be curved. The shape of the rim 1004 can vary across embodiments. In some cases, the shape of the rim 1004 matches a shape the entrance of the opening 1014. The rim 1004 can extend away from an upper surface 1004 of the grommet via an extension portion 1006. The upper surface 1004 of the grommet can extend radially about the rim 1004. The upper surface 1004 can be relatively flat.

The grommet 1000 includes an internal socket 1010 configured to receive a sensor assembly. The internal socket 1010 can include an attachment portion 1012 for facilitating securement of the sensor assembly to the grommet 1000. The attachment portion 1012 can be a t-shaped attachment portion. The attachment portion 1012 can extend from or be connected to an internal side of the internal socket 1010. The internal socket 1010 can be positioned such that it is at least partially surrounded by the rim 1004. In some cases, the grommet 1000 can include a groove 1008 between the internal socket 1010 and the rim 1004 or the extension portion 1006.

The grommet 1000 includes a lower portion 1003, which forms a portion of an outer structure of the grommet 1000. The lower portion 1003 includes a lower surface 1016. The lower surface 1016 can be relatively flat or convex. The lower surface 1016 can define an entrance to the opening 1014, described above.

FIG. 11 illustrates a cross sectional representation of a side view of an example sensor assembly 1126 removably inserted into an example grommet 1100 and the grommet 1100 attached to a socket 1130 of a POD. The sensor assembly 1126 can be an embodiment of the sensor assembly 202 of FIGS. 3A-3B or sensor assembly 203 of FIGS. 4A-4C. The grommet 1100 can be an embodiment of the sensor assembly grommet 1000 of FIGS. 10A-10C.

As shown, the sensor assembly 1126 is positioned within the grommet 1100 such that a support 1128 of the sensor assembly 1106 is substantially flush with a lower portion 1103 of the grommet 1100 and is positioned to contact a user of the POD (e.g., contact the user’s skin 1120). Furthermore, the grommet 1100 is positioned within a socket 1104 of a POD. For example, the socket 1104 can define an aperture 1140 through which the grommet 1100 can be inserted or attached. When attached through the aperture 1140 of the socket 1104, the grommet 1100 can coupled to the socket 1104 by sandwiching a portion of the socket 1130 between the rim 1104 of the grommet and an upper surface 1102 of the grommet such that the grommet 1100 is secured to the POD. The support 1128 may be biased toward the user’s skin by the springs, as described herein, to ensure contact with the user’s limb. The support 1128 may move distally or away from the user in response to muscle movements, thus causing the support 1128 and any sensors attached thereto, such as EMG pickups, accelerometers, Hall Effect sensors, magnets, other distance-detection components, etc. as described herein, to correspondingly move in that direction as well.

With reference to FIGS. 12A and 12B, another example embodiment of a sensor assembly 1200 is disclosed. The sensor assembly 1200 may include any of the features or functions of any other sensor assembly described herein, and vice versa, except as otherwise described. The sensor assembly 1200 may include a grommet 1210, a housing 1220, a magnet 380, springs 372, the circuit board 324, a support 1260, and/or an O-ring 1270. The grommet 1210 may removably receive the housing 1220, which can include the magnet 380, the springs 372, and the circuit board 324. In some embodiment, the circuit board 324 is mounted to the support 1260 and removably inserted into the housing 1220. The support 1260 may receive the O-ring 1270 and be removably coupled to the housing 1220. The support 1260 may define the central axis 316, which may intersect the support 1260. The axis 316 may be perpendicular to a plane defined by an upper surface of the support 1260. The support 1260 may translate along the central axis 316 in response to forces applied by a user’s limb. The housing 1220 and support 1260 may include any of the features or functions of any other housing and support, respectively, described herein, such as the housing 302 and support 304, respectively, and vice versa. For example, the support 1260 may be an overmold, have sidewalls connected to an end wall or upper surface, etc. as further described. Further, the support 1260 may include any of the sensors described herein, and the movement of the support 1260 in response to user muscle movement may generate the various sensor outputs described herein for control of the prosthetic.

FIGS. 13A-13D illustrate various views of the grommet 1210. The grommet 1210 may include any of the features or functions of any other grommet described herein, such as the grommet 1000, and vice versa, except as otherwise described. The grommet 1210 may include a bottom portion 1300 and a top portion 1302. The bottom portion 1300 may be rectangular in shape. The top portion 1302 may be oval or circular in shape. The bottom portion 1300 may include apertures 1306 that are formed on a sidewall (or sidewalls) of the bottom portion 1300. Additionally, the bottom portion 1300 may include an opening 1308 formed on a sidewall of the bottom portion 1300. The opening 1308 may allow wires from the circuit board 324 to extend through. Additionally, the bottom portion 1300 may include openings 1304 formed through a bottom surface of the bottom portion 1300 and partially through the adjacent sidewall. The openings 1304, 1306 may, for example, let air flow through when the housing 1220 is inserted into an opening 1310 of the bottom portion 1300.

The cavity or opening 1310 of the bottom portion 1300 may have a shape that corresponds to a shape of the housing 1220. In the illustrated embodiment, the opening 1310 and the housing 1220 are both rectangular in shape. The bottom portion 1300 may comprise a bottom surface and sidewalls extending upwardly therefrom. The upper portion 1302 may include a flange that extends radially outwardly away from the central axis and have a lower surface configured for contacting an outer surface of a fitting worn by the user. The outer edges of the upper portion 1302 may be located farther outward from the central axis than one or more of the sidewalls of the bottom portion 1300. The planform of the flange (as viewed from the top/bottom, e.g., see FIG. 13D) may be rounded, e.g. elliptical or oval as shown, or other shapes, such as circular, square, rectangular, etc. The bottom portion 1300 may have other shapes, such as square, segmented, polygonal, rounded, oval, circular, etc.

FIGS. 14A and 14B illustrate the housing 1220 and support 1260, and internal electronics and components, assembled together and without the grommet 1210. In some embodiments, the sensor assembly may include the housing 1220, support 1260, and internal electronics/components and not include the grommet. FIGS. 14A and 14B show different axial positions of the support 1260 along the central axis 316 with respect to the housing 1220. In FIG. 14A, the support 1260 is in a first position (that is, a raised position relative to the housing 1220) and in FIG. 14B, the support 1260 is in a second position (that is, a lowered or depressed position relative to the housing 1220). In some embodiments, a proximal-most portion of the support 1260 may protrude beyond a proximal-most portion of the housing 1220 in the raised position, so that the support 1260 is exposed beyond the housing 1220. In some embodiments, the support 1260 may not protrude proximally beyond the housing 1220 in the raised position.

The springs 372 may be positioned between the housing 1220 and the support 1260 to allow the support 1260 to move between the first position and the second position. In some embodiments, the springs 372 may bias the support 1260 to the first position (that is, the raised position). For example, the support 1260 may move to the second, lowered position when an external force is applied (for example, towards the housing 1220) and may move towards the first, raised position when the external force is removed. When the external force is applied, the springs 372 positioned between the support 1260 and the housing 1220 may be compressed and store elastic potential energy. When the external force is removed, the stored elastic potential energy of the springs 372 may be released and converted into a kinetic energy, pushing (or urging) the support 1260 towards the first position (that is, the raised position). The support 1260 may uniformly depress as shown in FIG. 14B, for instance where each spring is compressed a similar or same amount. In some embodiments, the support 1260 may depress different amounts at different locations of the support 1260, for instance where one or more of the springs compresses less or more than one or more of the other springs. The support 1260 may therefore by angled when depressed. Further, depending on the user, fitting and alignment when assembled, the support 1260 may or may not be uniform or level in the raised, first position shown in FIG. 14A. For instance, the support 1260 may be partially depressed in the first, raised position shown in FIG. 14A, and then depress further when acted on by a user’s muscle to take a second, even more depressed position shown in FIG. 14B. Thus, any movement between the raised and depressed position may be relative to the other, previous position.

FIGS. 15A and 15B show the support 1260 in isolation. The support 1260 may include a body 1502 having sidewalls 1503 connected to an upper surface 1512. The body 1502 may include one or more electrode surfaces 1500A, 1500B, 1500C. The positions of the electrode surfaces 1500A, 1500B, 1500C may correspond to positions of pins 1804 (see FIG. 18C). The electrode surfaces 1500A, 1500B, 1500C may be formed on the upper surface 1512 of the body 1502.

The body 1502 may include one or more locking devices 1506. There may be two locking devices 1506 located at opposite longitudinal ends of the body 1502 as shown (only one is visible in the figure as oriented). The locking device 1506 may include a first portion 1504A, such as a tab, extending downward from a side edge of the upper surface 1512 of the body 1502. The locking device 1506 may include a second portion 1504B (for example, a detent or flange) extending outward (for example, away from the body 1502) from a distal end of the first portion 1504A. The first portion 1504A may flex to allow the second portion 1504B to engage with an opening or recess in the housing 1220. The body 1502 may include a groove 1510 that may receive the O-ring 1270. The groove 1510 may be an inward recess of the body 1502 extending around the perimeter of the sidewalls 1503. The groove 1510 may be located in the sidewalls 1503 near the upper surface 1512. It is understood that the use of “upper,” “lower,” and the like is for sake of description only, and does not limit the scope of the disclosure. For example, when the sensor assembly is used, the “upper” surface 1512 may be facing upward, downward, or to the side, depending on the location of the sensor assembly on the user and fitting.

FIGS. 16A and 16B show top perspective and cross-section perspective views, respectively, of the housing 1220 in isolation. The housing 1220 may include a cavity 1600, protrusions 1602, cutouts 1604, a slot 1606, and/or openings 1608. The cavity 1600 may be formed by one or more sidewalls 1620 of the housing 1220 and may be dimensioned and shaped to slidingly receive the body 1502 (see FIG. 15A) of the support 1260. The sidewalls 1620 may extend away from a floor 1601. As shown in the illustrated embodiment, the cavity 1600 may be substantially rectangular in shape. In some embodiments, the cavity 1600, and/or the floor 1601 and sidewalls 1620, may have any of the shapes or configurations described herein with respect to the support 1260, such as square, rounded, segmented, etc. The protrusions 1602 may be upwardly extending columns or poles that receive the springs 372 slidingly over the protrusions 1602. In the illustrated embodiment, the protrusions 1602 may be positioned at the corners of the housing 1220 such that the springs 372 are positioned at the corners of the housing 1220 (see FIG. 18C). There may be four protrusions 1602 and springs 372 as shown, or there may be one, two, three, five, six, seven, eight or more of the protrusions 1602 and springs 372.

The cutouts 1604 may be formed on an inner surface of the one or more of the sidewalls 1620 of the housing 1220 that form the cavity 1600. The cutouts 1604 may extend completely through the sidewall 1620 such that it forms an opening through the sidewall 1620. The cutout 1604 may extend only partially into the sidewall 1620 such that it forms a recess in the inner surface of the sidewall (see, e.g., FIG. 16B). The cutouts 1604 may receive the second portion 1504B of the locking device 1506 of the support 1260. The cutouts 1604 may include a top edge 1604B and a bottom edge 1604A, which together can limit vertical movement of the second portion 1504B of the locking device 1506. For example, the second portion 1504B may be positioned between the top edge 1604B and the bottom edge 1604A. The top edge 1604B may contact an upper surface of the second portion 1504B to limit movement of the support 1260 away from the housing 1220, for example in the raised position. The bottom edge 1604A may contact a lower surface of the second portion 1504B to limit movement of the support 1260 toward the housing 1220, for example in the depressed position.

The openings 1608 may allow wires from the circuit board 324 to extend through and out of the housing 1220. The slot 1606 may receive and secure a component such as a magnet, for example the magnet 380 shown in FIG. 3B. The magnet 380 may be removably inserted into the slot 1606.

As shown in FIG. 16B, the O-ring 1270 may be positioned between the groove 1510 of the support 1260 and the inner surface of the sidewalls of the housing 1220. The contact between the O-ring 1270 and the groove 1510, and the contact between the O-ring 1270 and the inner surface of the sidewalls 1620 of the housing 1220 may create a seal between the housing 1220 and the support 1260 to prevent unwanted substances (for example, water or dust) from traversing the space between the housing 1220 and the support 1260 and entering the enclosure 1650 formed by the housing 1220 and support 1260 to prevent contamination of the electronics therein. The O-ring 1270 may be any type of sealing material, such as rubber, plastic, etc. The O-ring 1270 may have a circular shaped cross-section as shown, or any other shape, such as elongated, planar, etc. The O-ring 1270 may be located axially such that it provides a seal with the housing 1220 and support 1260 in any of the relative raised or depressed positions. The O-ring 1270 may therefore slide along the inner surface of the sidewalls 1620 of the housing 1220 as the support 1260 moves. There may be a single O-ring 1270 as shown, or there may be two, three, four or more O-rings 1270 located vertically separated from each other or adjacent each other.

The second portions 1504B of the locking device 1506 may extend into the cutouts 1604. As the support 1260 moves up and down along the central axis within the cavity 1600 of the housing 1220, the cutouts 1604 may facilitate and limit the movements (for example, upwards and downwards) of the second portions 1504B. The dimensions of the cutouts 1604 (for example, heights of the cutouts 1604) may determine the relative maximum positions of the support 1260 with respect to the housing 1220. For example, when the support 1260 is in the first position (that is, raised position), the second portions 1504B can abut against the top edge 1604B of the cutout 1604 as shown in FIG. 16B. The O-ring 1270 may seal off the lateral spaces between adjacent sidewalls of the housing 1220 and support 1260 in this raised position. When the support 1260 is in the second position (that is, lowered or depressed position), the second portions 1504B can abut against the bottom edge 1604A of the cutout 1604. The bottom edge of the cutout 1604 can prevent further downward movement of the support 1260 with respect to the housing 1220. The O-ring 1270 may seal off the lateral spaces between adjacent sidewalls of the housing 1220 and support 1260 in this depressed position.

FIG. 17A shows a close-up, perspective view of a corner portion of the groove 1510 of the support 1260. Corner portions of the groove 1510 may include one or more ridges 1700 protruding outwardly from within the groove 1510. In the illustrated embodiment shown in FIG. 17A, the groove 1510 may include two ridges 1700 extending axially and spaced apart. There may be three, four, five or more ridges 1700.

FIG. 17B illustrates a top, cross-sectional view of a corner portion of the groove 1510, showing cross-sectional shapes of the ridges 1700. As shown in FIG. 17B, the ridge 1700 may have a semi-circular cross-section shape, and/or have rounded edges. The ridge 1700 may be positioned at or about the corners of the groove 1510, and cause the O-ring 1270 to stretch a bit more at the corners. In some embodiments, the ridge 1700 can cause the O-ring 1270 to protrude outwards a bit to tighten the O-ring 1270 and ensure a secure fit around the groove 1510. Contact between the housing 1220 and the O-ring 1270 may, as shown in FIG. 17C, leave a gap 1702 between the housing 1220 and the body 1502 of the support 1260 to reduce friction during movement.

With reference to FIGS. 18A-18C, the support 1260 can receive the circuit board 324 and other electronic components. In some embodiments, the support 1260 may include the electronic components as described herein with respect to the sensor assembly 202. Further, the pins 1804 (see FIG. 18C) positioned on the circuit board 324 may extend through recesses or apertures 1800 in the upper surface 1512 and contact the electrode surfaces 1500A, 1500B, 1500C when assembled. The support 1260 can include protrusions 1802 that are positioned at the corners (see FIG. 16B). The protrusions 1802 may receive the springs 372 and secure them in place during use of the sensor assembly 1200. The protrusions 1802 of the support 1260 and the protrusions 1602 of the housing 1220 may be aligned when assembled. For example, the protrusions 1802 may be positioned above the protrusions 1602 when the support 1260 is placed (or inserted) within the cavity 1600 of the housing 1220. As such, the protrusions 1802 and the protrusions 1602 can together secure opposing ends of the springs 372 in place during use. To assemble the sensor assembly with the electronics and sensors therein, the first portions 1504A of the locking devices 1506 may be flexed inward to slide the support 1260 and housing 1220 together, and the first portions 1504A may then flex outward when located within the cutouts 1604.

Calculation of Muscle Movement From Magnetic Field Strength or Magnetic Flux Density Measurements

Embodiments of the sensor assemblies above may be utilized to calculate muscle movement using a displacement sensor. In some embodiments, muscle movement is calculated from a magnetic field strength or magnetic flux density measurement. In some embodiments, this can be achieved using a combination of a magnet and a magnetometer (e.g., a magnetic field strength or a magnetic flux density sensor, or any type of magnetic sensor) to measure relative movement between the moveable parts of the sensor assembly. Depending on the strength of the magnet, the relationship between relative displacement (i.e., distance between the magnet and the magnetometer) and the change in measured magnetic field strength or magnetic flux density can be calibrated stepwise or approximated through different simplifications. For example, magnets of a strength that result in an approximately quadratic relationship between change in distance and change in magnetic field strength or magnetic flux density may be used, and this relationship can then be approximated by a quadratic equation.

To calculate displacement from magnetic field strength or magnetic flux density, magnetic field or magnetic flux can be measured at two known distances. For example, a measurement may be taken at 0 mm displacement (i.e., the maximum possible distance between the magnet and magnetometer) and a 1 mm displacement (i.e., the magnet is moved towards magnetometer by 1 mm). These two reference points would suffice to fit a simplified quadratic equation in order to define a relationship between muscle displacement and magnetic field strength or magnetic flux density. The simplification is to assume a function of the form f(x)=ax²+bx+c where b=0, so that f(x)=ax²+c, leaving only two unknowns, “a” and “c”, that can be calculated from the two calibration measurements. To fully approximate a quadratic curve (i.e., assuming b ≠ 0), a third calibration point can be obtained, for example at 2 mm or 3 mm displacement. This method can be repeated with additional calibration points to further define the quadratic relationship, with there being no limit to the number of calibration points used. The calibration points can also be used for approximations with other suitable functions like piecewise polynomials or splines.

A sensor assembly may be calibrated as described above to determine an equation or formula to be applied when the sensor assembly is utilized. For example, when the sensor assembly is provided within a socket fitted onto a user’s limb, measurements of the change in magnetic field strength or magnetic flux density are taken by the magnetometer due to muscle movement causing the relative distance between the magnet and the magnetometer to change. These magnetic field strength or magnetic flux density measurements can be converted to displacement measurements utilizing the formula or equation determined during calibration. Processing of the measurements can be done with a processor within the sensor assembly, or the measurements can be transmitted to a remote processor for calculation, e.g., on a remote computer or a mobile device. The sensor assembly may further comprise other sensors as described above (e.g., EMG sensors) to obtain signals, readings or information that may be utilized in combination with the displacement measurements to assess muscle or other activity, as described further above and below.

Embodiments for Proportional Control and Baseline Correction

The apparatuses, methods and embodiments as described above may also be capable of achieving steadier speed control of a prosthetic system. The embodiments of the sensor system described herein may use a muscle displacement signal, derived from the change in magnetic field strength or magnetic flux density between a magnet and a magnetometer in the prosthetic socket, in order to achieve a better control signal for speed control of the prosthetic device.

Embodiments of the sensor system herein described may be considered a “hybrid sensor system” in that they incorporate two or more different types of sensors in the same assembly. Embodiments of the hybrid sensor system may be used to make a prosthesis less susceptible to unintended movement by using changes in magnetic field strength or magnetic flux density measured by a magnetometer to detect muscle displacement, in addition to the EMG sensor. Unlike an EMG sensor, magnetic sensor readings (whether raw or translated into a displacement measurement), are virtually immune to electrically noisy environments (e.g., power line interference). This in turn reduces the need in some embodiments for any digital filtering of the magnetic sensor or displacement signal.

Further, an EMG and a magnetic sensor measure muscle activity in intrinsically different ways. Therefore, when combining a displacement signal derived from changes in magnetic field strength or magnetic flux density and EMG features in applications such as pattern recognition, proportional control (with or without offset correction) or sensor fusion, a perturbation in one input is rarely seen simultaneously in the other input, making all algorithms more robust against unwanted movement. For example, a relative movement between skin and sensor might induce a voltage spike in the EMG signal, while having no effect on a Hall effect sensor/displacement reading. A sensor fusion algorithm, implemented for example as a Kalman filter, can be used to estimate the presence or level of noise in one sensor by combining readings from several sensors, resulting in a more accurate estimation of muscle activity than from a single sensor alone.

FIG. 19A shows an EMG signal 1901 with typical signal processing for proportional control measured from a prosthetic user flexing a muscle three times to control a prosthetic, resting the muscle before, after and between flexions. A signal strength range where the prosthetic limb is programmed to move at speeds proportional to the strength of the measured EMG signal 1901 is defined by an upper limit 1902 and a lower limit 1903. If the EMG signal 1901 is measured to be below the lower limit 1903, movement is stopped. If the EMG signal 1901 is measured to be higher than the upper limit 1902, proportional control is limited resulting in no further speed increase. FIG. 19B depicts a simultaneously recorded muscle displacement signal 1904 as measured with the hybrid sensor system described herein. While the EMG signal 1902 and the muscle displacement signal 1904 are similar in their general shape, the muscle displacement signal 1904 is much smoother. This advantageously results in the prosthetic limb moving at a far steadier speed, allowing for better control by the user. Not only is the muscle displacement signal 1904 less jittery, it also remains at a constant level, which better reflects user intent than the EMG signal 1902.

FIGS. 20A and 20B illustrate example control signal measurements from a user trying to maintain constant muscle force over a five second period. FIG. 20A depicts the measured EMG signal 2001 with typical signal processing for proportional control. FIG. 20B depicts the simultaneously recorded muscle displacement signal 2002 as measured by the hybrid sensor system described herein. While the measured EMG signal 2001 level drops significantly and shows jitteriness, the simultaneously recorded muscle displacement signal 2002 drops only marginally over the same time period, and is much smoother. In this example, the EMG signal 2001 drops by more than 50%, potentially dropping below the minimum signal threshold 2003 if continuing along the same trajectory. In contrast, the recorded muscle displacement signal 2002 does not drop, fluctuates by less than 10%, and remains steady and smooth during the same period.

An associated challenge of using muscle displacement as a control signal is that the baseline sensor output when there is no muscle activity and the usable range of travel (i.e. the displacement controllable by the user) is affected by a plurality of extrinsic factors. These factors may include, but are not limited to, limb volume changes, changes in elbow flexion angle when a limb is squeezed into a socket, external loads on the limb and/or socket, and inconsistent alignment between a limb and sensors during socket donning. As a result of such factors, using an unprocessed or unfiltered sensor output could result in unintended movement of the prosthetic.

To address this challenge, embodiments of the hybrid sensor system herein described may be used to estimate muscle activity or inactivity, and based on this assessment a baseline displacement measure can be calculated and subtracted from the displacement reading. This results in the proportional control algorithm remaining unaffected when the displacement is due to something else than muscle activity.

One non-limiting example method for estimating whether a muscle is active or inactive is to use a threshold on the EMG signal. If the EMG signal is above the threshold, the muscle may be classed as active. If the EMG signal is below the threshold, the muscle may be classed as inactive. The baseline displacement value is constantly updated and subtracted from the current reading, resulting in a readout of 0 mm displacement. When the EMG signal rises above the threshold, the baseline value is not updated anymore until it falls below the threshold again. The baseline is subtracted from new displacement readings, resulting in sensor readouts greater than 0 mm. This algorithm can be extended for example with timing constraints or other conditions, where the activation or deactivation is a combination of the EMG signals strength in relation to the threshold and the duration of the signal being above or below the threshold.

FIGS. 21A and 21B illustrate example graphs of sensor signals measured from a prosthetic user performing three muscle contractions typical for controlling a prosthetic device, where a dynamically updated baseline correction is applied to the muscle displacement signal. FIG. 21A illustrates the measured EMG signal 2101 and threshold 2102, and FIG. 21B shows the raw muscle displacement signal 2103, the dynamically updated baseline 2104 and the corrected displacement signal 2105 that results from subtracting the baseline 2104 from the raw displacement 2013. The dynamically updated baseline 2104 is based on whether the simultaneously measured EMG signal 2101 is above or below the threshold 2102. When the measured EMG signal is 2101 is below the threshold 2102, the baseline value 2104 is constantly updated and subtracted from the current reading, resulting in a readout of 0 millimeters. If the measured EMG signal 2101 is above the threshold 2102, then the muscle is classed as active, and the dynamically updated baseline value 2104 stops updating until the measured EMG signal falls below the threshold 2102. In the example depicted in FIG. 21B, the resulting corrected displacement signal is clipped at zero so as not to allow for negative displacement, but in other embodiments negative displacement can also be implemented depending on application.

If several sensors are used, this baseline correction can be applied in a way that the muscle activity that is determined by one or several sensors affects the dynamic baseline calculated for a different sensor. Instead of each sensor dynamically updating its displacement baseline as long as the EMG signal from that sensor is below a threshold, the displacement correction can be calculated in a centralized manner (e.g. a master microcontroller, for example in the controlled prosthetic), so that if any one sensor registers an EMG signal above the threshold, the displacement baseline is not updated on any sensor anymore. This combined offset correction can be used, for example, when using the displacement measurement for pattern recognition in order to reduce feature similarity.

FIGS. 22A and 22B are block diagrams showing example schematics of dynamic baseline correction in a socket system with two hybrid sensors such as the ones herein described. FIG. 22A depicts an example where baseline correction is implemented on each sensor individually. In other words, each sensor dynamically updates its displacement baseline as long as the EMG signal from that sensor is below a threshold. FIG. 22B illustrates an example of a combined baseline correction, where baseline stops updating for both the sensors, once any one sensor registers an EMG signal above the threshold. While the systems depicted in FIGS. 22A and 22B contain two sensors, these methods of combined baseline correction may also be used on systems with more than two sensors. Further, these methods of combined baseline correction may be used in systems containing distinct groups of sensors, for example, such as in a four-sensor system having a first group comprising the first and second sensors and a second group comprising the third and fourth sensors, and where the two sensors in each group work together in the described manner, but there is no link between the two groups.

Embodiments for Pattern Recognition

In further embodiments, the displacement signals measured by the hybrid sensor system may be used as a feature for improved pattern recognition control. This may be achieved by using displacement as a feature for pattern recognition, which advantageously allows for adding another pattern recognition feature without the need to increase the number of sensors or EMG features being measured. Prosthetists can therefore manufacture sockets as per their current training and workload, while gaining access to a multi-dimensional feature set for pattern recognition which is less correlated than with purely EMG-based features. This is preferable to other pattern recognition systems for prosthetics that require prosthetists to deviate from the common clinical practice of placing two sensors on a agonist-antagonist muscle pair.

FIGS. 23A and 23B show data captured by one sensor while a user performed five different distinct hand movements for pattern recognition. The graph depicted in FIG. 23A shows the strong correlation between two commonly used features of the EMG signal, the mean average value (MAV) and the waveform length (WFL). The appearance of an almost perfectly straight line indicates a strong linear correlation between the features, which is not desirable. In contrast, the graph depicted in FIG. 23B does not show the same line-like appearance, implying a far weaker correlation between MAV and muscle displacement across the five different hand and wrist movements, which is desirable for pattern recognition purposes.

In use, the combination of EMG signals and muscle displacement measurements form patterns that can be generalized and recognized by a pattern recognition algorithm. Each of the patterns may correspond to a desired movement to be executed based on a control signal transmitted once one of the patterns is recognized by a controller or processor. Movement of a prosthetic device connected to a socket including the hybrid sensor can be controlled based on determining whether the combined EMG signal and muscle displacement measurements satisfies one of the patterns.

Calculation of Muscle Movement From Force or Pressure Measurements

Some embodiments may have the same or similar features and/or functions as any of the embodiments of the sensor assemblies described herein, such as the sensors 202, 203, 602 etc., and vice versa except that they include one or more other sensors 614 embodied as one or more force or pressure sensors 614. For example, in such embodiments, the sensor assembly 602 may include an EMG sensor 604, an IMU 608, and one or more force sensors 614. One force sensor 614 may be placed in such a way that securing the sensor assembly 602 in a prosthetic socket results in a measurable signal indicating pressure when there is force against the EMG side of the sensor assembly 602. In some embodiments, several pressure or force sensors 614 may be used, and the sensors 614 may be placed in a socket of the POD in such a way that it is possible to measure not only the total force being applied against the sensor assembly 602, but also how centered or off-centered the force is being applied against the sensor assembly 602. In such embodiments, the sensor assembly 602 may only move vertically due to pressure being applied to the one or more force sensors 614.

“Force sensor” as used herein has its usual and customary meaning and includes, without limitation, devices configured to translate applied mechanical forces, such as tensile and compressive forces, into output signals whose value can be used to reflect the magnitude of the force. The signals may be received by a processor or controller to provide control over the POD and/or provide feedback on use of the POD. The force sensor may be, for example, a transducer, a force sensitive resistor (“FSR”), a force sensitive capacitor (“FSC”), a strain gauge, a load cell, and the like. In some embodiments the force sensor receives a force over a known area of a sensor contact surface, and which may be used to calculate a pressure.

“Pressure sensor” as used herein has its usual and customary meaning and includes, without limitation, devices for tactile pressure measurement and stated in terms of force per unit area. The pressure sensor may act as a transducer and generate an electrical signal as a function of the magnitude of the pressure imposed.

Further, a force sensor may include, or be used as part of, a pressure sensor. Thus, any description herein of a force sensor applies equally to use of a pressure sensor, unless otherwise indicated. Thus, the sensor assembly 602 may include an EMG sensor 604, an IMU 608, and one or more pressure sensors 614, etc. as described above. A pressure sensor may be used as an alternative to a force sensor. For example, a pressure sensor may substitute for the force sensor. As further example, a force sensor may be used, and then a pressure may be calculated based on the detected force and an area over which the force is applied. The force or pressure sensors may be used to measure a magnitude of a force or pressure and/or a presence of a positive force or pressure (any magnitude). The pressure sensor may be, for example, a capacitive pressure sensor. In another example, a gas-filled piston or compartment with an integrated pressure sensor may be used. Such a piston or compartment could replace one or all springs 372 in the design.

In a non-limiting example, one force sensor 614 may be placed in each of the four corners of a socket of the POD such that the sum of measured forces would equal the total force being applied. The relative proportion measured by each force sensor 614 would indicate if the force being applied is centered or off-centered. All the force sensors 614 measuring the same or similar force would indicate that the force is being applied centrally. Two neighboring force sensors 614 measuring the same or similar force that is higher than the force measured by the remaining force sensors 614 would indicate that the force is off-centered and applied on one axis. One force sensor 614 measuring a force that is higher than the force measured by the remaining force sensors 614 would indicate that the force is off-centered and applied on two axes.

Some embodiments do not require springs, as in some of the displacement-based embodiments described herein. For example, relative movement between both walls of the sensor assembly 202, 203, 602 may be ensured either by a sensor casing that is made up of two parts such as the housing 302 and support 304, or a casing material that is flexible enough that the force from the muscle movement deforms or flexes the force sensor(s) sufficiently and in such a way that the force can be measured by the integrated force sensor(s). Such embodiments advantageously benefit from a lower profile (i.e., it is thinner) than some of the displacement-based embodiments described herein. Such embodiments may further include a signal conditioning circuit that linearizes the output from the force sensor(s) 614 in cases where the force sensor(s) 614 (i.e., FSRs) have a non-linear response to force and can benefit from such a conditioning circuit. In some embodiments, the one or more force sensors 614 may be pressure sensors, as described.

In some embodiments, the sensor assembly with the force or pressure sensor has components that only move along a single axis. The support 304 and housing 302 may only move vertically with respect to each other. The support 304 may only translate relative to the housing 302. The parts may physically move or flex with respect to each other. In such embodiments, only a force or pressure sensor may be used, and no other type of sensor (e.g. EMG, IMU, magnetic, etc.) is used. In some embodiments, the sensor assembly include one or more force sensors (or one or more pressure sensors), and relative movement (sliding or flexing) may be along a single, vertical axis.

Other Embodiments and Aspects

Although many of the embodiments above are directed to upper limb prosthetics, other embodiments utilizing the sensor assembly can also be applied to lower limb prosthetic devices. FIG. 24 illustrates an example of a prosthetic knee 2401 and socket 2402 capable of housing the hybrid sensor systems described herein. The socket 2402 is configured to receive a lower limb and may include one or more sensor assemblies as described above. For example, the sensor assemblies may comprise hybrid sensors including at least a force or displacement sensor and an EMG sensor. Any of the embodiments as described herein may be applied to powered prosthetic devices comprising a powered or motorized actuator, or they may alternatively be applied to passive prosthetic devices that do not include a powered or motorized actuator.

It will be appreciated that the embodiments described throughout this specification may include more than one of the hybrid sensor assemblies herein described. For example, a socket may have two sensor assemblies on an agonist-antagonist muscle pair. Additional embodiments can have three, four, five, six, seven, eight or more sensor assemblies within the same socket. Accordingly, where reference is made to the fitting or socket containing “one sensor” or “the sensor”, it should be interpreted as “at least one sensor” or the “the sensors”.

Any calculations and/or determinations described above can be performed individually or by any combination of the sensor system described herein, the prosthetic or orthotic device, or another computing device. Although some of the embodiments described herein may describe one or more determinations and/or calculations being performed by a specific computer implemented method, the description is not limiting.

The sensor assemblies described herein may comprise or be in communication with a processor, such as a processor located in a remote or mobile computing device. The processor may be configured to store and/or analyze sensor data and/or calculate or determine any of the steps of the methods as described herein. The processor may be further configured to enter a control mode in response to receiving the sensor data and to generate a control signal. The processor may be further configured to monitor movement of the residual limb or muscle activity to generate a movement or alert threshold.

A person/one having ordinary skill in the art would understand that information and signals can be represented using any of a variety of different technologies and techniques. For example, data, instructions, commands, information, signals, bits, symbols, and chips that can be referenced throughout the above description can be represented by voltages, currents, electromagnetic waves, magnetic field strengths or particles, optical fields or particles, or any combination thereof. A person/one having ordinary skill in the art would further appreciate that any of the various illustrative logical blocks, modules, controllers, means, circuits, and algorithm steps or blocks described in connection with the aspects disclosed herein can be implemented as electronic hardware (e.g., a digital implementation, an analog implementation, or a combination of the two, which can be designed using source coding or some other technique), various forms of program or design code incorporating instructions (which can be referred to herein, for convenience, as “software” or a “software module”), or combinations of both. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps or blocks have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the overall system. Skilled artisans can implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present disclosure.

The aspects disclosed herein and in connection with the figures can be implemented within or performed by an integrated circuit (IC), an access terminal, or an access point. The IC can include a general purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, electrical components, optical components, mechanical components, or any combination thereof designed to perform the functions described herein, and can execute codes or instructions that reside within the IC, outside of the IC, or both. The logical blocks, modules, and circuits can include antennas and/or transceivers to communicate with various components within the network or within the device. A general purpose processor can be a microprocessor, but in the alternative, the processor can be any conventional processor, controller, microcontroller, or state machine. A processor can also be implemented as a combination of computing devices, e.g., a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration. The functionality of the modules can be implemented in some other manner as taught herein. The functionality described herein (e.g., with regard to one or more of the accompanying figures) can correspond in some aspects to similarly designated “means for” functionality in the appended claims.

If implemented in software, the functions can be stored on or transmitted over as one or more instructions or code on a computer-readable medium. The steps or blocks of a method or algorithm disclosed herein can be implemented in a processor-executable software module which can reside on a computer-readable medium. Computer-readable media includes both computer storage media and communication media including any medium that can be enabled to transfer a computer program from one place to another. A storage media can be any available media that can be accessed by a computer. By way of example, and not limitation, such computer-readable media can include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store desired program code in the form of instructions or data structures and that can be accessed by a computer. Also, any connection can be properly termed a computer-readable medium. Disk and disc, as used herein, includes compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk, and Blu-ray disc where disks usually reproduce data magnetically, while discs reproduce data optically with lasers. Combinations of the above should also be included within the scope of computer-readable media. Additionally, the operations of a method or algorithm can reside as one or any combination or set of codes and instructions on a machine readable medium and computer-readable medium, which can be incorporated into a computer program product.

Various modifications to the implementations described in this disclosure may be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other implementations without departing from the spirit or scope of this disclosure. Thus, the disclosure is not intended to be limited to the implementations shown herein, but is to be accorded the widest scope consistent with the claims, the principles and the novel features disclosed herein. The word “example” is used exclusively herein to mean “serving as an example, instance, or illustration.” Any implementation described herein as “example” is not necessarily to be construed as preferred or advantageous over other implementations.

Any specific order or hierarchy of steps or blocks in any disclosed process is an example of a sample approach. Based upon design preferences, it is understood that the specific order or hierarchy of steps or blocks in the processes may be rearranged while remaining within the scope of the present disclosure. Any accompanying that claims present elements of the various steps or blocks in a sample order are not meant to be limited to the specific order or hierarchy presented.

A person/one having ordinary skill in the art would appreciate that any of the various illustrative logical blocks, modules, controllers, means, circuits, and algorithm steps or blocks described in connection with the aspects disclosed herein may be implemented as electronic hardware (e.g., a digital implementation, an analog implementation, or a combination of the two, which may be designed using source coding or some other technique), various forms of program or design code incorporating instructions (which may be referred to herein, for convenience, as “software” or a “software module”), or combinations of both.

Certain features that are described in this specification in the context of separate implementations also may be implemented in combination in a single implementation. Conversely, various features that are described in the context of a single implementation also may be implemented in multiple implementations separately or in any suitable sub-combination. Moreover, although features may be described above as acting in certain combinations and even initially claimed as such, one or more features from a claimed combination may in some cases be excised from the combination, and the claimed combination may be directed to a sub-combination or variation of a sub-combination.

Similarly, while operations are depicted in the drawings in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed, to achieve desirable results. In certain circumstances, multitasking and parallel processing may be advantageous. In some cases, the actions recited in the claims may be performed in a different order and still achieve desirable results.

In general, terms used herein are generally intended as “open” terms (e.g., the term “including” should be interpreted as “including but not limited to,” the term “having” should be interpreted as “having at least,” the term “includes” should be interpreted as “includes but is not limited to,” etc.). If a specific number of an introduced claim recitation is intended, such an intent will be explicitly recited in the claim, and in the absence of such recitation no such intent is present.

For example, as an aid to understanding, the following appended claims may contain usage of the introductory phrases “at least one” and “one or more” to introduce claim recitations. However, the use of such phrases should not be construed to imply that the introduction of a claim recitation by the indefinite articles “a” or “an” limits any particular claim containing such introduced claim recitation to embodiments containing only one such recitation, even when the same claim includes the introductory phrases “one or more” or “at least one” and indefinite articles such as “a” or “an” (e.g., “a” and/or “an” should typically be interpreted to mean “at least one” or “one or more”); the same holds true for the use of definite articles used to introduce claim recitations.

In addition, even if a specific number of an introduced claim recitation is explicitly recited, those skilled in the art will recognize that such recitation should typically be interpreted to mean at least the recited number (e.g., the bare recitation of “two recitations,” without other modifiers, typically means at least two recitations, or two or more recitations). Furthermore, in those instances where a convention analogous to “at least one of A, B, and C, etc.” is used, in general such a construction is intended in the sense one having skill in the art would understand the convention (e.g., “a system having at least one of A, B, and C” would include but not be limited to systems that have A alone, B alone, C alone, A and B together, A and C together, B and C together, and/or A, B, and C together, etc.). In those instances where a convention analogous to “at least one of A, B, or C, etc.” is used, in general such a construction is intended in the sense one having skill in the art would understand the convention (e.g., “a system having at least one of A, B, or C” would include but not be limited to systems that have A alone, B alone, C alone, A and B together, A and C together, B and C together, and/or A, B, and C together, etc.). It will be further understood by those within the art that virtually any disjunctive word and/or phrase presenting two or more alternative terms, whether in the description, claims, or drawings, should be understood to contemplate the possibilities of including one of the terms, either of the terms, or both terms. For example, the phrase “A or B” will be understood to include the possibilities of “A” or “B” or “A and B.”

	Number	Date	Country
	63363563	Apr 2022	US
	63363565	Apr 2022	US

SENSOR SYSTEM AND METHOD FOR CONTROL OF PROSTHETIC DEVICES

Information

Publication Number

Date Filed

Date Published

Inventors

Original Assignees

CPC

International Classifications

Abstract

Description

Claims

INCORPORATION BY REFERENCE TO ANY PRIORITY APPLICATIONS

Provisional Applications (2)