The present invention relates to a system, method and apparatus for control of a device and more particularly, to a system, method and apparatus for orientation control of the device.
Many remote controls have been designed to manipulate robotic devices, mechanical devices, and virtual devices. There is a desire for a control system that may process user signals quickly and accurately while providing smooth directional and proportional control of associated objects.
In accordance with one aspect of the present invention, a control apparatus for a device is disclosed. The control apparatus includes at least one sensor module having an inertial measurement unit and at least one device module in communication with the at least one sensor module. The at least one sensor module is adapted to be worn by a user on at least one body part with the inertial measurement unit detecting body input and transmitting the body input to the device module. The device module translates the body input into an orientation of the at least one body part and commands the device based on the orientation.
In accordance with another aspect of the invention, the at least one sensor module is disposed in footwear.
In accordance with another aspect of the invention, the at least one sensor module is in wireless communication with the device module. In accordance with a further aspect of the invention, the device module is in wireless communication with the device.
In accordance with another aspect of the present invention, a method for controlling the device is disclosed. The method comprises sensing body input and communicating the sensed body input to a device module. The method also includes translating the body input into a body orientation and controlling the device based on the orientation.
These aspects of the invention are not meant to be exclusive and other features, aspects, and advantages of the present invention will be readily apparent to those of ordinary skill in the art when read in conjunction with the appended claims and accompanying drawings.
These and other features and advantages of the present invention will be better understood by reading the following detailed description, taken together with the drawings wherein:
Referring to
The sensor module 7015 includes one or more Inertial Measurement Units (IMUs) 7096 connected to a sensor central processing unit (sensor CPU) 7019 that is connected to a sensor module communicator 7020. The one or more IMUs 7096 detect orientation, as will be discussed in greater detail below, which may be transmitted to device module 7017 for controlling the associated device 7012. The sensor CPU 7019 inputs data from the one or more IMUs 7096 and filters and/or converts the data to generate orientation signals. The orientation signals are then sent to the device module 7017 by the sensor module communicator 7020. The sensor module communicator 7020 may be hard wired to the device module 7017 or may transmit the user input signals wirelessly, for example, but not limited to, through a radio transmitter, Bluetooth® or the like. Thus, by altering the orientation of the one or more IMUs 7096, the user may control the device 7012 in a desired manner. The sensor module may also include one or more sensors 7018 adapted to be disposed at various locations on the user to sense additional body input 7016 from the user.
In some embodiments, the device module 7017 includes a device CPU 7026 connected to a device controller 7027. The device CPU 7026 receives the orientation signals from the sensor module communicator 7020. Based on the orientation signals from the sensor module communicator 7020, the device CPU 7026 calculates device commands that are sent to the associated device 7012 by the device controller 7027 to control the device.
Referring to
Referring to
Referring to
In the embodiment of the IMU(s) 8096 shown in
The gyroscopes 8116 may provide many benefits over the use of accelerometers 8114. These include but are not limited to some of the following. A single algorithm may be used to estimate the Euler angles for all three axes, X, Y and Z. The gyroscopes 8116 are less sensitive to use in a non-inertial frame (e.g., car, boat, etc.) compared with accelerometers 8114. Additionally, there are no dynamic range/resolution issues due to initial inclination when control angle is re-zeroed, which may be present with accelerometers 8114. Additionally, in embodiments using three (3) accelerometers 8114 and/or gyroscopes 8116, user walking may be detected using a threshold rate with the assumption that the user moves their foot faster when walking than when controlling the associated device 7012, shown in
However, in some embodiments, the IMU 8096 having two accelerometers 8114 (or a two-axis accelerometer) and one gyroscope 8116 may be preferable over embodiments having three gyroscopes 8116 for reasons that may include, but are not limited to, any one or more of the following. The orientation signal provided by gyroscopes 8116 may drift over time, while there is no need to de-drift accelerometer axes. It may be simpler for the sensor CPU 8019 to estimate Euler angles using accelerometers 8114 than it is using gyroscopes 8116. In particular, the algorithm used by the sensor CPU 8019 for processing signals from the accelerometers 8114 requires less processing power than gyroscopes 8116. This may be particularly advantageous in many situations including with respect to use of the IMU(s) 8096 to control a prosthetic arm where the sensor module(s) 8019 are located on the user's ankle(s), where it may be advantageous and desirable to employ a wireless signal transfer between the sensor module 8019 and the device module 7017, shown in
Additionally, the accelerometers 8114 may themselves draw less power and be smaller in size than gyroscopes 8116. Also, the accelerometers 8114 may not require a specific DC range for power, which may allow for use of a non-changing and smaller range.
As discussed above, in some embodiments, the sensor CPU 8019 may filter the signals collected by the IMU 8096 to remove sensor noise and to provide a more clean signal. However, providing this functionality may result in a sensor module 8015 that is large and/or heavy and/or has large power requirements. Thus, it may be desirable, in some embodiments, to use a sensor module 8015 with less functionality that includes the capability of collecting “raw” data that may be used to determine pitch, roll and yaw. For instance, in some embodiments, the sensor module 7015, shown in
In some embodiments, the power supply 8107, shown in
Referring to
Referring back to
The accelerometers 8114, shown in about the X axis relative to the gravitational direction G and roll
about the Y axis relative to the gravitational direction G. The gyroscope 8116, shown in
about the Z axis. Thus, by using two IMUs 7096, one IMU 7096 on each foot 7021, the user is able to control the associated device 7012, shown in
Each IMU 7096 is arranged with one accelerometer 8114, shown in and/or roll
.
Although shown in , and a roll offset,
, when initialized or reinitialized during use.
Referring to detected by the IMU 7096 may be configured to command the associated device 7012, shown in
where,
is the pitch detected by the IMU 7096 relative to the gravitational direction G;
is the preset value calibrating the IMU 7096 discussed above; and
is a present minimum pitch angle that must be exceeded to ensure that the detected pitch
is a desired command and not due to unintentional movement of the user's foot 7021, shown in
In one embodiment, the command generated by the device module 7017, shown in may be a switch that alternates between an “on state” and an “off state” each time
. In another embodiment, pitch
may command the device module 7017 to toggle through a list. For example, each instance that
is exceeded, the device module 7017 may toggle forward through the list if
is a positive value and may toggle backward, i.e. in reverse, through the list if
is a negative value.
In one embodiment, the command generated by the device module 7017, shown in , of the associated device 7012, shown in
. For example, when
the device module 7017, shown in
is a positive value and may command downward movement if
is a negative value. In another embodiment, the movement may be commanded using the equation:
where,
and
are gains that may be preset based on the type of movement desired. The movement
may be set to correspond to a variety of possible movements of the associated device 7012, shown in
may be a distance of deflection in a direct direction or a speed of travel in a direction.
Referring to detected by the IMU 7096 may also be configured to command the associated device 7012, shown in
. For example, the device module 7017, shown in
where,
is the roll detected by the IMU 7096 relative to the gravitational direction G;
is the preset value calibrating the IMU 7096 discussed above; and
is a present minimum roll angle that must be exceeded to ensure that the detected roll
is a desired command and not due to unintentional movement of the user's foot 7021, shown in
In one embodiment, the command generated by the device module 7017, shown in may be a switch that alternates between an “on state” and an “off state” each time
. In another embodiment, roll
may command the device module 7017, shown in
is exceeded, the device module 7017, shown in
is a positive value and may toggle backward, i.e. in reverse, through the list if
is a negative value.
In one embodiment, the command generated by the device module 7017, shown in , of the associated device 7012, shown in
.
For example, when the device module 7017, shown in
is a positive value and may command movement to the left if
is a negative value. In another embodiment, the movement may be commanded using the equation:
where,
and
are gains that may be preset based on the type of movement desired. The movement
may be set to correspond to a variety of possible movements of the associated device 7012, shown in
may be a distance of deflection in a direct direction or a speed of travel in a direction.
Referring to as the rate of angular rotation relative to the Z axis. Thus, yaw
about the Z axis is detectable by the IMU 7096 when the user's foot 7021 moves about the Z axis. Unlike the pitch
and roll
, which are each detected relative to a fixed reference, i.e. the gravitational direction G, the yaw
is detected by the gyroscope 7116 with respect to the reference frame of the gyroscope 7116. Thus, the gyroscope 7116 effectively resets its frame of reference after each angular deflection of the IMU 7096. For example, if after moving from the first position P1 to the second position P2, the user then moves the IMU 7096 to a third position P3, the yaw
detected by the IMU 7096 as the IMU 7096 moves from the second position P2 to the third position P3 would be relative to the second position P2. This yaw
detected by the IMU 7096 may be configured to command the associated device 7012, shown in
For example, the device module 7017, shown in
where,
is the yaw detected by the IMU 7096; and
is a present minimum yaw angular rotation that must be exceeded to ensure that the detected yaw
is a desired command and not due to unintentional movement of the user's foot 7021, shown in
Advantageously, since the yaw detected by the gyroscope 8116, shown in
In one embodiment, the command generated by the device module 7017, shown in may be a switch that alternates between an “on state” and an “off state” each time
. In another embodiment, yaw
may command the device module 7017, shown in
is exceeded, the device module 7017, shown in
is a positive value and may toggle backward, i.e. in reverse, through the list if
is a negative value.
In one embodiment, the command generated by the device module 7017, shown in , of the associated device 7012, shown in
. For example, when
the device module 7017, shown in
at a preset velocity in a preset direction, e.g. the device module 7017 may command movement to the right at the preset velocity if
is a positive value and may command movement to the left if
is a negative value. In this exemplary embodiment for commanding right and left movement, it may also be desirable to halt right and left movement using the detected yaw
. For example, if the device module 7017 has commanded movement
to the right, based on a positive
, a subsequently detected negative
that satisfies the relationship
may generate a command to stop moving to the right, rather than a command to move to the left. From the stopped position, another negative
that satisfies the relationship
would then command leftward movement or, alternatively, a positive
that satisfies the relationship
would then command rightward movement. Similarly, if the device module 7017 has commanded movement
to the left, based on a negative
, a subsequently detected positive
that satisfies the relationship
may generate a command to stop moving to the left, rather than a command to move to the right. From the stopped position, a negative
that satisfies the relationship
would then command leftward movement or, alternatively, a positive
that satisfies the relationship
would then command rightward movement.
For exemplary purposes, the pitch , roll
and yaw
have been described as commanding specific movements in connection with
However, it should be understood that the pitch , roll
and yaw
may be programmed within the device module 7017, shown in
It should be understood that although the use of at least one IMU 7096 for control of the associated device 7012, shown in
In some embodiments, as discussed in connection with sensor module 9015 of , roll
and yaw
in substantially the same manner discussed above in connection with the IMU 8096, shown in
Referring back to and/or roll
, the vector sum of the accelerations Ax, Ay and Az in the X, Y and Z directions, respectively, will still equal 1G.
If the user begins to walk, the vector sum of the accelerations Ax Ay and Az detected by each of the three accelerometers 9114 will be substantially greater than 1G, since the act of walking will cause additional acceleration, other than gravity, to be detected by the IMU 7096. Thus, once the IMU 7096 detects the accelerations Ax, Ay and Az, the vector sum of the accelerations may be compared to a walk detect limit. In some embodiments, the walk detect limit may be set at approximately 1.2G. If the vector sum of the accelerations is lower than the walk detect limit, the device module 7017, shown in , roll
and/or yaw
detected by the IMU 7096. However, if the walk detect limit is exceeded by the vector sum of the accelerations, the device module 7017, shown in
Referring back to
In some embodiments of the present invention, the device module 7017 will send a zero command to the sensor module 7015, which the sensor module 7015 may use to redefine its zero position or orientation to be the current position or orientation. For example, the device module 7017, shown in , and a roll offset,
. This zero command may be used to compensate for orientation changes of the foot/feet 7021, shown in
The sensor module of the present invention is advantageous because it is able to detect the orientation of a user's body part, for commanding an associated device, using an IMU having fewer orientation sensors, i.e. accelerometers and/or gyroscopes, than conventional IMUs. This is beneficial because it reduces the overall size of the IMU, which, in turn, reduces the size and weight of the sensor module. Additionally, by reducing the number of orientation sensors as compared to conventional IMUs, the present invention also advantageously reduces both the power required for the IMU and the processor size required to process the signals from the IMU. This also advantageously reduces the size of the sensor CPU and the sensor module battery, thereby further reducing the size of the sensor module. This size reduction beneficially provides for a single-component compact sensor module adapted to be attached on essentially and user body part with minimal discomfort.
While the principles of the invention have been described herein, it is to be understood by those skilled in the art that this description is made only by way of example and not as a limitation as to the scope of the invention. Other embodiments are contemplated within the scope of the present invention in addition to the exemplary embodiments shown and described herein. Modifications and substitutions by one of ordinary skill in the art are considered to be within the scope of the present invention.
This application is a continuation of U.S. patent application Ser. No. 13/907,030, filed May 31, 2013, which is a continuation of U.S. patent application Ser. No. 12/706,471, filed Feb. 16, 2010, which is a continuation-in-part of U.S. patent application Ser. No. 12/027,116, filed Feb. 6, 2008, which claims priority from U.S. Provisional Patent Application Ser. No. 60/899,834, filed Feb. 6, 2007, and U.S. Provisional Patent Application Ser. No. 60/963,638, filed Aug. 6, 2007, each of which is hereby incorporated by reference in its entirety. This application also claims priority to U.S. Provisional Patent Application Ser. No. 61/168,832, filed Apr. 13, 2009, and U.S. Provisional Patent Application Ser. No. 61/221,858, filed Jun. 30, 2009, each of which is also hereby incorporated by reference in its entirety.
| Number | Date | Country | |
|---|---|---|---|
| 60899834 | Feb 2007 | US | |
| 60963638 | Aug 2007 | US | |
| 61168832 | Apr 2009 | US | |
| 61221858 | Jun 2009 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | 13907030 | May 2013 | US |
| Child | 14840559 | US | |
| Parent | 12706471 | Feb 2010 | US |
| Child | 13907030 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | 12027116 | Feb 2008 | US |
| Child | 12706471 | US |