Various embodiment of the invention relate generally to motion sensor data and particularly to integration of gyroscope data to generate motion data.
In applications using gyroscopes, gyroscope data from a gyroscope is collected along with data from a compass and/or an accelerometer and other sensors. The collected data is then fused together to determine the movement of the device. The collected data is also integrated over time and at a high rate to reduce errors. Such high rate data is typically sent from a semiconductor device (or integrated circuit) to a processor. Due to the high rate associated with the integrated data, transmitting high rate data to the processor causes increased power. There is therefore a need for efficient transmission of gyroscope data with a decrease in system power.
Briefly, a method of the invention includes generating motion data by receiving a gyroscope data from a gyroscope, performing integration using the gyroscope data and generating an integrated gyroscope data using a first processor. The method further includes receiving a data from one or more sensors, other than the gyroscope, and performing sensor fusion using the integrated gyroscope data and the data to generate motion data using a second processor.
A further understanding of the nature and the advantages of particular embodiments disclosed herein may be realized by reference of the remaining portions of the specification and the attached drawings.
In the following description of the embodiments, reference is made to the accompanying drawings that form a part hereof, and in which is shown by way of illustration of the specific embodiments in which the invention may be practiced. It is to be understood that other embodiments may be utilized because structural changes may be made without departing from the scope of the present invention. It should be noted that the figures discussed herein are not drawn to scale and thicknesses of lines are not indicative of actual sizes.
In accordance with an embodiment and method of the invention, integration of gyroscope data is performed in a single device, such as a single semiconductor device. Gyroscope integration is performed at a rate that is higher than the integrated output data rate to reduce the in-device processing requirements. In an embodiment of the invention, integration of the gyroscope data is performed using a curve-fit to reduce the in-device processing requirements. Integration over time, at a higher rate, causes fewer errors. Yet, the integrated data is sent at a lower data rate thereby reducing power consumption. The integrated gyroscope data can also be passed at the same rate as the gyroscope data to reduce computation on the second processor and maintain a consistency for the type of data being sent. In the described embodiments, the integrated gyroscope data is also referred to as a low power quaternion (LPQ).
In other embodiments and methods of the invention, sensor fusion is performed at a lower rate than the rate of the gyroscope data integration. The output of a gyroscope-only quaternion data is saved in a register. In another embodiment, the quaternion data is stored in a first-in-first-out (FIFO) that resides in the same device that includes the gyro. In the case of a semiconductor, the “same device” as used herein refers to the same chip. In some embodiments, the integration of a three-axis gyroscope in the same package as the gyroscope sensor.
In yet another embodiment and method of the invention, removal of a known gyroscope bias is performed after integration. In yet other embodiments the gyroscope bias is removed before integration.
In yet other embodiments, a 9-axis (gyroscope, accelerometer, and compass) motion tracking is employed in a variety of sensor platform architectures, some of which are disclosed herein, and the remainder of which are too numerous to list but that are contemplated.
In the described embodiments, a motion tracking device also referred to as Motion Processing Unit (MPU) includes at least one sensor in addition to electronic circuits. The sensors, such as the gyroscope, the compass, the accelerometer, microphone, pressure sensors, proximity, ambient light sensor, among others known in the art, are contemplated. Some embodiments include accelerometer, gyroscope, and magnetometer, which each provide a measurement along three axis that are orthogonal relative to each other referred to as a 9-axis device. Other embodiments may not include all the sensors or may provide measurements along one or more axis. The sensors are formed on a first substrate. Other embodiments may include solid-state sensors or any other type of sensors. The electronic circuit in the motion tracking device receive measurement outputs from the one or more sensors. In some embodiments, the electronic circuit processes the sensor data. The electronic circuit is implemented on a second silicon substrate. The first substrate is vertically stacked and attached to the second substrate in a single device.
In an embodiment of the invention, the first substrate is attached to the second substrate through wafer bonding, as described in commonly owned U.S. Pat. No. 7,104,129 (incorporated herein by reference) that simultaneously provides electrical connections and hermetically seals the MEMS devices. This fabrication technique advantageously enables technology that allows for the design and manufacture of high performance, multi-axis, inertial sensors in a very small and economical package. Integration at the wafer-level minimizes parasitic capacitances, allowing for improved signal-to-noise relative to a discrete solution. Such integration at the wafer-level also enables the incorporation of a rich feature set which minimizes the need for external amplification.
In the described embodiments, raw data refers to measurement outputs from the sensors which are not yet processed. Motion data refers to processed raw data. Processing may include calibration to remove error or applying a sensor fusion algorithm or applying any other algorithm. In the case of the sensor fusion algorithm, data from one or more sensors are combined to provide an orientation of the device. In the described embodiments, a motion processing unit (MPU) may include processors, memory, control logic and sensors among structures.
Referring now to
In operation, MPU 14 generally processes gyroscope data 9 (“gyroscope measurement output” or “gyroscope data”) transmitted by gyroscope 18 in DMP 3 and outputs the orientation of the gyroscope in the form of the integration gyroscope data 9a. MPU 14 may transmit data 7 from other sensors, such as but not limited to an accelerometer and/or a compass. MPU 14 generates a gyroscope data output 5 to processor 12. Data 5 has a data rate that is lower than the rate of the gyroscope data output 9. A part of the processing performed by the MPU 14 is quaternion and/or integration in DMP 3.
Similarly, MPU 14 receives raw motion data 11 from accelerometer 20 and the compass 22 and stores raw motion data 11 in FIFO 24 before transmitting to the processor 12.
In
In some embodiments, processor 12 and MPU 14 are formed on different substrates and in other embodiments they reside on the same substrate. In yet other embodiments, the sensor fusion is performed externally to the processor 12 and MPU 14. In still other embodiments, the sensor fusion is performed by MPU 14.
Referring still to
Register 26 and FIFO 24 each provide a low rate output to processor 12. Processor 12 further performs processing by performing a 6 or 9-axis sensor fusion calculation based upon the integrated 3-axis gyroscope data 5 and accelerometer and compass data 7. Data integration at high rate at the block 16 advantageously increases data reliability whereas the low rate between MPU 14 and processor 12 advantageously reduces power consumption. It is noteworthy that block 16, gyroscope 18, compass 22 and accelerometer 20 reside on the same package, in an embodiment of the invention and reside on the same substrate in another embodiment of the invention.
Separation of the MPU 14 and the processor 12 allows users of the MPU 14 to flexibly use the output of the MPU 14. For example, different users of the device 10 may choose the flexibility to use various sensor fusion algorithms while still lowering the power from a reduced transmission rate.
Integration may be performed using sampling that starts from the gyroscope measurement output. In other embodiments of the invention, the sampling starts after a predetermined number of samples and continues over a predetermined time period. In yet other embodiments of the invention, during a dead zone where the input of the integrator or Quaternion (gyroscope data) is close to zero, the output is made to remain the same.
FIFO/Register 24 is also provided with a requested data rate at 210 and the requested data rate at 210 uses one of the outputs of the sensor fusion 208 to determine the data rate being requested before passing along this information to FIFO/Register 24. Another output of sensor fusion 208 is shown to be the motion data at 216.
Sensor fusion 208 is shown to further receive measurement outputs from accelerometer data 202 and compass data 204. If all three types of data, i.e. gyroscope data, accelerometer data, and compass data, are employed by the sensor fusion 208, the output 216 is a 9-axis Quaternion. In another embodiment, sensor fusion 208 may receive measurement outputs from other sensors such as pressure sensor or microphone. In some embodiments, sensor fusion block 208 may be part of MPU 14 or part of AP 12.
In some embodiments, a bias removal block 214 for removing an inherent bias in the gyroscope. For example, after the gyroscope data at 200 is provided, bias removal 214 can use used to remove the gyroscope's inherent bias. Alternately, bias removal 214 can be applied after integrating gyroscope data in the sensor fusion 208. As will be evident shortly, bias removal may be employed at stages other than at the output at 200.
Referring now to
In
In
In
The sensor fusion of the various embodiments of the invention calculates the quaternion in accordance with the algorithm and relationships below.
A unit quaternion, also referred to as “quaternion”, is a 4-element vector that describes how to go from one orientation to another orientation. Other equivalent ways to describe going from one orientation to another orientation include a rotation matrix and Euler angles. A unit quaternion has a scalar term and 3 imaginary terms and a magnitude of one for the four element vector. For the purpose of discussion herein, the scalar term is placed first followed by the imaginary term. One skilled in the art would be able to compute slightly different formulas using a different definition of a quaternion. Euler angles describe how to go from one orientation to another orientation by rotating 3 different angles about 3 different axes of rotation. Instead of moving 3 angles about 3 different axes of rotation, the same rotational motion can be described by rotating one angle around a single vector. In Equation 1 below, for a quaternion, the angle θ is the amount rotated about the unit vector, [ux, uy, uz].
A quaternion multiplication is defined in Equation 2 as found in literature. The notation “⊗”, as used herein, represents quaternion multiplication in this document.
q1w·q2x+q1x·q2w+q1y·q2z−q1z·q2y,
q1w·q2y−q1x·q2z+q1y·q2w+q1z·q2x,
q1w·q2z+q1x·q2y−q1y·q2x+q1z·q2w] Equation 2
A quaternion inverse is defined as:
Sensor fusion will take measurements from various sensors, such as a gyroscope, an accelerometer and a compass then combine the measurements to determine the orientation of a device referenced to a fixed frame. Each sensor has good points and bad points when combining the data. A rate gyroscope (gyroscope hereafter) is good indicating how the orientation changes overtime without being affected by magnetic disturbance or acceleration. However, a gyroscope often has an offset which leads to angular drift over time. There are many methods of implementing the sensor fusion, but one common aspect of many of those is that the gyroscope data must be integrated. If a gyroscope sensor was perfect and integration was perfect, there would be no need for sensor fusion. However, sensors are not perfect and integration is not perfect, so other sensors are used to correct the orientation. One of the errors incurred, is integration of the gyroscope data. To lower the integration error, the gyroscope data is often integrated at a high rate. Typically a sensor fusion algorithm will take raw sensor data and compute an orientation. The method presented here is to integrate the gyroscope data on one processor then the integrated data is sent to a different processor to do the sensor fusion. When the integrated gyroscope data is sent to the second processor, the integrated gyro data rate can be less than the original gyroscope data rate. The processor doing the gyroscope data integration can be designed to take less power than the processor doing the sensor fusion. Sending the integrated gyroscope data at a lower data rate than the data rate of the gyroscope data has similar performance to sending the gyroscope data out at the original gyroscope data rate. Running sensor fusion at the integrated gyroscope data rate has similar performance to running sensor fusion at the gyroscope data rate. Lowering the sample rate reduces data traffic. Lowering the sensor fusion rate reduces computation. The gyroscope data is not required to be sent out for the sensor fusion, which leads to further data traffic reductions. Approximate gyroscope data can be computed from the integrated gyroscope data being sent out, by computing the derivate of the integrated gyroscope data. The estimate gyroscope data derived from taking the derivative of the integrated gyroscope data (LPQ) can be used for other algorithms such as determining the gyroscope data bias. Sending out integrated gyroscope data also allows a sensor with a gyroscope to output this integrated gyroscope data as an output, and have others write a sensor fusion algorithm using a reduced data rate. In one embodiment, the gyroscope integration will be done into a quaternion and can be done in software, firmware, or hardware on a separate device from the sensor fusion. In an embodiment of the invention, the gyroscope and the processor, performing the integration are in separate packages. In yet another embodiment of the invention, the gyroscope and the processor, performing the gyroscope integration, are in the same package or substrate.
There are several methods of integrating gyroscope data. Given angular velocity in radians/second in Equation 4 with magnitude ωm shown in Equation 5.
ω=[ωx,ωy,ωz] Equation 4
ωm=√{square root over (ωx·ωx+ωy·ωy+ωz·ωz)} Equation 5
a quaternion can be defined as shown in Equation 6.
Where Δt is time elapsed between time step M−1 and time step M. To integrate a quaternion using constant angular velocity from time step M−1 to time step M, the formula in Equation 7 is used.
There are other approximations that can be used to integrate quaternion data such as:
In Equation 8, the previous quaternion is multiplied by the rotation rate from the gyroscope (in radians) using a quaternion multiply. This is scaled by the time between samples over 2 and added to the previous quaternion. Next, the quaternion is divided by the magnitude to maintain a magnitude of 1. Note, optimizations that may be used would be to use raw gyroscope units and to scale by a constant that encompasses both the conversion to radians and the sample rate over 2. Other optimizations for the normalization to 1, would be to use a Taylor series to avoid the inverse square root. Furthermore, this could be converted to fixed point or integer math by one skilled in the art.
The gyroscope data typically has noise on it. Integrating noise on the gyroscope data will cause the quaternion to drift. To overcome this drift in a stationary position, the gyroscope data could be set to zero if the absolute value of the gyroscope data is below a threshold.
If the gyroscope bias is known, the gyroscope bias could be subtracted out of the gyroscope data before integrating the gyroscope data. This can be done in software, firmware or hardware.
With each sample of quaternion data that is received, independent of the rate at which the angular velocity was integrated at, an angular velocity bias can be removed even after the bias was integrated into the solution. A quaternion going from one time step to another can be modeled as Equation 9 where
Equation 7 and
Equation 8 going from
The next step is to maintain a unit quaternion variable with the gyroscope bias removed after the gyroscope bias has been integrated into the quaternion. After combining these ideas, Equation 10 shows how to remove a constant bias.
Where
After doing sensor fusion and updating
If the bias correction is done before integrated the gyroscope data into a quaternion,
The basic idea presented so far has the quaternion integration starting from zero and sending out the quaternion referenced from that point. Instead, the starting point could be reset at each time output step so only the change from one time step to the next is sent out. This will simplify some of the equations. This change can be represented as
Further optimization would allow the quaternion integration change with bias correction to be applied directly to the sensor fusion quaternion.
Similarly, the
Noted, that while outputting the integration over one time step yields simpler equations, it is less immune to dropping samples. Often, if a system gets overloaded, samples could get lost, if the integration is done cumulatively across many samples, losing a sample would causes less error.
To further reduce the data traffic, it can be noted, that a quaternion has magnitude one, doing so would allow 3 components to be transferred across a data line, and to have the fourth element reconstructed. When trying to reconstruct the fourth element, there can be some sign ambiguity. To overcome the sign ambiguity, it is possible to force the sign of the missing component to have a known value. This is because in quaternion space,
Other embodiments would include but not limited to other ways to pass information equivalent to quaternion data. As a quaternion represents an orientation in space, an orientation matrix or Euler angles could be substituted for the quaternion.
There are various ways to integrate a quaternion. Below is a description on quaternion integration along with new methods.
Zero'th Order Quaternion Integration:
Integrating a quaternion is equivalent to solving the corresponding first order ordinary differential equation:
Where
is a skew symmetric matrix. Assuming the angular velocity vector ω is constant between time t and t+Δt. The generalized solution to this differential equation is
In order to calculate the above solution, the matrix exponent of the skew-symmetric matrix Ω has to be calculated. Calculating the exponent of a matrix numerically is very expensive. In real-time implementations, a simplified approximation is used, with the procedure:
Δ
Where (1) is used with a Forward Euler approximation, and
In this approximation, the change in the quaternion is not computed in the rotation space, i.e. Equation 18 violates the rules of quaternion algebra. This causes small errors in the quaternion calculation, which can grow over time. In order to maintain an accurate quaternion with unit norm, the quaternion is re-normalized, as shown in the final step Equation 19.
An alternative method to compute Equation 16 more accurately is as follows. First, the matrix
is expanded in a Taylor Series as follows:
Using the properties of the skew-symmetric matrix Ω(ω) and the Euler formula, as discussed in the reference “Matrix Mathematics: Theory, Facts, and Formulas”, Princeton University Press, Second Edition, 2009, by Dennis S. Bernstein, p. 741, exponents of skew-symmetric matrices can be rewritten as:
Where ωm is the magnitude of the angular rate ω, as defined in ωm=√{square root over (ωx·ωx+ωy·ωy+ωz·ωz)} Equation 5. Then,
Close inspection of Eqn. Equation 22 reveals that this matrix-vector multiplication is equivalent to a quaternion multiplication:
Where
is the 4×1 quaternion that represents the quaternion rotation induced by the angular velocity co over time span Δt. Comparing this with Equation 18, it is noted that the formula in Equation 23 is exact, and it preserves the quaternion norm, i.e. no re-normalization of the quaternion is necessary.
Note that for very small angular velocity ω, the above expression can cause numerical instability. In this case, a first order approximation of Equation 21 can be used:
First Order Quaternion Integration:
The zero'th order quaternion integration assumes that the angular velocity ω is constant over the period Δt. A better and more accurate integration can be derived if the angular velocity is assumed to be evolving linearly over the time Δt. For this purpose, ω(tk) is introduced for the value of velocity ω at time tk, and ω(tk+1) for ω at time tk+1, with Δt=tk+1−tk.
The average angular velocity over the time Δt is:
Also, define the derivative of the angular velocity as {dot over (ω)}. The corresponding matrix Ω({dot over (ω)}), in the linear case, is constant:
And the higher order derivatives are zero due to the assumption of linearly changing angular velocity ω. In the following, Ω(ω) is represented as Ω, and Ω({dot over (ω)}) as {dot over (Ω)}, for conciseness. As described in the publication “Spacecraft Attitude Determination and Control”, edited by J. R. Wertz, D. Reidel Publishing Co., Dordrecht, The Netherlands, 1978, p. 565, Eqn. (17-21)] in order to derive the higher order quaternion derivative expression, the Taylor series expansion of the quaternion is done at time instant tk+1 in terms of the quaternion and its derivatives at time tk is as follows:
Using the fact that
Ω(
Ω(
Using Equation 29 the terms in Equation 27 can be rearranged as follows (Eqn. (17-23 in Wertz)):
The first of the two terms on the right-hand side of Equation 30 is similar to the Taylor Series expansion of Equation 16, and thus has the same form. Note that this differs from Equation 16 in using time-averaged rate information rather than instantaneous rates. The second term with the derivatives of the angular rate can be expanded using Equation 26. This gives:
In order to simplify the expression in Equation 30, the following is noted.
First, form the cross product of the angular velocities ω(tk) and ω(tk+1):
ωΔ=ω(tk)×ω(tk+1) Equation 32
For the second expression in Equation 31 the following holds:
Ω(ωΔ)=2Ω(ω(tk+1))Ω(ω(tk))−Ω(ω(tk))Ω(ω(tk+1)) Equation 33
i.e. the skew-symmetric form of ωΔ is the same as the right hand side of Equation 33. This can be verified by expanding the terms and comparing the elements. Introducing Equation 32 and Equation 33 into Equation 31, the following is derived:
It is noted that the second term of Equation 34, can be rewritten as:
Comparing the above with Equation 20-Equation 23, the following can be derived:
Where
is the 4×1 quaternion that represents the quaternion rotation induced by the average angular velocity
Note that the term (Δ
To summarize, the computational steps for the first order quaternion integration are as follows:
Calculate
Calculate quaternion
Calculate
Calculate quaternion Δ
Calculate
A computationally simpler version of the first order quaternion integration is as follows: Given angular rates ω(t−Δt), ω(t), at time instants t and t−Δt, and the current quaternion
Then, compute:
Δ
and, as in Equation 39 the quaternion is computed and normalized with:
Although the description has been described with respect to particular embodiments thereof, these particular embodiments are merely illustrative, and not restrictive.
As used in the description herein and throughout the claims that follow, “a”, “an”, and “the” includes plural references unless the context clearly dictates otherwise. Also, as used in the description herein and throughout the claims that follow, the meaning of “in” includes “in” and “on” unless the context clearly dictates otherwise.
Thus, while particular embodiments have been described herein, latitudes of modification, various changes, and substitutions are intended in the foregoing disclosures, and it will be appreciated that in some instances some features of particular embodiments will be employed without a corresponding use of other features without departing from the scope and spirit as set forth. Therefore, many modifications may be made to adapt a particular situation or material to the essential scope and spirit.
This application is a continuation application of U.S. application Ser. No. 16/372,238 filed on Apr. 1, 2019, titled “Method to Reduce Data Rates and Power Consumption Using Device Based Attitude Generation”, by Keal et al., which is a continuation of U.S. patent application Ser. No. 14/194,799, filed on Mar. 2, 2014, now U.S. Pat. No. 9,880,185, titled “Method to Reduce Data Rates and Power Consumption Using Device Based Attitude Generation”, by Keal et al., which is a continuation of U.S. patent application Ser. No. 13/965,131, filed on Aug. 12, 2013, now U.S. Pat. No. 10,247,748, titled “Method to Reduce Data Rates and Power Consumption Using Device Based Attitude Generation”, by Keal et al., which claims priority to U.S. Provisional Application No. 61/786,414 filed on Mar. 15, 2013, titled “Method to Reduce Data Rates and Power Consumption Using Device Based Axis Quaternion Generation”, by Keal et al., the disclosures of which are incorporated herein by reference as though set forth in full.
Number | Name | Date | Kind |
---|---|---|---|
4338818 | Hill | Jul 1982 | A |
5871517 | Abrams | Feb 1999 | A |
7104129 | Nasiri | Sep 2006 | B2 |
7216055 | Horton | May 2007 | B1 |
7932570 | Eskridge | May 2011 | B1 |
8362949 | Yang | Jan 2013 | B2 |
9880185 | Keal | Jan 2018 | B2 |
10247748 | Keal | Apr 2019 | B2 |
20050251328 | van der Merwe | Nov 2005 | A1 |
20070032951 | Tanenhaus | Feb 2007 | A1 |
20090184849 | Nasiri | Jul 2009 | A1 |
20100121227 | Stirling | May 2010 | A1 |
20110157318 | Nalibotski | Jun 2011 | A1 |
20110270554 | Hsieh | Nov 2011 | A1 |
20120053869 | Owen | Mar 2012 | A1 |
20120065883 | Williamson | Mar 2012 | A1 |
20120084050 | Sato | Apr 2012 | A1 |
20120148113 | Fraisse | Jun 2012 | A1 |
20120209555 | Tang | Aug 2012 | A1 |
20120221244 | Georgy | Aug 2012 | A1 |
20120326922 | Yang | Dec 2012 | A1 |
20130103344 | Bye | Apr 2013 | A1 |
20130158928 | Hodgal | Jun 2013 | A1 |
20130211711 | Kelly | Aug 2013 | A1 |
20140081592 | Bellusci | Mar 2014 | A1 |
20140278215 | Keal | Sep 2014 | A1 |
20190227093 | Keal | Jul 2019 | A1 |
Number | Date | Country |
---|---|---|
10-0643400 | Nov 2006 | KR |
Entry |
---|
Wertz and Members of the Technical Staff Attitude Systems Operation Computer, Sciences Corporation, “Spacecraft Attitude Determination and Control”, Kluwer Academic Publishers (1978). |
Bernstein, Dennis, “Matrix Mathematics Theory, Facts, and Formulas”, Princeton University Press, Princeton and Oxford (2009). |
Blyth, Russel. “http://www.mathmistakes.info/facts/CalculusFacts/learn/doi/doi.html”, Derivative of an Integral (Fundamental Theorem of Calculus), Copyright © 2006-2008. |
Number | Date | Country | |
---|---|---|---|
20210025914 A1 | Jan 2021 | US |
Number | Date | Country | |
---|---|---|---|
61786414 | Mar 2013 | US |
Number | Date | Country | |
---|---|---|---|
Parent | 16372238 | Apr 2019 | US |
Child | 17071782 | US | |
Parent | 14194799 | Mar 2014 | US |
Child | 16372238 | US | |
Parent | 13965131 | Aug 2013 | US |
Child | 14194799 | US |