This disclosure generally relates to mobile devices, and more particularly to improving the quality of accelerometer and/or other motion sensor generated user input to mobile devices while the user and device are both moving.
Playing video games on mobile devices while sitting in a moving vehicle (e.g., car, bus, train, airplane, boat, etc.) is an unpredictable experience because every acceleration that the vehicle makes imparts unintended acceleration input data to the application running on the mobile device. The mobile device cannot differentiate between intended acceleration (e.g., the player intended to tilt the phone to provide input to a software application) versus unintended acceleration (e.g., the moving vehicle turned or went over a bump in the road). These unintended accelerations due to operating in the context of a moving vehicle are “noise” that are transmitted to the mobile device via the user. Such noise may negatively impact the operation or accuracy of the software application (e.g., a game) running on the mobile device.
Advances in motion sensor technologies have provided for new forms of interaction between users and handheld computers or mobile devices (e.g., mobile phones, tablets, personal digital assistants, handheld video game consoles, wearable computing devices, virtual reality helmets, etc.). In particular, user input may be in the form of movement of the handheld computer. For example, a user may tilt, turn, rotate, or shake a mobile device to control actions or movement of a character in a video game, scroll through a document, open or close a software application, select a function of a software application, or provide other types of user input. As discussed above, when a user does not have control of the surrounding environment, such as when the user is a passenger in a moving vehicle, unintended accelerations on the user may cause unintended user input to the mobile device.
In certain embodiments disclosed herein, an accelerometer is mounted to a moving vehicle to provide (e.g., wirelessly or through a communication cable) a data stream that the mobile device can use to cancel out unwanted or unintended acceleration due to the moving vehicle. Because the moving vehicle and the mobile device undergo approximately the same accelerations in each axis, the mobile device is configured to exploit the differential nature of the signal to subtract the noise (the data from the remotely mounted accelerometer) from the mobile device's accelerometer signal (the data that the mobile device presents in its unfiltered data stream). The resulting signal has less noise than that of the uncompensated motion signal. Thus, the sensor driver on the mobile device presents software applications with a data stream that more accurately reflects what the user intended, e.g., as if the user and the device were not undergoing unintended acceleration. The user's experience (e.g., game play or other interaction with the mobile device) is improved due to the enhanced noise immunity derived from the use of the differential signal.
In certain embodiments, a peripheral device includes a system on chip (SoC), a communication interface, and a 3-axis accelerometer. The peripheral device, also referred to herein as a remote motion sensor device, may be either permanently or temporarily mounted to a fixed location on a moving vehicle. The peripheral device constantly communicates 3-axis accelerometer data to a driver or module residing on a mobile device connected through the communication interface. In one embodiment, the communication interface is a wireless communication interface, such as a Bluetooth or WiFi (under the Institute of Electrical and Electronics Engineers (IEEE) 802.11 standard) radio communications subsystem. Thus, the peripheral device may be paired with one or more handheld mobile devices that are granted access to the 3-axis accelerometer data. In addition, or in other embodiments, the peripheral device may be tethered to a mobile device via a cable plugged into an available auxiliary port on the mobile device. While a wireless link may provide a higher level of convenience, embodiments that include only the tethered option provide a lower cost and lower power alternative due to the removal of the radio communications subsystem.
In certain embodiments, a mobile device includes a driver or module for the remote motion sensor device that is configured to receive a data stream including 3-axis acceleration data from a remote sensor, and to provide the data stream to a replacement accelerometer driver. The replacement accelerometer driver is configured to subtract the remote sensor waveform data from the local sensor waveform data through, for example, the use of digital audio signal processing techniques. The waveform subtraction process yields motion compensated user input data to a software application running on the mobile device. In certain embodiments, the replacement accelerometer driver interfaces with the mobile device system using the same application programming interface definition as that of an original accelerometer driver on the mobile device, which enables seamless integration into the underlying system software of the mobile device. Thus, the new accelerometer driver can replace the old accelerometer driver without the need to modify or notify the software application running on the mobile device that receives the motion compensated user input data. Such embodiments provide a minimally invasive change to the software environment of the mobile device.
Example embodiments are described below with reference to the accompanying drawings. Many different forms and embodiments are possible without deviating from the spirit and teachings of the invention and so the disclosure should not be construed as limited to the example embodiments set forth herein. Rather, these example embodiments are provided so that this disclosure will be thorough and complete, and will convey the scope of the invention to those skilled in the art. In the drawings, the sizes and relative sizes of components may be exaggerated for clarity. The terminology used herein is for the purpose of describing particular example embodiments only and is not intended to be limiting. As used herein, the singular forms “a,” “an” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms “comprises” and/or “comprising,” when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof. Unless otherwise specified, a range of values, when recited, includes both the upper and lower limits of the range, as well as any sub-ranges therebetween.
The mobile device 110 comprises the accelerometer driver 114. Although not shown in
In the example of
The mobile device 110 may be any device configured to receive user generated motion input. For example,
The mobile device 110 shown is not limited to devices that receive input from a user's hands. For example,
The communication interface module 410 may include any wired and/or wireless communication device for providing the remote accelerometer sensor data 120 to the mobile device 110. The communication interface module 410 may include, for example, a universal serial bus (USB) driver or any other serial or parallel communication port. In addition, or in other embodiments, the communication interface module 410 may include a radio communication subsystem for wireless communications, such as a Bluetooth or WiFi radio.
The motion sensor module 414 includes one or more accelerometers. For example,
In certain embodiments, the motion sensor module 414 includes one or more sensors in addition to accelerometers. For example,
Returning to
The alignment module 610 is configured to align a coordinate system of the mobile device 110 and a coordinate system of the remote motion sensor device 112.
The alignment module 610 may perform an initial alignment process that includes, for example, requesting a user to move the mobile device 110 in a first direction in a horizontal plane (e.g., along an estimated x-axis direction of the remote motion sensor device 112), and then to move the mobile device 110 in a second direction in the horizontal plane (e.g., along an estimated y-axis direction of the remote motion sensor device 112 that is perpendicular to the x-axis). The alignment module 610 may then use its acceleration data generated during the movement in the first direction and the second direction to establish an initial orientation of the mobile device 110 with the coordinate system of the remote motion sensor device 112. In certain embodiments, local motion data received by the alignment module 610 includes data provided by a gyroscope device and/or a magnetometer device of the mobile device 110. In such embodiments, the alignment module 610 is configured to track the orientation of the mobile device 110 with respect to the initial orientation corresponding to the coordinate system of the remote motion sensor device 112. Thus, the user can shift positions within a seat, or even change seats or locations within a vehicle, without realigning the user's mobile device 110 with the remote motion sensor device 112.
The alignment module 610 provides aligned local acceleration data 614 and remote acceleration data 616 to the motion compensation module 612. As discussed above, when the user and mobile device 110 are located in a moving vehicle monitored by the remote motion sensor device 112, the local acceleration data 614 includes a superposition of intended user input accelerations and unintended accelerations to the moving vehicle. The unintended accelerations appear as noise corresponding to the remote acceleration data. To remove the noise, the motion compensation module 612 is configured to subtract the remote acceleration data 616 from the local acceleration data 614 through, for example, the use of digital audio signal noise cancelation techniques.
For example,
The compensated acceleration data signal 618 shown in
The following are examples of further embodiments. Examples may include subject matter such as a method, means for performing acts of the method, at least one machine-readable medium including instructions that, when performed by a machine cause the machine to perform acts of the method, or of an apparatus or system for improving input to a mobile device according to the embodiments and examples described herein.
Example 1 is a system to improve user input to a mobile device. The system includes a remote motion sensor device configured to generate remote acceleration data corresponding to movement of a vehicle. The system also includes an accelerometer module configured to receive the remote acceleration data generated by the remote motion sensor, and to receive local acceleration data corresponding to movement of the mobile device. The accelerometer module includes a motion compensation module configured to subtract the remote acceleration data from the local acceleration data to generate compensated acceleration data. The accelerometer module is configured to provide the compensated acceleration data as user input to a software application on the mobile device.
Example 2 includes the subject matter of Example 1, wherein the remote motion sensor device includes a motion sensor module comprising at least one remote accelerometer to generate the remote acceleration data. The remote motion sensor may also include a communication interface module configured to transmit the remote acceleration data to a mobile device, and a processor to control at least one of the motion sensor module and the communication interface module.
Example 3 includes the subject matter of Example 2, wherein the at least one remote accelerometer includes a 3-axis accelerometer, and wherein the remote acceleration data indicates movement of the vehicle in an x-axis direction, a y-axis direction, and a z-axis direction of the 3-axis accelerometer.
Example 4 includes the subject matter of any of Examples 1-3, wherein the motion sensor module further comprises at least one of a gyroscope device and a magnetometer device.
Example 5 includes the subject matter of any of Examples 1-4, wherein the communication interface module is configured for serial communications with the mobile device over a communication wire.
Example 6 includes the subject matter of any of Examples 1-5, wherein the communication module comprises a wireless communication subsystem.
Example 7 includes the subject matter of any of Examples 1-6, wherein the accelerometer module further include an alignment module configured to align the remote acceleration data with the local acceleration data for processing by the motion compensation module. The alignment module may align the remote acceleration data with the location acceleration data by an alignment of a first coordinate system of the remote motion sensor with a second coordinate system of the mobile device.
Example 8 includes the subject matter of Example 7, wherein the alignment module is further configured to determine an initial orientation of the second coordinate system with respect to the first coordinate system, and to track deviations from the initial orientation based on at least one of local gyroscope data and local magnetometer data.
Example 9 includes the subject matter of any of Examples 1-8, wherein the motion compensation module is further configured to subtract the remote acceleration data from the local acceleration data using a digital audio signal noise cancelation algorithm.
Example 10 includes the subject matter of any of Examples 1-9, wherein the remote motion sensor comprises one or more mounting member selected from a group comprising a magnet, a clamp, a fastening member, and a weighted member.
Example 11 is a method for controlling a user application on a mobile device. The method includes receiving first motion sensor generated data comprising a superposition of user input motion to the mobile device and surrounding environment motion, receiving second motion sensor generated data comprising a measurement of the surrounding environment motion, subtracting the second motion sensor generated data from the first motion sensor generated data to generate compensated motion data, and providing the compensated motion data as user input to the user application on the mobile device.
Example 12 includes the subject matter of Example 11, wherein the first motion sensor generated data comprises a first acceleration waveform, the second motion sensor generated data comprises a second acceleration waveform. The subtracting may include inverting the second acceleration waveform to produce an inverted waveform, and adding the inverted waveform to the first acceleration waveform.
Example 13 includes the subject matter of any of Examples 11-12, and further includes aligning a first coordinate system of a local accelerometer in the mobile device with a second coordinate system of a remote accelerometer configured to measure the surrounding environment motion.
Example 14 includes the subject matter Example 13, and further includes determining an initial orientation of the first coordinate system with respect to the second coordinate system, and tracking deviations from the initial orientation.
Example 15 includes the subject matter of any of Examples 11-14, and further includes tracking the deviations based on at least one of gyroscope data and magnetometer data.
Example 16 includes the subject matter of any of Examples 11-15, wherein the subtracting comprises executing a digital audio signal noise cancelation algorithm.
Example 17 is at least one computer-readable storage medium having stored thereon, the instructions when executed on a machine cause the machine to perform the method of any of Examples 11-16.
Example 18 is an apparatus comprising means to perform the method in any of Examples 11-16.
Example 19 is at least one computer-readable storage medium having stored thereon instructions that, when executed by a processor, cause the processor to perform operations including receiving first motion sensor generated data comprising a superposition of user input motion to the mobile device and surrounding environment motion, receiving second motion sensor generated data comprising a measurement of the surrounding environment motion, subtracting the second motion sensor generated data from the first motion sensor generated data to generate compensated motion data, and providing the compensated motion data as user input to the user application on the mobile device.
Example 20 includes the subject matter of Example 19, wherein the first motion sensor generated data comprises a first acceleration waveform, the second motion sensor generated data comprises a second acceleration waveform, and the subtracting includes inverting the second acceleration waveform to produce an inverted waveform and adding the inverted waveform to the first acceleration waveform.
Example 21 includes the subject matter of any of Examples 19-20, the operations further comprising aligning a first coordinate system of a local accelerometer in the mobile device with a second coordinate system of a remote accelerometer configured to measure the surrounding environment motion.
Example 22 includes the subject matter of any of Examples 19-21, the operations further comprising determining an initial orientation of the first coordinate system with respect to the second coordinate system; and tracking deviations from the initial orientation.
Example 23 includes the subject matter of Example 22, the operations further comprising tracking the deviations based on at least one of gyroscope data and magnetometer data.
Example 24 includes the subject matter of any of Examples 19-23, wherein the subtracting comprises executing a digital audio signal noise cancelation algorithm.
Example 25 is a portable motion sensor device including means for affixing the portable motion sensor device to a vehicle, a processor, and a motion sensor module communicatively coupled to the processor. The motion sensor may include at least one accelerometer to generate acceleration data. The portable motion system may further include a communication interface module communicatively coupled to the processor. The communication interface module configured to selectively establish a communication link with a mobile device and to transmit the acceleration data to the mobile device.
Example 26 includes the subject matter of Example 25, wherein the means for temporarily affixing comprises one or more of a magnet, a clamp, a fastening member, a weighted member, and/or other device for temporarily or permanently affixing the portable motion sensor device to a vehicle.
Example 27 includes the subject matter of any of Examples 25-26, wherein the at least one remote accelerometer comprises a 3-axis accelerometer, and wherein the remote acceleration data indicates movement of the vehicle in an x-axis direction, a y-axis direction, and a z-axis direction of the 3-axis accelerometer.
Example 28 includes the subject matter of any of Examples 25-27, wherein the motion sensor module further comprises at least one of a gyroscope device and a magnetometer device.
Example 29 includes the subject matter of any of Examples 25-28, wherein the communication interface module is configured for serial communications with the mobile device over a communication wire.
Example 30 includes the subject matter of any of Examples 25-29, wherein the communication module comprises a wireless communication subsystem.
Example 31 is a system to improve user input to a mobile device, and optionally includes any of: means for providing remote acceleration data; means for communicating the remote acceleration to a local device; means for aligning, at the local device, the remote acceleration data with local acceleration data; means for subtracting, at the local device, the remote acceleration data from the local acceleration data to generate compensated motion data; and means for providing the compensated motion data to a user application on the local device.
Example 32 includes the subject matter of Example 31, either including or omitting any optional features, and further including, means for providing remote gyroscope data. The means for communicating the remote acceleration data in Example 32 may further communicate the remote gyroscope data to the remote device. In addition, the means for aligning uses the gyroscope data may track deviations from an initial orientation with respect to the local device.
Example 33 includes the subject matter of any of Examples 31-32, and further includes means for affixing the portable motion sensor device to a vehicle.
Example 34 includes the subject matter of Example 33, wherein the means for affixing comprises one or more of a magnet, a clamp, a fastening member, and a weighted member.
The above description provides numerous specific details for a thorough understanding of the embodiments described herein. However, those of skill in the art will recognize that one or more of the specific details may be omitted, or other methods, components, or materials may be used. In some cases, well-known features, structures, or operations are not shown or described in detail.
Furthermore, the described features, operations, or characteristics may be arranged and designed in a wide variety of different configurations and/or combined in any suitable manner in one or more embodiments. Thus, the detailed description of the embodiments of the systems and methods is not intended to limit the scope of the disclosure, as claimed, but is merely representative of possible embodiments of the disclosure. In addition, it will also be readily understood that the order of the steps or actions of the methods described in connection with the embodiments disclosed may be changed as would be apparent to those skilled in the art. Thus, any order in the drawings or Detailed Description is for illustrative purposes only and is not meant to imply a required order, unless specified to require an order.
Embodiments may include various steps, which may be embodied in machine-executable instructions to be executed by a general-purpose or special-purpose computer (or other electronic device). Alternatively, the steps may be performed by hardware components that include specific logic for performing the steps, or by a combination of hardware, software, and/or firmware.
Embodiments may also be provided as a computer program product including a computer-readable storage medium having stored instructions thereon that may be used to program a computer (or other electronic device) to perform processes described herein. The computer-readable storage medium may include, but is not limited to: hard drives, floppy diskettes, optical disks, CD-ROMs, DVD-ROMs, ROMs, RAMs, EPROMs, EEPROMs, magnetic or optical cards, solid-state memory devices, or other types of medium/machine-readable medium suitable for storing electronic instructions.
As used herein, a software module or component may include any type of computer instruction or computer executable code located within a memory device and/or computer-readable storage medium. A software module may, for instance, comprise one or more physical or logical blocks of computer instructions, which may be organized as a routine, program, object, component, data structure, etc., that performs one or more tasks or implements particular abstract data types. In certain embodiments, the described functions of all or a portion of a software module (or simply “module”) may be implemented using circuitry.
In certain embodiments, a particular software module may comprise disparate instructions stored in different locations of a memory device, which together implement the described functionality of the module. Indeed, a module may comprise a single instruction or many instructions, and may be distributed over several different code segments, among different programs, and across several memory devices. Some embodiments may be practiced in a distributed computing environment where tasks are performed by a remote processing device linked through a communications network. In a distributed computing environment, software modules may be located in local and/or remote memory storage devices. In addition, data being tied or rendered together in a database record may be resident in the same memory device, or across several memory devices, and may be linked together in fields of a record in a database across a network.
It will be understood by those having skill in the art that many changes may be made to the details of the above-described embodiments without departing from the underlying principles of the invention. The scope of the present invention should, therefore, be determined only by the following claims.