In computing, a mouse is a pointing device that allows for fine control of a graphical user interface on a computer. Computer mice, however, are external to computers and thus may lack portability. As a result, laptops, netbooks, and other portable computing devices typically do not include mice. Instead, trackpads, pointing sticks, or touchscreens have been used. Though portable, such pointing devices suffer from poor precision and inferior usability when compared to computer mice.
This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter.
Aspects of the present technology are directed to utilizing a portable electronic device that a user already carries as a pointing device for a computer (e.g., a laptop). The portable electronic device can be a smartphone, a personal data assistant, a portable navigation device, and/or other types of electronic device. The portable electronic device can include a processor, a memory, and at least one of an accelerometer, a gyroscope, a magnetometer, or other suitable types of inertial measurement unit. The portable electronic device can be operatively coupled to the computer via a wired (e.g., a USB connection) or wireless (e.g., Bluetooth, WIFI, etc.) connection.
In operation, the inertial measurement unit can detect and measure accelerations of the portable electronic device along at least two dimensions in a coordinate system. The processor, executing instructions stored in the memory, then receives and double integrates the measured accelerations with respect to time to obtain position changes of the portable electronic device along the at least two directions. The processor can then transmit the calculated position changes to the computer via the wired or wireless connection. In response, the computer can move a computer cursor on a graphical user interface in accordance with the calculated position changes of the portable electronic device. As a result, by using the portable electronic device as a pointing device, a user can obtain fine control of the graphical user interface on the computer generally similar to using a computer mouse without carrying additional components (e.g., a computer mouse).
Various embodiments of systems, components, modules, routines, and processes for using portable electronic device for user input are described below. In the following description, example software codes, values, and other specific details are included to provide a thorough understanding of various embodiments of the present technology. A person skilled in the relevant art will also understand that the technology may have additional embodiments. The technology may also be practiced without several of the details of the embodiments described below with reference to
As discussed above, computer mice may lack portability for using with laptops, netbooks, and other portable computing devices. Trackpads, pointing sticks, touchscreens, or other pointing devices, though portable, suffer from poor precision and inferior usability when compared to computer mice. Several embodiments of the present technology are directed to using portable electronic devices that users already carry as pointing devices. As discussed below, portable electronic devices with an inertial measurement unit can be configured to provide generally similar usability as computer mice. As a result, users can achieve fine control of graphical user interfaces on computers without carrying additional components.
As used herein, the term an “inertial measurement unit” generally refers to a measuring component configured to measure at least one of a velocity, orientation, or gravitational force of a physical mass. An inertial measurement unit can include at least one of an accelerometer, a gyroscope, a magnetometer, or other suitable types of inertial sensing element. Also used herein, the term “accelerometer” generally refers to a sensing element configured to measure a proper acceleration of a mass. A proper acceleration measured by an accelerometer is not necessarily a coordinate acceleration (i.e., a rate of change of velocity). For example, an accelerometer at rest on a surface of the Earth would measure a proper acceleration of 9.81 m/s2 due to its weight. By contrast, an accelerometer in free fall would measure a proper acceleration of zero.
As shown in
In operation, a user 101 can move the portable electronic device 120 along the x-, y-, or z-axis on the surface 102. The portable electronic device 120 can then determine a position change along at least the x-, y-, or z-axis from a first position 129a to a second position 129b (shown in phantom lines for clarity), as indicated by the arrow 106. The portable electronic device 120 then transmits the determined position change as a cursor control signal or other suitable types of user input signal to the computer 110 via the connection 104. In response, the computer 110 can use the user input signal to, for example, control a cursor position on the display 116, execute computing commands, and/or perform other suitable actions based on the user input signal. For instance, as shown in
As shown in
As show in
The inertial measurement unit 126 can include at least one of an accelerometer, a gyroscope, a magnetometer, or other suitable types of inertial sensing element. For example, in certain embodiments, the inertial measurement unit 126 can include a piezoelectric, piezo-resistive, or capacitive accelerometer. One suitable accelerometer can be a 3-axis accelerometer (Model No. LIS3DH) provided by STMicroelectronics of Geneva, Switzerland. In other embodiments, the inertial measurement unit 126 can include a pendulous integrating gyroscopic accelerometer, a surface micro-machined capacitive accelerometer, and/or other suitable types of accelerometer.
The device processor 122 can be configured to execute instructions of software components. For example, as shown in
In operation, the input module 132 can accept input data 150 (e.g., sensor readings from the inertial measurement unit 126), and communicates the accepted input data 150 to other components for further processing. The database module 134 organizes records, including measurement records 142, and facilitates storing and retrieving of these records to and from the device memory 123. The measurement records 142 may include contemporaneous and/or historical values of at least one of acceleration, speed, or position in at least one dimension. Any type of database organization may be utilized, including a flat file system, hierarchical database, relational database, or distributed database. The process module 136 analyzes the input data 150 from the input module 132 and/or other data sources to determine a position change of the portable electronic device 120. The output module 138 generates output signals 152 based on the analyzed input data 150 and transmits the output signals 152 as user input signals to the computer 110 via the connection 104. Embodiments of the process module 136 are described in more detail below with reference to
The sensing module 160 is configured to receive the input data 150 and converting the input data 150 into suitable metrics in engineering or other units. For example, the sensing module 160 may receive sensor readings from the inertial measurement unit 126 (
The calculation module 166 may include routines configured to perform various types of calculations to facilitate operation of other modules. In one example, the calculation module 166 can include a subtractor routine that subtract the measured acceleration vector from gravitational acceleration, as follows:
ānet=ā−
where ānet is the net acceleration vector; ā is the measured acceleration vector; and
where
where
where τ is a mean lifetime that may be user selected or otherwise suitably determined. In other examples, the calculation module 166 can include linear regression, polynomial regression, interpolation, extrapolation, and/or other suitable subroutines. In further examples, the calculation module 166 can also include counters, timers, and/or other suitable routines.
The analysis module 162 can be configured to analyze the various sensed and/or calculated metrics to determine if the portable electronic device 120 is moving. For example, the analysis module 162 can compare a current acceleration vector ā to a previous acceleration vector ā, for instance stored in the device memory 123 as the position records 142. If a magnitude change of the acceleration vector ā is greater than a threshold (e.g., a moving average of the magnitude or other suitable values), the analysis module 162 can indicate that the portable electronic device 120 is moving. If the magnitude change of the acceleration vector ā is not greater than the threshold for a window of time (e.g., 0.5 seconds or other suitable values), the analysis module 162 can indicate that the portable electronic device 120 is stationary and can set the current acceleration vector ā as a current value of the gravitational acceleration
The control module 164 may be configured to control the operation of the portable electronic device 120 based on analysis results from the analysis module 162. For example, in one embodiment, if the analysis module 162 indicates that the portable electronic device 120 is moving, the control module 164 can instruct the calculation module 166 to calculate the position change vector
In other embodiments, the control module 164 may also generate the output signals 152 based on user input 154. For example, the user 101 (
Even though the portable electronic device 120 is described above as having various components and modules for determining position changes of the portable electronic device 120, in other embodiments, the computer 110 (
As shown in
The process 200 can also include determining if the portable electronic device 120 is moving at stage 204. In certain embodiments, the determination may be made with the analysis module 162 (
If the portable electronic device 120 is determined to be stationary, the process 200 can optionally include storing the acquired sensor signal at stage 206. In one embodiment, the acquired sensor signal can include an acceleration vector, and can be stored as the gravitational acceleration for the portable electronic device 120 in, for example, the device memory 123 (
If the portable electronic device 120 is determined to be moving, the process 200 can proceed to optional stage 207 to calibrate the sensor signal. In one embodiment, the sensor signal may be calibrated by retrieving the gravitational acceleration of the portable electronic device 120 from, for example, the device memory 123 and subtracting the retrieved gravitational acceleration from the acquired sensor signal. As a result, a net acceleration vector accounting for the gravitational acceleration experienced by the portable electronic device 120. In other embodiments, the calibration of the sensor signal at stage 207 may be omitted.
The process 200 can then include determining a position change of the portable electronic device 120 at stage 208. In certain embodiments, the position change may be determined as a position change vector by double integrating the net acceleration with the calculation module 166, as discussed above with reference to
The process 200 can include outputting the determined position change as pointer data to the computer 110 (
As shown in
To correct for such an error, the acceleration as shown in
Specific embodiments of the technology have been described above for purposes of illustration. However, various modifications may be made without deviating from the foregoing disclosure. In addition, many of the elements of one embodiment may be combined with other embodiments in addition to or in lieu of the elements of the other embodiments. Accordingly, the technology is not limited except as by the appended claims.
Number | Name | Date | Kind |
---|---|---|---|
6104380 | Stork et al. | Aug 2000 | A |
6212296 | Stork et al. | Apr 2001 | B1 |
7870496 | Sherwani | Jan 2011 | B1 |
8065508 | Rubin et al. | Nov 2011 | B2 |
20030117370 | Van Brocklin et al. | Jun 2003 | A1 |
20040236500 | Choi et al. | Nov 2004 | A1 |
20060182316 | Bang et al. | Aug 2006 | A1 |
20060187208 | Wenstrand et al. | Aug 2006 | A1 |
20070171202 | Yang et al. | Jul 2007 | A1 |
20080120448 | Shi et al. | May 2008 | A1 |
20080255795 | Shkolnikov | Oct 2008 | A1 |
20080291163 | Liberty | Nov 2008 | A1 |
20090298538 | Gao et al. | Dec 2009 | A1 |
20100033352 | Chuang et al. | Feb 2010 | A1 |
20100042358 | Moussavi | Feb 2010 | A1 |
20110001663 | Anand et al. | Jan 2011 | A1 |
20110199301 | Zhao et al. | Aug 2011 | A1 |
20110304577 | Brown et al. | Dec 2011 | A1 |
20120179408 | Goto et al. | Jul 2012 | A1 |
20130069917 | Wang et al. | Mar 2013 | A1 |
Entry |
---|
International Preliminary Report on Patentability for PCT Patent Application No. PCT/US2014/020055, Mailed Date: Feb. 9, 2015, Filed Date: Mar. 4, 2014, 11 Pages. |
“International Search Report & Written Opinion for PCT Patent Application No. PCT/US2014/020055”, Mailed Date: Jul. 8, 2014, Filed Date: Mar. 4, 2014, 9 Pages. |
Smurf, Lazy, “WiFi Mouse: Utilize your Android Smartphone as a Mouse, Keyboard and Trackpad”, Retrieved at <<http://www.themobimag.com/wifi-mouse-utilize-smartphone-as-mouse/>>, Retrieved Date: Feb. 28, 2013, pp. 3. |
Dong, Chelsea, “SmartMouse: Turn Your Smart Phone into a Wireless Multi-touch Magic Mouse”, Retrieved at <<http://technode.com/2012/09/17/smartmouse-turn-your-smart-phone-into-a-wireless-multi-touch-magic-mouse/>>, Sep. 17, 2012, pp. 3. |
“DroidPad: Use Your Smartphone as Your PC Mouse or Joystick [Android]”, Retrieved at <<http://www.makeuseof.com/dir/droidpad-use-smartphone-as-mouse/>>, Retrieved Date: Feb. 28, 2013, pp. 4. |
Rhee, Ed, “How to Use Your Android Device as a Cordless Mouse and Keyboard”, Retrieved at <<http://howto.cnet.com/8301-11310—39-20070948-285/how-to-use-your-android-device-as-a-cordless-mouse-and-keyboard/>>, Jun. 15, 2011, pp. 6. |
Kiran R, et al., “Implementing Mobile Phone as a Multi-Purpose Controller using 3D Sensor Technology”, Retrived at <<http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=5563675>>, In 3rd IEEE International Conference on Computer Science and Information Technology, Jul. 9, 2010, pp. 5. |
Number | Date | Country | |
---|---|---|---|
20140253443 A1 | Sep 2014 | US |