This invention relates to electronic devices, more specifically, to the control of display content for electronic devices.
Portable electronic devices are generally controlled by input devices located on the portable device, such as a button, or pen touch computing. A typical portable electronic device 20 is depicted in Prior Art
Other methods have been developed to control the display of a portable electronic device with the use of user initiated motion. One such method invented by the applicant is embodied in U.S. Pat. No. 6,184,847, which is represented by prior art
Additionally, the calculation required from the input of several accelerometers can add greatly to the computational problems of controlling a display with such types of motions sensors. Although, simple threshold types of motions such as a 45 degree tilt can easily control a display for simple commands such a scroll left or scroll right, calculation that requires more fine tuning will use a great deal more computing power to integrate the data from the three accelerometer or gyroscope readings. For example the motions of the portable device will result in logic having to integrate the voltage readings from two accelerometers.
The accelerometer can be used to measure motion in any direction, including a “tilting” of the device. The distance an object has been moved is the integral of velocity over time. Essentially, the integral is a fairly complicated calculation on a device that may be running less than 8 M or RAM and while single calculations do not present a serious problem, continuous calculation from constant movement will require a great deal of system resources.
Normally, to measure the distance and object has traveled with accelerometers, it is necessarily to integrate twice. This amount of integration can lead to a great deal of error, even with the most accurate of accelerometers, what is needed is a way to predict some of the parameters of motion based on a normal users movement, instead of integrating several accelerometers at the same time.
What is needed is a method of calculating the motion or position of a portable device that does not require the expense and computational complexity of multiple accelerometers or other motion sensor, in order to control the display of the device.
The present invention addresses the above described problems by simplifying the assumptions on possible motions made by a user. This simplification process can take place by making an assumption that the motion of the device will travel along a certain preferential motion arc, which will be referred to herein as the “Javal arc.” Calculations of motions of the device can be made based on these Javal arcs. Additionally, the double integration of each accelerometer is now eliminated and the expensive accelerometers are replaced by two magnetic sensors.
The present invention is illustrated by way of example, and not by way of limitation, in the figures of the accompanying drawings and in which like reference numerals refer to similar elements and in which:
The Javal arc is mainly used in connection with optical measurement devices and it is generally used as a medical term. However, since the display of a computer or electronic device has to do with vision, the Javal arc can be applied to the system of screen display viewing as well. Javal was a French physician who developed measurements for lines of sight, which has been applied mainly to opthamalogic devices for almost a century. However, the Javal arc can be used to measure the natural motion of lines of sight and motion related thereto. In the present invention the Javal arc is used as a computational assumption or a constant in order to eliminate expensive accelerometers and valuable computing power in a portable electronic device. There are 3 components to the Javal arc that extends out from the comfortably stretched arm of the user.
In order to avoid the difficulty of double integration in the calculation method for determining the motion of the device, a simpler and more accurate method of the present invention teaches a way to more accurately calculate movement from less expensive components, such as magnetometers. In one embodiment this control method requires only one accelerometer, and two inexpensive magnetometers. In yet another embodiment, two inexpensive magnetometers and a gyroscope may be used in the place of an accelerometer.
Distance traveled is the integral of velocity over time. Needless to say, for motions that last a fraction of a second, this can be very taxing on the calculation resources on a small electronic device where computing resources are already scarce. It can also be wildly inaccurate as can be appreciated by those skilled in the art.
Generally speaking, a small portable display screen will not have the capacity to display an entire desktop computer screen worth of graphics and text at one time. Although, some solutions, like for those on PDAs have proposed a new set of display requirements, such as web clipping, which are wholly inadequate to display the graphics of a normal computer display. The invention teaches a way to control the buffered display of an entire virtual display which will not fit on the display screen of the device, using the Javal arc motion and related computation aspects described herein.
Control of the display 26 of the portable electronic device 20 will be effectuated by motion of the device by a user. This is described above in the prior art description of control of the display of a device through a motion of the user. However, the user of a portable electronic device will not move the device equivalent to a normal Cartesian set of coordinates, or any other perfectly ordered geometrical system (cylindrical, etc.). The user will move the device in a manner that is natural to the user's eye and hand coordination, and is described with respect to a Javal arc movement coordinate system.
The present invention uses the expression “virtual display space” to define the entire of amount of graphics that may be presented.
Other expressions used in this capacity are “performance space” and “virtual desktop.” The expression “virtual screen” is a related concept to virtual display space, but is more limited in that the virtual screen has the equivalent virtual display space of only one desktop screen. The present invention distinguishes between virtual display space and a virtual screen, because of the manner in which the display memory will store and access such data. For example, a PDA graphic memory may store an entire virtual screen in a easily accessed buffer, but other screens are stored in cache or loaded as they become needed in the buffer. A more powerful device with a larger display buffer may be able to load an entire document. For the purposed of this disclosure, the expression virtual display space is primarily used. When discussing content from the Internet or an equivalent computer display, the expression virtual screen may be used but is distinguished from virtual display space.
Referring now to
Referring now to
Thus the “panning” motion of the portable electronic device 20 will generally travel on a portion of a spheroid 101 like coordinate system 100 in which the two Javal arcs which meet at the central viewing point 199 and define the surface of motion in which the user moves the device. The user does not need to move the device along either of the arcs 104110 to effectuate control of the display of the device but will tend to move the device anywhere on the surface of the spheroid 101. The angle of the movement along either arc 104110 will be proportional the angle that the user 10 is holding the device 20 away from its normal axis. These angles are described in detail below.
Referring now to
Referring now to
In
The invention includes logic means 2000, which is shown in
Because accelerometers can effectively measure motion in one direction without integration, that one direction can be easily calculated, especially in the accelerometers most sensitive mode which is vertical because of the measure of gravity. Normally, to detect the motion in two directions, two accelerometers can be placed on a device, but this requires a great deal of integration which leads to enormous inaccuracies. In the present invention, the single accelerometer 1002 will measure the vertical movement or position of the portable electronic device 20.
Magnetometers can also easily measure the angle of the device 20 relative to its normal orientation as depicted in
The resulting voltage inputs to 2200 and 2300 from the magnetometers calculations of the changes in the magnetic field H, generally means that each axis of sensitivity of both H(x) and H(y) will have to be determined from H(⊥) and H(//) based on the angles α(2) 40 and α(4) 60 from the axes. The formula for the determination of the resulting voltage signal output into the inputs 2200 and 2300 is included herein.
The voltage signals from the changes in the respective magnetic fields H(x) and H(y) and which are detected by the magnetometers 1004 and 1006, are compiled with the output from the accelerometer 1002 and then compiled in the motion calculation unit 2500 based on the formulas described herein. It should be noted that the entire motion detection logic and control system 2000 is virtual, and any one of the components of the system may be located at any place through out the hardware of the device 20. For example, the accelerometer input unit 2100 may actually be located on the accelerometer 1002 instead of in the ASIC 2001 or in a discrete location in the distributed system alternative. The logic of the motion calculation unit 2500 for determining the motion and position of the device of is based on formulas disclosed herein.
One method for calculating position and distance traveled along the Javal spheroid 101 is the instantaneous velocity of an origin point 2 on the electronic device 20 is defined as: V(x)=R*dα(1)/dt; V(y)=R*sin α(1)*dα(2)/dt in the “portrait” orientation, and V(x)=R*dα(4)/dt; V(y)=R*sin α(1)*dα(3)/dt in the “landscape” orientation. Although this requires the conversion of coordinates based on the changing coordinate systems, the conversion of the Cartesian coordinates to Spherical coordinates in well known to those skilled in the art and is described in appendix A in the additional disclosure of the invention.
Referring now to
Depending on the user preferences, the movement and position of the device can be programmed to control the display differently. For example a user may wish to have the upward movement along the vertical Javal arc 104 to have the PDA content screen 600 to pan downward. Other users may wish to have such movement pan upward in an alternate embodiment.
In other preferred embodiments, means for detecting the distance a user 10 is away from the device may include echo location functions, light intensity functions. Zooming functions may be activated in different ways depending on the efficiency of manufacture. Other useful features of the invention may include such embodiments which include (1) threshold control for the elimination of the movement “noise;” (2) threshold control for the activation of the display control mechanism, (3) alternate display controls, such as horizontal movement is a display vertical scroll and vice versa; (4) lock and unlock controls of the arc movement navigation (where the horizontal is locked, but the vertical continues to scroll) and other useful navigation features.
In another embodiment represented by
When a user holding a PDA or smart phone device in his hand, moves it around while looking at the device display, the following assumptions apply:
These constraints define a situation in which the center of the display O1 will move over the surface of its sphere, centered at the middle point of the segment joining the user's two eyes O, and of radius Rconf; with the display remaining tangent to the sphere at all times.
It can be assumed, without any loss of generality, a global frame of reference (O, X, Y, Z) so that OZ is vertical with the gravity vector oriented downward, and so that the Earth magnetic field {right arrow over (H)} is in the (OX, OZ) plane. One defines a device frame that (O1, X111, Y111, Z111) attached to the device display.
The device frame is related to the global frame by three successive rotations. A first rotation of the global frame around its OZ axis by angle Φ (the azimuth angle), defines the first intermediate frame (O1, X1, Y1, Z1), with Z1=Z. Then a rotation of the first intermediate frame around the Y1 axis by an angle θ (the tilt angle), defines the second intermediate frame (O1, X11, Y11, Z11), with Y11=Y1. Finally a rotation of the second intermediate frame around the Z11 axis by an angle γ (the orientation angle), defines the device frame (O1, X111, Y111, Z111), with Z111=Z11.
γ=zero corresponds to a portrait orientation of the device, while γ=π/2 corresponds to a landscape orientation.
Assuming that the motion of the device in space is constrained as previously explained, the instantaneous velocity of O1 in the global frame, projected along the base vectors of the device frame, is given by:
From the previous equations it can be seen that the determination of the instantaneous velocity of O1 has been reduced to the determination of the time derivatives of θ and Φ.
Model of the Sensor Assembly:
In order to simplify the formulation of the problem, and without any significant loss of generality, one assumes that all the sensors are in the plane of the display (the (O′X′″, O′Y′″) plane).
One infers that:
{right arrow over (μ)}α=Cos ξα{right arrow over (μ)}X′″+Sin ξα{right arrow over (μ)}Y′″ (5)
With the angle ξα assumed to be known (calibration).
Tilt and Orientation Determination:
Assuming two accelerometers in the (O′X′″, O′Y′″) plane, with sensitivity axis {right arrow over (μ)}α1, {right arrow over (μ)}α2; one has:
{right arrow over (μ)}α1=Cos ξα1{right arrow over (μ)}X′″+Sin ξα1{right arrow over (μ)}Y′″ (9.1)
{right arrow over (μ)}α2=Cos ξα2{right arrow over (μ)}X′″+Sin ξα2{right arrow over (μ)}Y′″ (9.2)
And the output of these two sensors are given by:
Vα1=Vα10+Sα1{right arrow over (g)}·{right arrow over (μ)}α1 (10.1)
Vα2=Vα20+Sα2·{right arrow over (μ)}α2 (10.2)
Where the offset voltage Vα0 (in V), and the sensitivities Sα (in V m−1, δ), are assumed to be known (calibration).
From (10), one has:
From (9), one has:
γα1={right arrow over (g)}·(Cos ξα1{right arrow over (μ)}X′″+Sin ξα1{right arrow over (μ)}Y′″) (12.1)
γα2={right arrow over (g)}·(Cos ξα2{right arrow over (μ)}X′″+Sin ξα2{right arrow over (μ)}Y′″) (12.2)
i.e.,
Cos ξα1({right arrow over (g)}{right arrow over (μ)}X′″)+Sin ξα1({right arrow over (g)}{right arrow over (μ)}Y′″)=γα1
Cos ξα2({right arrow over (g)}{right arrow over (μ)}X′″)+Sin ξα2({right arrow over (g)}{right arrow over (μ)}Y′″)=γα2
i.e.,
One introduces:
γX′″={right arrow over (g)}{right arrow over (μ)}X′″
γY′″={right arrow over (g)}{right arrow over (μ)}Y′″ (14)
From (7) and (8), one has:
{right arrow over (μ)}X′″=Cos γ{right arrow over (μ)}X″+Sin γ{right arrow over (μ)}Y″
{right arrow over (μ)}Y′″=−Sin γ{right arrow over (μ)}X″+Cos γ{right arrow over (μ)}Y″ (15)
{right arrow over (μ)}X″=Cos θ{right arrow over (μ)}X′ Sin θ{right arrow over (μ)}Z′
{right arrow over (μ)}Y″={right arrow over (μ)}Y′
and {right arrow over (g)}=−{right arrow over (μ)}Z′ (17)
γX′″=+g Cos γ Sin θ (18.1)
γY′″=−g Sin γ Sin θ (18.2)
i.e.,
i.e.,
i.e.,
As by definition
and
If
for portrait orientation
for landscape orientation with ε=Sig (Sin γ)
Azimuth Determination:
One has the in-plane component of the local magnetic field given by:
{right arrow over (H)}″=(Hpoc Cos δ){right arrow over (μ)}X (23)
Where the amplitude Hpoc and the inclination angle δ may be slowly varying functions off position and time.
And the perpendicular component:
{right arrow over (H)}⊥=(−Hpoc Sin δ){right arrow over (μ)}Z (24)
Assuming two magnometers in the (O′X′″, O′Y′″) plane, with sensitivity axis {right arrow over (μ)}M1 and {right arrow over (μ)}M2, one has:
{right arrow over (μ)}M1=Cos ξM1{right arrow over (μ)}X′″+Sin ξM1{right arrow over (μ)}Y′″ (25.1)
{right arrow over (μ)}M2=Cos ξM1{right arrow over (μ)}X′″+Sin ξM2{right arrow over (μ)}Y′″ (25.2)
And the output of these two sensors are given by:
VM1=V0M1+SM1{{right arrow over (H)}//·{right arrow over (μ)}M1+{right arrow over (H)}⊥·{right arrow over (μ)}M1} (26.1)
VM2=V0M2+SM2{{right arrow over (H)}//·{right arrow over (μ)}M2+{right arrow over (H)}⊥·{right arrow over (μ)}M2} (26.2)
Where the offset voltage V0α (in V), and the sensitivities Sα (in V·Oe−1) are assumed to be known (calibration).
From (26), one has:
i.e., from (15), one has:
i.e.,
h
X′″=({right arrow over (H)}//+{right arrow over (H)}⊥)·{right arrow over (μ)}X′″ (30.1)
hY′″=({right arrow over (H)}//+{right arrow over (H)}⊥)·{right arrow over (μ)}Y′″ (30.2)
For (6), (7) and (8), one has:
{right arrow over (μ)}X′″=Cos γ Cos θ Cos Φ{right arrow over (μ)}X+Cos γ Cos θ Sin Φ{right arrow over (μ)}Y
Cos γ Sin θ{right arrow over (μ)}Z−Sin γ Sin Φ{right arrow over (μ)}X+Sin γ Cos Φ{right arrow over (μ)}Y=(Cos γ Cos θ Cos Φ−Sin γ
Sin Φ){right arrow over (μ)}X+A{right arrow over (μ)}Y−Cos γ Sin θ{right arrow over (μ)}Z (31.1)
{right arrow over (μ)}Y′″=(Sin γ Cos θ Cos Φ−Cos γ Sin Φ){right arrow over (μ)}X+B{right arrow over (μ)}Y−Sin γ Sin θ{right arrow over (μ)}Z (31.2)
i.e.,
hX′″=H//(Cos γ Cos θ Cos Φ−Sin γ Sin Φ)−H⊥ Cos γSin θ (32.1)
hy′″=H//(Sin γ Cos θ Cos Φ−Cos γ Sin Φ)+H⊥ Sin γ Sin θ (32.2)
Sin θ is known from (21):
Therefore Cos θ=ε(1−Sin2 θ)1/2 (33)
And Sin γ and Cos γ are known from (19):
Thus,
ε Cos γ|Cos θ|(H// Cos Φ)−Sin γ(H// Sin Φ)=[hx′″−H⊥ Cos γ Sin θ]
ε Sin γ|Cos θ|(H// Cos Φ)−Cos γ(H// Sin Φ)=[hy′″−H⊥ Sin γ Sin θ]
i.e.,
hX′″=ε Cos γ|Cos θ|(H// Cos Φ)−Sin γ(H// Sin Φ)+H⊥ Cos γ Sin θ (35.1)
hY′″=ε Sin γ|Cos θ|(H// Cos Φ)−Cos γ(H// Sin Φ)+H⊥ Sin γ Sin θ (35.2)
Sin γhX′″−Cos γhY′″=2ε Sin γ Cos γ|Cos θ|(H// Cos Φ)−Cos2 γ Sin2 γ(H// Sin Φ)
Sin γhX′″+Cos γhY′″=−(H// Sin Φ)+2H⊥ Sin γ Cos γ Sin θ
i.e.,
2ε Sin γ Cos γ|Cos θ|(H// Cos Φ)+Cos2 γ Sin2 γ(H// Sin Φ)=(Sin γhX′″−Cos γhY′″) (36.1)
(H// Sin Φ)+(2 Sin γ Cos γ Sin θ)H⊥=(Sin γhX′″+Cos γhY′″) (36.2)
H⊥=εM(1−H//2)1/2 (37)
Where εM is known (north or south of magnetic equator).
Thus:
2ε Sin γ Cos γ|Cos θ|(H// Cos Φ)+(Cos2 γ−Sin2 γ)(H// Sin Φ)=(Sin γhX′″−Cos γhY′″) (38.1)
(H// Sin Φ)+εM(Sin γ Cos γ Sin θ)(Hpoc2−H//2)1/2=(Sin γhX′″+Cos γhY′″) (38.2)
If the magnetic inclination δ and the field amplitude Hpoc are assumed to be known:
H//=Hpoc Cos δ (9.1)
H⊥=Hpoc Sin δ (39.2)
[2H// Sin γ Cos γ(ε|Cos θ|)]Cos Φ+(Cos2 γ−Sin2 γ)(H// Sin Φ)=(Sin γhX′″−Cos γhY′″) (40.1)
H// Sin Φ+2(Sin γ Cos γ Sin θ)H⊥=(Sin γhX′″+Cos γhY′″) (40.2)
i.e.,
H// Sin Φ=2(Sin γ Cos γ Sin θ)H⊥−(Sin γhX′″+Cos γhY′″) (41.1)
γ≃0 then H// Sin Φ≃Cos γhY′″
hX′″≃ε|Cos θ|(H// Cos Φ)+H⊥ Sin θ
i.e.,
ε|Cos θ|(H// Cos Φ)≃(hX′″−H⊥ Sin θ)
(Cos γhX′″−Sin γhy′″)=ε|Cos θ|(H// Cos Φ)+(Cos2 γ−Sin2 γ)H⊥ Sin θ
(Sin γhX′″+Cos γhy′″=−(H// Sin Φ)+2(Sin γ Cos γ)H⊥ Sin θ
i.e.,
ε|Cos θ|(H// Cos Φ)+(Cos2 γ−Sin2 γ)(H⊥ Sin Φ)=(Cos γhX′″−Sin θhy′″) (42.1)
−(H// Sin Φ)+2(Sin γ Cos γ)(H⊥ Sin θ)=(Sin γhX′″+Cos γhy′″) (42.2)
If γ=0, one has in first approximation:
ε|Cos θ|(H// Cos Φ)+(H⊥ Sin θ)=hx′″
−H// Sin Φ=hy′″
i.e.,
i.e.,
Number | Date | Country | Kind |
---|---|---|---|
PCT/US02/24981 | Aug 2002 | WO | international |
Number | Name | Date | Kind |
---|---|---|---|
5506605 | Paley | Apr 1996 | A |
5602566 | Motosyuku et al. | Feb 1997 | A |
6072467 | Walker | Jun 2000 | A |
6184847 | Fateh et al. | Feb 2001 | B1 |
6201554 | Lands | Mar 2001 | B1 |
6288704 | Flack et al. | Sep 2001 | B1 |
6466198 | Feinstein | Oct 2002 | B1 |
6690358 | Kaplan | Feb 2004 | B2 |
6847351 | Noguera | Jan 2005 | B2 |
Number | Date | Country | |
---|---|---|---|
20040227742 A1 | Nov 2004 | US |