Patent Application
20110069007
This application is a continuation of and claims the benefit of International Application Number PCT/GB2009/000647 tiled Mar. 11, 2009, which is hereby incorporated herein by reference in its entirety.
This invention relates to pointing apparatus for use with display equipment.
Pointing devices for controlling the position of a cursor on a display device have been used for decades. A variety of techniques have been used to implement these pointing devices, the devices becoming more sensitive, user-friendly and ergonomic as technology has progressed. With the recent development of reliable wireless communication between human input devices and the equipment they control, a demand has arisen for a reliable wireless pointing device that can be hand-held and control a cursor on a display by simply pointing the device at the display.
Nintendo's® Wii® Remote is one example of such a pointing device. in the Wii Remote there is an image sensor that detects the two groups of infra-red LEDs present in a sensor bar, which is placed next to the screen connected to a Wii console. This allows the Wii Console to determine the direction in which the Wii Remote is pointing and display a cursor on the screen corresponding with the pointing direction. However, such a system requires an expensive, complicated image sensor in every controller that is to be used with the console, and requires the controller to be no more than about 3 m from the Sensor Bar in order to function correctly. Furthermore, relaying the information from the image sensor to the console may require a relatively high bandwidth connection, adding complexity and cost.
Other attempts to provide such a pointing device have been based around the use of gyroscopes in the pointing device to detect its motion. The problem with such an approach is that gyroscope readings tend to drift over time due to rounding errors and noise, leading to inaccurate readings. In the case where such problems are not addressed, the user of such an input device can end up pointing the device substantially away from the display on which the cursor the user is controlling is displayed, resulting in un-intuitive use. Furthermore, the zero point of a gyroscope can drift slightly over time, necessitating dynamic recalibration of the device.
It is an object of this invention to provide a pointing apparatus which will go at least some way in addressing the aforementioned problems; or which will at least provide a novel and useful alternative.
According to a first aspect of the invention, there is provided pointing apparatus comprising a fixed (i.e., stationary) component and a mobile component, the mobile component having a gyroscope for detecting its rotation, the pointing apparatus determining a pointing direction for the mobile component based on the readings from the gyroscope, the pointing apparatus being adapted in use to detect when the mobile component changes state from pointing away from the fixed component to pointing substantially at the fixed component, this detection being used to reset the determined pointing direction for the mobile component by: defining a region of influence around the fixed component; determining the direction of rotation of the mobile component at the point in time when its pointing at the fixed component was detected; extrapolating the negative of that direction of rotation from a point in the region of influence to the edge of the region of influence; and setting the pointing direction of the mobile component, at the point in time when its pointing at the fixed component was detected, as the point where the negative of the direction of rotation met the edge of the region of influence.
Preferably, the pointing apparatus further comprises a light receiver sensitive to at least one wavelength of light and a light emitter of at least one wavelength of light, the light receiver and light emitter being located in a respective one of each component, wherein the detection of when the mobile component is being pointed substantially at the fixed component is carried out using the signals from the light receiver. This provides a simple and effective mechanism for determining when the mobile component is pointed substantially at the fixed component.
Preferably, the light receiver is sensitive to at least one wavelength of infra-red light and the light emitter emits at least one wavelength of infra-red light. The use of infra-red light has the advantage that it is a frequency of light normally invisible to the human eye, and does not interfere with light conditions in the room it is used.
Optionally, the light receiver is a two-dimensional (2D) light receiver such as a camera. This allows richer interaction between a user and a games console, and may also offer additional functionality beyond that used in this invention.
Optionally, the 2D light receiver may be used to establish the starting pointing direction of the mobile component by detecting the initial direction of approach of the light emitter. This allows non-linear movements of the mobile component to be easily tracked.
Preferably, the gyroscope is used in conjunction with the 2D light receiver to establish the starting pointing direction of the mobile component. This provides a robust and accurate method for determining this starting pointing direction.
Alternatively, the starting pointing direction of the mobile component may be determined using an ultrasonic sensor.
Additionally, the 2D light receiver may be used to determine when the mobile component is substantially stationary for recalibrating the zero point of the gyroscope. This allows for more accurate recalibration of the gyroscope.
Optionally, the mobile component may further comprise an accelerometer for detecting when the mobile component is in a steady state, such as being stationary or steadily moving, for recalibrating the zero point of the gyroscope.
According to a second aspect of the invention, there is provided a method for ameliorating the effects of drift in the readings of a pointing apparatus, the pointing apparatus comprising a fixed component and a mobile component, the mobile component further comprising acceleration sensing means such as a gyroscope, the method comprising defining a region of influence around said fixed component; detecting when the mobile component is pointed at said fixed component; determining from the readings of the acceleration sensing means the vector of rotation direction of the movement of the mobile component towards the fixed component; and determining a new starting pointing direction of the mobile component by extrapolating the negative of the vector of rotation direction from the center of the region of influence until it reaches the edge of the region of influence.
Optionally, the method may comprise an additional step of feeding readings from the acceleration sensing means into a recalibration algorithm when the mobile component is substantially stationary.
According to a third aspect of the invention there is provided a pointing apparatus comprising a light receiver sensitive to at least one wavelength of light and a light emitter of at least one wavelength of light, the apparatus comprising a fixed component and a mobile component separate from one another, the light receiver and light emitter being located in a respective one of each component; the mobile component further comprising a gyroscope; the apparatus being arranged such that in use the light receiver and light emitter detect when the mobile component is being pointed substantially at the fixed component, whereupon the apparatus uses readings from the gyroscope to determine the precise pointing direction of the mobile component.
The gyroscope may be used to establish the starting pointing direction of the mobile component, by determining the vector of rotation of the mobile component upon the light receiver detecting light from the light receiver. This provides a simple and effective way of correcting for drift in the gyroscope readings.
The invention will now be described, by way of example, with reference to the drawings in which:
The present embodiments depict a pointing device for use with a video games console. However, the principles described may readily be adapted for use in other applications such as a pointing device for a personal computer, or other electronic devices capable of accepting a pointing input.
The term “gyroscope”, and derivatives thereof, is used herein to refer to acceleration or motion detecting means having a similar function to a gyroscope or gyroscopic sensor. This term is used to include, but is in no means limited to, devices such as gyroscopes; fiber optic gyroscopes; laser gyroscopes; vibrating structure gyroscopes; hemispherical resonator gyroscopes; quartz rate sensors and magnetohydrodynamic sensors. The term is intended to encompass a single such detecting means or group thereof acting jointly or independently. This term is also used to refer to accelerometers arranged to produce similar readings to those of gyroscopes, which may he through an electronic or software interface.
Games console 10 is capable of rendering a cursor 12, or other pointing indicator such as a cross-hair, for display on display device 4. This cursor 12 may be present continuously; only visible when required by an application on games console 10 or only visible when the pointing apparatus is in use. Alternatively, the cursor may he invisible to the user. Mobile component 6 contains a gyroscope fir detecting the movements of mobile component 6 and moving the cursor 12 on display device 4 in a corresponding manner. The readings from the gyroscope in mobile component 6 are communicated to games console 10 by a wireless RF connection (or other communication means). The signal sent to the games console 10 may be the raw readings returned from the sensors in mobile component 6, or alternatively these readings may be processed before sending so as to, for example, reduce the bandwidth needed to send the signals. According to the invention, there is provided a pointing detection mechanism for detecting when mobile component 6 is being pointed substantially at fixed component 8, and hence display device 4. Use of this pointing detection mechanism enables the pointing apparatus to correct for the drift in gyroscope readings due to rounding errors and other errors.
Correction for drift in the gyroscope can be achieved using the following method. Whenever the mobile component 6 changes state from pointing away from display device 4 to pointing towards display device 4, the location of the cursor on the screen can be re-established. To do this, first a vector of rotation is generated based on the x and y components of the readings from the gyroscope in mobile component 6. This gives the direction in which mobile component 6 was moved to bring it to bear on display device 4.
There is defined around the display device 4 a “region of influence” 14, shown in
If the assumption is made that the mobile component 6 is being moved to point towards the center of display device 4, which is not always the case but is generally an acceptable assumption to make, then the vector of rotation 16 can be used to define a point on the edge of this region of influence. This is done by taking the negative value of the vector of rotation 16 and projecting it from the center of display device 4 until it reaches the edge of region of influence 14. The center of display device 4 is preferably at the center of the region of influence, but alternatively may be another point in the region of influence depending on, for example, the shape of the region of influence. The cursor 12 can then be started from this point, providing a cursor 12 which is substantially in the direction pointed by mobile component 6. Once the pointer is re-positioned its position may then he adjusted or moved based on the last gyroscope reading and the time interval since that gyroscope reading. This action helps compensate for synchronization issues between sensor and gyroscope readings. The detection of extremely rapid acceleration and deceleration events when entering and remaining in the region of influence 14 can allow an application on games console 10 to assume that the user is pointing towards the center of display device 4. Under this assumption, the application can choose to reposition the cursor 12 at or near the center of display device 4. As, during use, the mobile component 6 will often be pointed away from the display device 4, this method provides correction for drift in the readings from the acceleration sensors. The pointing detection mechanism can, as will be described later, be achieved simply and easily using low cost components, resulting in a robust pointing apparatus. The simplicity of the pointing detection reduces the bandwidth requirements of the communication link between mobile component 6 and games console 10, and is robust over relatively long distances.
It is also possible to detect an invalid break in the pointing detection, such as might be caused by a person walking in between user 2 and fixed component 8. In this case, the pointing signal will be rapidly removed without any corresponding signals on the gyroscope. It is thus possible to maintain pointing control in this situation until pointing detection stored or a predetermined period of time has passed, which may be a function of how much the mobile component 6 is moved during this time.
A simple arrangement to implement this pointing detection mechanism is an infra-red emitter and an infra-red receiver. However, the pointing detection mechanism may also be implemented using an emitter and sensor operating at other wavelength of light, such as visible light or any of the ultra-violet wavelengths. The term “light” is used herein to refer to both visible light and electromagnetic radiation having wavelengths in the invisible light regions of infra-red and ultra-violet, and any wavelengths in between. In the case that a 2D device, such as a CCD camera, is used as an infra-red receiver, an improved method of dynamically recalibrating the zero reference point of the gyroscope can be carried out. Recalibration of a gyroscope can be carried out using various known filters and averages of readings from the gyroscope. These techniques can be significantly improved by only incorporating readings from the gyroscope when it is known to be stationary, or only slightly moving. The use of a 2D infra-red sensor means that the image of the infra-red emitter can be analyzed to establish when the mobile component 6 has very little or no movement, and thus only feed readings from the gyroscope into the calibration algorithms at these times. Furthermore, the use of a 2D infra-red sensor allows refinement of the starting location of cursor 12 when mobile component 6 is pointed at display device 4, by detecting the image of the infra-red emitter and its location in space.
This detection of when mobile component 6 is not moving can also be achieved by including an accelerometer in mobile component 6. This can detect when the mobile component 6 is either substantially stationary or moving at a steady speed, under both of which conditions the recalibration can be effectively performed. This allows a simple and effective way of recalibrating the gyroscope without the need for a 2D sensor.
In this configuration, the infra-red emitter on fixed unit 8 need only be a simple IR LED or group thereof for sufficient brightness. In order to reduce the possibility of a false reading from the IR sensor 18, the IR LED on fixed unit 8 may be tuned to a particular frequency or oscillating with known frequency. This oscillation may be constant, or occur on request by the games console 10. The oscillation may take place as a full on/off cycle, or alternatively simply vary the intensity of illumination of the IR LED. In the extreme, the LED may be simply blinked off for a brief period on request from the console 10. This can be used to distinguish between when pointing device 6 is pointing towards display device 4 (and hence fixed unit 8) and when it is pointing towards another powerful IR source such as a halogen light-bulb or the sun. The previously mentioned ability to detect an invalid break in the pointing signal can be used to ensure that transmission of control signals is not intermittent in the case that the IR LED is oscillating.
IR sensor 18 is connected to processor 22 that determines when the mobile component 6 is pointing at display device 4. When it is, processor 22 starts sending, via RF transceiver 24, signals from gyroscope 26. These signals may be processed before sending so as to reduce the amount of data sent. Processor 22 may send various signals whether or not the mobile component 6 is pointing at display device 4 depending on the application to which the apparatus is put. Mobile unit 6 further has a button 28 for user input. This button can be used to send a signal to the games console 10 that can be used as an indication to perform an action (such as selecting an option on-screen). Alternatively, button 28 may be used to perform internal functions within mobile unit 6, such as stopping signals being sent from the gyroscope 26 so as to hold the cursor 12 in one position on display device 4. Mobile component 6 may be powered by a rechargeable battery 30, which has connections for providing power to each internal component (not shown). The functions of processor 22 need not necessarily be implemented in mobile component 6, signals from all of the subunits could be continuously sent via the communication link and the processing functions of processor 22 could be carried out in fixed component 8 or games console 10.
The use of a simple IR LED in fixed component 8 for the pointing detection mechanism means that more than one mobile component 6 can be used with the same fixed component 8. This leads to simple implementation of multi-user interaction with games console 10. This is the case whether the IR sensor 18 in mobile unit 6 is a simple photodiode or a 2D CCD camera, although a photodiode offers a simpler, cost effective system.
In this embodiment, with an infra-red LED 32 or array thereof at the front, mobile component 6 is significantly simpler than that of the first embodiment and has lower processing requirements. Consequently, processor 18 may be substantially more simple than in the first embodiment, but is still useful for controlling the various functions of mobile component 6. As in the first embodiment, mobile component 6 has a button 24, an RF transceiver 20, gyroscope 22 and battery 26. Mobile component 6 may also include moving parts such as a vibrator, or light sources to improve its interactivity as a controller.
Both of the described embodiments therefore provide a mechanism for detecting when the mobile component 6 is pointing towards display device 4, allowing the methods described herein to be used with them so as to mitigate problems with gyroscopic drift. The skilled man will realize that there are other modifications that may be made to the described embodiments within the scope of the invention, and the various described features may be combined in different ways other than the examples set out above.
| Number | Date | Country | |
|---|---|---|---|
| Parent | PCT/GB2009/000647 | Mar 2009 | US |
| Child | 12881097 | US |
