The present invention claims priority to TW 100126483, filed on Jul. 26, 2011.
1. Field of Invention
The present invention relates to a remote controllable image display system, and a controller and a processing method therefor; in particular, the present invention relates to such remote controllable image display system, controller and processing method which are capable of determining a position or a rotation of the controller by means of an image sensor.
2. Description of Related Art
Conventional remote controllers of image display systems (TVs, video recorders and so on) are merely capable of inputting simple instructions, such as power-on, power-off, channel change, and volume tuning, etc. to their corresponding image player hosts, but cannot act as a computer mouse to arbitrarily move a cursor on a screen, for example to select a certain icon on a screen for executing a corresponding function or program. Therefore, such conventional remote controllers cannot be used as a tool for controlling functions relating to Internet connection or for playing a video game. On the other hand, although a wireless mouse can be used as a tool for controlling functions relating to Internet connection or for playing a video game, it can not output an instruction by its rotation, and therefore a user can not use it to intuitively output a rotation instruction, for example for rotating an image (such as for browsing photographs) or controlling the rotation of an object in a video game. In addition, the wireless mouse needs to be placed on a table, otherwise it cannot effectively function; hence, it is not suitable for use in an interactive video game system which requires producing/sensing various actions.
In current interactive video game systems, joysticks or remote controllers are often necessary for users to play the games by actions, e.g. to drive a race car, to swing a golf club, etc. Such joystick or remote controller typically includes a gyro, an accelerometer, or an image sensor. In a joystick or remote controller which employs the gyro and the accelerometer, the rotation of the joystick or remote controller can be detected by the cooperation of the gyro and the accelerometer. However, the resolutions of these two sensing devices are insufficient for recognizing a fine or slow action. Furthermore, they have relatively high costs such that the price of the joystick or remote controller cannot be reduced.
In the joystick or remote controller which employs the image sensor to sense images and thereby control a cursor on a screen or select a certain icon on the screen for executing a corresponding function or program, although the resolution of the image sensor is better than that of the gyro and the accelerometer, the image sensor cannot detect the rotation of the joystick or remote controller by the user; such rotation for example may be a rotation action or a rotation instruction, such as fine tuning a quasi-analog knob.
In view of above, the present invention overcomes the foregoing drawbacks by providing a remote controllable image display system, a controller, and a processing method, wherein the coordinate system of the image sensor of the joystick or remote controller is effectively calibrated and the image sensor is capable of correctly detecting the rotation of the joystick or remote controller.
An objective of the present invention is to provide a remote controllable image display system.
Another objective of the present invention is to provide a controller of a remote controllable image display system.
Another objective of the present invention is to provide a processing method for a remote controllable image display system.
To achieve the foregoing objectives, in one aspect, the present invention provides a remote controllable image display system which comprises: an image display showing images generated by a program; a light source generating at least a light beam; a controller controlling a current image according to displacement or rotation of the controller, the controller including at least one image sensor sensing the light beam to obtain a first frame having at least two light spots; and a processor obtaining a first angle between a main operation surface of the controller and a basis plane according to differences between coordinates of the two light spots in the first frame.
In the foregoing remote controllable image display system, preferably, the displacement or the rotation of the controller is transferred to a displacement instruction or a rotation instruction to the current image.
In the foregoing remote controllable image display system, preferably, the displacement instruction adjusts a moving direction according to the first angle.
In the foregoing remote controllable image display system, preferably, the image sensor captures a second frame having the two light spots, and the processor obtains a second angle between the main operation surface and the basis plane from the second frame, whereby the rotation instruction is generated according to the first angle and the second angle.
In one embodiment of the foregoing remote controllable image display system, the light source includes two lighting units each emitting a light beam, and the controller includes one image sensor to receive the light beams.
In another embodiment of the foregoing remote controllable image display system, the light source includes one lighting unit, and the controller includes two image sensors to receive the light beam; the first frame is obtained by superposing two frames captured by the two image sensors.
In the foregoing remote controllable image display system, preferably, execution of the rotation instruction is started by a trigger mechanism.
In one embodiment of the foregoing remote controllable image display system, the rotation instruction changes a rotation angle or a rotation speed of the current image or an object in the current image.
In yet another aspect, the present invention provides a controller of a remote controllable image display system, the controller receiving at least one light beam generated by the system and controlling an image shown by the system according to displacement or rotation of the controller. The controller comprises: at least one image sensor sensing the light beam to obtain a first frame having at least two light spots; and a processor obtaining a first angle between a main operation surface of the controller and a basis plane according to differences between coordinates of the two light spots in the first frame.
In yet another aspect, the present invention provides a processing method for a remote controllable image display system, the method processing an action instruction on an image, the action instruction being generated by displacement or rotation of a controller of the system. The method comprises: receiving at least one light beam generated by the system; obtaining a first frame having at least two light spots according to the light beam; and obtaining a first angle between a main operation surface of the controller and a basis plane according to differences between coordinates of the two light spots in the first frame.
In the foregoing processing method, preferably, the rotation instruction provides one or more of the following functions: changing a rotation angle or a rotation speed of the image or an object in the image; scrolling, switching, magnifying, shrinking, or rotating the image or the object in the image; or adjusting a volume or a playing speed corresponding to the image.
The objectives, technical details, features, and effects of the present invention will be better understood with regard to the detailed description of the embodiments below, with reference to the drawings.
In the following description, embodiments of the present invention will be described by taking an interactive video game system as an example, but it should be noted that the present invention is applicable to other types of video or interactive systems such as for remote-control of a video player, remote-control of a smart TV, browsing homepages on the Internet, etc.
The light source 23 includes lighting units 231 which emit light such as infrared rays; as the user operates the controller 21, the emitted light is received by the image sensor 211 of the controller 21. In this embodiment of the present invention, there are multiple lighting units 231 and one CMOS image sensor chip; the lighting units 231 are arranged so that an image frame captured by the image sensor 211 includes at least two light spots which can be distinguished from one the other. A processor 213 calculates the rotation according to the changes in the positions of the at least two light spots. The processor 213 can be disposed in the controller 21 (as in this embodiment), or can be disposed in the game host 22 in another embodiment. The game host 22 includes a transceiver 221, which communicates with the transceiver 212 of the controller 21 through the radio frequency signals RF1 and RF2 (or IR) for bidirectional data transmission.
Coordinate Calibration
Because there are at least two light spots, the present invention can determine how the controller is held and compensate the rotation offset angle due to the gesture of the user's hand holding the controller; as a result, the wrong movement of the cursor in the prior art as shown in
Following is an example of the detailed formulas for calibrating the coordinate system when the user holds the controller by a rotation offset angle θ, but it should be noted that the present invention is not limited to these formulas. The coordinates of the light spots 41 and 42 in the XY coordinate system are respectively (X1, Y1) and (X2, Y2); the rotation offset angle θ and the transformation between the two coordinate systems can be obtained by the following formulas:
According to the above formulas, the coordinates of the light spots 41 and 42 in the calibrated coordinate system X′Y′ can be obtained and they should be located at the same horizontal level, that is, Y1′=Y2′. In this manner, the coordinates of the light spot are compensated by the offset angle θ during the user's operation.
Rotation Angle Calibration
The gestures for different users to hold the controller are often different, and accordingly, when different users rotate the controller, the rotation angle of the rotation action or the rotation instruct may be wrongly detected. Let us assume that the user holds the controller 21 with the main surface rotated by an offset angle θ, and triggers the rotation function (i.e., starts a meaningful rotation of the controller 21) whereby the controller 21 is rotated clockwise by an angle θ″. Referring to
Rotation Action or Rotation Instruction
Because there may be meaningless rotations of the controller 21 (e.g. by the gesture of the user), the rotation function (rotation action or rotation instruction) should preferably be triggered and executed only when desired. To this end, according to the present invention, one embodiment is to provide a button or a switch on the controller 21 for the user to trigger the rotation function by pressing it. Another embodiment is to provide a specific area or icon on the screen, so that when the controller 21 moves the cursor to the specific area or icon, the rotation function is triggered.
Referring to
Applications of Rotation Action or Instruction
2. Browsing Pictures
3. Listening to Music
4. The rotation function can be used in a game of balance, such as maintaining the center of gravity, or, in a car race game for driving the wheel.
5. Watching TV
6. Seeing Movie
7. Reading Electrical Book
The present invention has been described in considerable detail with reference to certain preferred embodiments thereof. It should be understood that the description is for illustrative purpose, not for limiting the scope of the present invention. Those skilled in this art can readily conceive variations and modifications within the spirit of the present invention. For example, the rotation function can be triggered by other mechanisms using hardware or software. For another example, an image frame having two light spots can be obtained by one image sensor and multiple lighting units, or by superposing two image frames obtained by two image sensors from a single lighting unit, or by multiple image sensors and multiple lighting units. The applications of the rotation action or the rotation instruction are not limited to the above enumerated embodiments; the rotation action or rotation instruction can be applied to executing other functions such as screen brightness and contrast adjustments. The calibrations mentioned above can be processed by an external processor instead of the processor in the controller. Thus, the present invention should cover all such and other modifications and variations, which should be interpreted to fall within the scope of the following claims and their equivalents.
Number | Date | Country | Kind |
---|---|---|---|
100126483 | Jul 2011 | TW | national |