Patent Application
20080291179
The invention relates to a light pen user interface technology which may be used with large area non-CRT displays.
A traditional light pen is an input device which comprises a light sensor for detecting light emitted from a cathode ray tube (CRT) display. The light pen technology is nowadays mostly used in gaming arcades as input device, for example as a gun. A light pen is connected with a computer, for example by a wire, and comprises a switch for inducing an action to be performed by the computer, for example for firing shots in a video game. When an activation of the switch is detected by the computer, the scan time of the electron beam inside a CRT display tube from the starting point of the electron beam on the screen until the light sensor detects light is measured. Since the “way” of the electron beam on the screen and the coordinates of its starting point are known, the horizontal and vertical position of the light pen pointer on the display screen may be calculated from the measured time. In this technology, the display serves as light emitter and the light pen as receiver. This technology requires a raster-scanned display such as a CRT display. However, it does not work with modern non-CRT display technologies such as the liquid crystal display (LCD), thin film transistor (TFT) or plasma display technology. EP 0 786 107 B1 discloses a light pen input system which can be used with a LCD display. The disclosed system comprises a light sensing device with a planar array of light sensing elements in rows and columns. However, the required light sensing elements make the system costly to implement.
It is an object of the present invention to provide an improved light pen input system and method, particularly for use with large area non-CRT displays.
In order to achieve the object defined above, the invention provides a light pen input system, wherein the system comprises the following characteristic features:
a light pen being adapted to generate at least one scanning light line sweep for scanning a surface,
a light sensing element placed at a known position for detecting a scanning light line,
time measurement means being adapted to measure a time duration t−tstart from receiving a scanning start signal until receiving a light detection signal from the light sensing element, and
position detection means being adapted to determine the coordinates of the pointing position of the light pen with regard to the surface from measured time durations t−tstart.
In order to achieve the object defined above, the invention further provides a method for determining the pointing position of a light pen, wherein the method comprises the following characteristic features:
the light pen generates at least one light line sweep for scanning a surface,
a light sensing element placed at a known position detects a scanning light line,
time measurement means measure a time duration t−tstart from receiving a scanning start signal until receiving a light detection signal from the light sensing element, and position detection means determine the coordinates of the pointing position of the light pen with regard to the surface from measured time durations t−tstart.
The characteristic features according to the invention provide the advantage that the invention may be used with non-CRT displays, particularly large area non-CRT displays such as liquid crystal displays (LCD), thin film transistor (TFT) or plasma displays. In principle, the invention is display independent. Furthermore, the invention does not require several light sensing elements arranged in an array with rows and columns as known from EP 0 786 107 B1 and, therefore, may be implemented at low cost. On the light receiving side, i.e. on the surface or display side, the invention merely requires at least one light sensing element placed at a known position, for example located at a border of a surface such as a display screen. Furthermore, the invention makes the interaction with a large area display by pointing much easier than with the traditional light pen technology since it does not require to point inside the borders of a display screen or of a light sensing device. Thus, the invention is principally independent from the size of a display screen. It may be used either with large area or small area display screens since the determination of the pointing position of the light pen is in principle independent from pointing inside or outside the display screen. If a user points outside a surface such as a large area display screen, the pointing position may also be detected by the system according to the invention and, for example, mapped back as a “mouse pointer” movement on the screen. Thus, users do not have to point so precisely, which increases the ease of operation, particularly with large area display screens. It should be noted that a display is not a prerequisite for this invention. The invention is suitable for consumer electronics, industrial, military, and medical applications as a pointing, orientation and positioning method in general. When used with consumer electronics applications, the invention is suitable to be implemented on existing remote controls and electronic systems such as TV systems.
It should be noted that the term “light pen” means a pointing device which emits light in contrast to traditional light pens which receive light emitted from a CRT display.
The term “scanning light line” used herein should be understood as a line of light projected on a surface similar to a light line generated by a typical barcode scanner.
The term “scanning light line sweep” means that the light line scans a predefined area when projected on the surface, i.e., a light line scan is moved in a certain direction over the predefined area in order to scan the area. In case of two scanning light lines for scanning the predefined area in two different directions, the scanning light lines may be moved in an orthogonal direction with respect to each other over the predefined area. Thus, the predefined area may be scanned for example in a horizontal and a vertical direction.
A “sweep” means moving the scanning light line from a scan starting position to a scan end position. The scan start and end position of the line determine the predefined area to be scanned.
The term “light sensing device” comprises any device sensitive to light emitted by the light pen such as photo detectors or photodiodes.
The term “position detection means” comprises any means able to determine the pointing position of the light pen. Particularly, the position detection means comprise an algorithm for calculating the pointing position from the time durations measured by the time measurement means. This algorithm may be adapted to calculate the distance of the pointing position of the light pen with regard to the surface, i.e., the x- and y-coordinates of the pointing position in the plane of the surface from the time durations measured by the time measurement means. The x- and y-coordinates may be calculated from the measured time durations and the known velocity of the sweeps of scanning light lines over the surface. It should be noted that the mentioned algorithm may be implemented in soft- or hardware.
The term “pointing position” may be explained by the position of an intersection of the pointing axis of a light point and the plane of the surface when the light pen is directed to the surface similar to the spot generated by a laser pointer pointing on a surface.
The coordinates of the pointing position are the two-dimensional Cartesian coordinates in the plane of the surface, i.e., the x- and y-coordinate of a point in the plane of the surface.
The basic idea of the invention is detecting a pointing position of a light pen with regard to a surface in that the light pen scans a predetermined area with at least one scanning light line and the time duration from starting a scan until detecting light at a certain and known position on the border of the surface is measured. With this information and few further known parameters such as the time duration of an entire sweep or the coordinates of the light sensing element in the plane of the surface, the x- and y-coordinates of the pointing position of the light pen may be determined.
With one scanning light line sweep, the surface may be scanned in one direction, and the position detection means may determine a coordinate of the pointing position in the sweep or scanning direction. In order to determine two coordinates in the plane of the surface, the light pen may be adapted to generate two scanning light line sweeps for scanning the surface in two different directions. Thus, the pointing position of the light pen may be determined with a higher accuracy than with one scanning light line sweep. The two scanning light lines generated by the light pen may have different wavelengths which allows a simultaneous scanning of the surface. This may be faster than a serial scanning with two scanning light lines having the same wavelength.
Since the invention may be preferably applied to display screens, the surface has usually a rectangular shape. Thus, it is preferred if one of the two scanning light lines is moved in a first direction, and the other one of the two scanning light lines is moved in a second direction which is orthogonal to the first direction. Thus, the surface may be scanned in a horizontal direction by one of two scanning light lines and in a vertical direction by the other one of the two scanning light lines.
Preferably, each of the two moving directions of the scanning light lines corresponds to a coordinate in the two-dimensional coordinate system of the plane of the surface. For example, the first direction may be used to determine the x-coordinate of the pointing position and the second direction may be used to determine the y-coordinate. In order to determine these coordinates, the position detection means may be adapted to determine a x-coordinate of the pointing position of the light pen from a measured time duration tx−txstart triggered from the scanning light line sweep in the first direction and a y-coordinate of the pointing position of the light pen on the surface from a measured time duration ty−tystart triggered from the scanning light line sweep in the second direction.
The light pen input system according to the invention may also be used to determine movements of the light pen, i.e., when the light pen position changes from one to another position due to a movement of the light pen, for example when the light pen is used to control a cursor of a graphical user interface shown on a display screen. For performing this task, the position detection means may be adapted to determine a movement of the light pen from at least two consecutive measured time durations t0−t0start and t1−t1start. The two consecutive measured time durations may correspond to two scans. Therefore, when the second scan starts after the first scan and the position of the light pen has changed, the second measured time duration corresponding to the second scan differs from the first measured time duration. Since the time durations correspond to the pointing position, this means that the distance between the light pen position and the light sensing element has changed. The difference of the distances derived from the two measured time durations may then be mapped to a corresponding movement of the light pen.
In order to accurately determine the pointing position, the light pen may be adapted so that the time duration of a scanning light line sweep is about 5 to 10 milliseconds. This means that a scan in each direction performed by the light pen takes about 5 to 10 milliseconds. Usually, this time is so short that it is unlikely that a user moves the light pen within a shorter time and the determined pointing position is inaccurate. However, it should be noted that the time duration of a scanning light line sweep may be selected from a time range from about 1 to about 100 ms, wherein a short time duration has the advantage that movements of the light pen may be detected faster than with a long time duration.
The determination of the pointing position of the light pen may be simplified if the angular velocity of movement of a generated scanning light line is nearly constant and/or predefined. For example, the distances corresponding to the measured time durations can be calculated by multiplying the time durations with the known velocity of movement of a scanning light line.
For detecting a skew of the light pen, for example if a user rotates the light pen such that the surface is not scanned in a horizontal and vertical direction, a second light sensing element being aligned to the first light sensing element may be provided for detecting skew. The time measurement means may be further adapted to measure a first time duration t1−t1start from receiving a scanning start signal until receiving a light detection signal from the first light sensing element and a second time duration t2−t2start from receiving the scanning start signal until receiving a light detection signal from the second light sensing element, and the position detection means may be adapted to determine a skew of the light pen with regard to the surface from the first and second measured time durations.
Preferably, the surface has a rectangular or quadratic shape. In such a case, both light sensing elements may be located in opposite corners of the surface and the position detection means may be further adapted to determine the size of the surface from the first and second measured time durations, to store the determined size, and to use the determined size for the determination of the coordinates of the pointing position of the light pen with regard to the surface.
The light sensing element may be located at a border or corner of the surface, integrated into, located underneath, besides or in front of the surface. It should be located such that it is within the scanning range of a scanning light line sweep when a user points with the light pen at or near the surface in order to determine the pointing position.
According to a further aspect, the invention relates to a light pen being adapted for use with a light pen input system according to any of the before discussed embodiments, wherein the light pen comprises at least one laser and optical means for generating two scanning light lines.
According to an embodiment of the light pen, it may comprise two lasers capable of generating laser beams with different wavelengths. Instead of two lasers, the light pen may also comprise one laser and the optical means may comprise a beam splitter for splitting the laser beam generated by the laser into two laser beams.
According to an embodiment of the light pen, the optical means may comprise a first barrel lens for creating a vertical scanning light line, a second barrel lens for creating a horizontal scanning light line, a first movable mirror for sweeping the vertical scanning light line over a predefined area, and a second movable mirror for sweeping the horizontal scanning light line over a predefined area.
According to a further embodiment, the light pen may be adapted for transmitting a scanning start or synchronization signal to the time measurement means. The scanning start or synchronization signal may be transmitted either over a wire line or wireless connection to the time measurement means. For example, the light pen may be connected by a wire to a display screen unit comprising the time measurement means and the position detection means, or it may comprise a wireless module for communicating with a wireless module of the display screen unit and transmitting the scanning start signal over a wireless communication connection. For example, the scanning start signal may be transmitted by light to the timing measurement means, for example by an infrared light emitting diode contained in the light pen which generates a infrared pulse which is transmitted to an infrared receiver in a display screen unit comprising the time measurement means. It should be noted that the time measurement means do not necessarily be integrated in a display device. For example, it may also be implemented as a stand-alone box connectable to a computer or some other device that needs a coordinate input or gesture input (derived from a series of coordinate information). According to an alternative embodiment, the light pen may also be adapted to receive the scanning start or synchronization signal from the time measurement means or the position detection means. In such case, the scanning is initiated by the time measurement means or position detection means. The time measurement means and position detection means may be implemented as a stand alone unit which comprises a first interface for connecting with a computer or consumer electronics device such as a TV set and a second interface over which the scanning start or synchronization signal is sent out to the light pen.
The invention relates also to a display screen unit comprising a light pen input system according to any of the before discussed embodiments. The light pen input system may also be suitable to be used with multiple light pens according to embodiments of the invention. The multiple light pens may be discriminated by one system using, for example, different color frequencies per light pen.
The display screen unit may comprise a communication interface adapted for communicating with a light pen according to any of the before discussed embodiments of the light pen according to the invention.
The display screen unit may further comprise processing means for controlling the position of a cursor displayed on the display screen of the display screen unit based on the coordinates of the pointing position of the light pen determined by the position detection means of the light pen input system.
According to an embodiment of the display screen unit, the first light sensing element may be located at the border of the display screen and the unit may comprise the time measurement and position detection means.
Particularly, the invention is suitable to be used with any kind of flat panel display screens such as a LCD or TFT, plasma, OLED, LCOS display or even a display generated by a video projector. However, the invention may also be used without a display or display screen.
The invention finally relates to a bar which is adapted for use with a light pen input system according to any of the above discussed embodiments, wherein the bar comprises photodiodes on each edge for detecting a scanning light line. Such a bar may be used for example with a video projector. The bar may be placed on an edge of the projection area and used to detect the pointing position of a light pen on the projected display. This is helpful when a video projector is used to project a computer display and the light pen is used as input device for controlling for example a mouse pointer of the computer display.
These and other aspects of the invention will be apparent from and elucidated with reference to the embodiment(s) described hereinafter.
The invention will be described in more detail hereinafter with reference to exemplary embodiments. However, the invention is not limited to these exemplary embodiments.
In the following, functional similar or identical elements may have the same reference numerals.
The light pen 12 generates two scanning light lines 14 and 15 which are arranged orthogonal. Thus, the first scanning light line 14 is provided for scanning the display screen 16 in a first horizontal direction 26, as shown in the left picture of
The photodiode 18 is provided as a light sensing element and located at the lower boundary of the display screen 16. The photodiode 18 generates a light detection signal if a scanning light line 14 or 15 is moved over it. The light detection signal and a scanning start signal generated by the light pen 12 are supplied to the time measurement means 20. The time measurement means 20 may be implemented by a counter which is started upon receipt of the scanning start signal (time tstart), and stopped upon receipt of the light detection signal (time t). The counting value corresponds to the time duration t−tstart and is transmitted to position detection means 22 which may be implemented by a processor programmed to determine the pointing position of the light pen 12 from the time duration t−tstart.
For determining the x-coordinate of the pointing position of the light pen 12, the first scanning light line 14 sweep is started. For synchronizing the whole system, a first scanning start signal is generated at time t1start, for example by the light pen 12 and transmitted to the time measurement means 20. The scanning start signal may also be generated by other components of the system, for example by the time measurement means 20 or the position detection means 22, or by a computer which generates and communicates a message like “you can start now” to the light pen 12. The scanning start signal triggers the counter of the time measurement means 20 to start counting. Then the scanning light line 14 is moved across the entire display screen 16 in the horizontal direction 26 at a constant rate, i.e., at a constant angular displacement rate or velocity. When the scanning light line 14 passes the photodiode 18, the photodiode 18 is illuminated for a certain time. Thus, the photodiode 18 generates a light detection signal and transmits it to the counter of the time measurement means 20. When the counter receives the light detection signal it stops counting at time t1. The counting value represents now the time duration Tx=t1−t1start from the start of the scanning light line 14 sweep and the light detection by the photodiode 18. The counting value is transmitted to the position detection means 22 which calculate the x-coordinate of the light pen's 12 pointing position from the counting value.
After finishing the first scanning light line sweep, the second scanning light line sweep in the vertical direction 27 is started. Again, for synchronizing the whole system 10, a scanning start signal is generated, for example by the light pen 12 and transmitted to the time measurement means 20 triggering the counter to start counting at time t2start. Then, the scanning light line 15 is moved over the entire display screen 16 in the vertical direction 27. When the scanning light line 15 illuminates the photodiode 18 by passing it, the photodiode 18 generates a light detection signal and transmits it to the counter triggering the counter to stop counting at time t2. The position detection means 22 receive the measured time duration Ty=t2−t2start from the time measurement means 20 and calculate from this time duration t2-t2start the y-coordinate of the light pen's 12 pointing position. It should be noted that the system described above may also be designed to be used by multiple light pens. Then, each light pen must be discriminated by the system, for example by using different color frequencies for each light pen. The photodiode or photodiodes used with a multiple light pen input system may then be sensitive to different wavelengths of received electromagnetic radiation, and generate different output signals depending on the wavelength of a received radiation in order to allow discriminating the light detection signals generated by a photodiode receiving electromagnetic radiation from a light pen.
In the following, it is explained how the x- and y-coordinates of the light pen's 12 pointing position may be calculated from the measured time durations Tx and Ty. If the angular velocity of the movement of the scanning light lines 14 and 15 is known and constant, and the distance of the light pen 12 from the display screen 16 is known, the time durations Tx and Ty, the distance between the light pen 12 and the display screen 16, and the angular velocity of the scanning light lines 14 and 15 may be used to calculate distances Δx and Δy. These are the distances between the starting point of each scanning light line sweep and the position of the photodiode 18. Thus, knowing the x- and y-coordinate of the location of the photodiode may be used to determine the starting point of the scanning light line sweeps. Another way to determine the x- and y-coordinates of the light pen's 12 pointing position is to calculate the fraction Tx/T and Ty/T with T is the time duration of an entire scanning light line sweep. The fractions can then be multiplied by the horizontal and vertical display size, respectively, in order to obtain the x- and y-position of a cursor on the display screen 16. A ratio, for example the ratio of the width and height of a display screen may be introduced by a scaling factor into the above formulas for determining the x- and y-coordinates. For example, such a scaling factor may be desirable to map large light pen movements onto smaller cursor movements on a display screen. This is one of the specific benefits of the present invention because it offers a user of the light pen input system to point in a larger area than the display screen area and, thus, makes the usage of the system easier.
For determining the distance between the light pen 12 and the display screen 16, a further photodiode (not shown) may be used. The output signal of this photodiode may depend on the light intensity of the laser light lines generated by the light pen 12. Thus, the distance may be determined by processing the output signal of this further photodiode. The determined distance between the light pen 12 and the display screen 16 may be processed as an additional input, for example for 3-dimensional applications controlled by the light pen 12.
The determined x- and y-coordinates of the pointing position of the light pen 12 may be transmitted as data 28 for further processing, for example by a computer or a display processor (not shown) which are programmed to display a GUI on the display screen. The computer or processor may use the x- and y-coordinates to control the display of a mouse pointer or a cursor of the GUI. It should be noted that the light pen input system 10 according to the invention may operate in an absolute mode and a relative mode. In the absolute mode, the measured time durations may be approximated to be directly proportional to the position assumed by a mouse pointer or cursor shown on the display screen 16. The start times t1start and t2start of the two scanning light line sweeps in horizontal and vertical direction may be mapped to the left and right location of the display screen 16. However, the system 10 may also be operated in a relative mode. In this mode, a first scanning light line sweep serves as a reference for following scanning light line sweeps. When time shifts are detected with the succeeding scanning light line sweeps, the pointing position of the light pen 12 has changed or—in other words—the mouse pointer or cursor was moved.
In the ideal case as shown in
When the light pen 12 is held untilted, as it is the case in the left picture of
It should be noted that a rotation of the light pen 12 may also be processed as further input data in addition to the pointing position. This feature may be of interest for 3-dimensional applications or 3d displays. For example, the rotation may be evaluated in a computer game as further input controlling the movement of a player. This functionality may be activated by a further rotation button of the light pen 12. When a user presses this rotation button and rotates the light pen 12, the resulting skew incurring the above explained time difference may be processed as a further input, for example by the position detection means. When releasing the rotation button again, the skew may be compensated by the position detection means as normal function of the light pen input system.
Now, it will be explained how a movement of a cursor controlled by the light pen 12 may be determined.
The cursor x-position for the first position of the light pen 12 may be calculated as follows: cursor x0-position=Tx0/T*horizontal display size. The cursor x-position for the second position of the light pen 12 may be calculated as follows: cursor x1-position=Tx1/T*horizontal display size. These formulas only deliver accurate results if the scan size is equal to the display screen size. The horizontal and vertical screen size may be determined using two photodiodes located in opposite corner of the display screen 16 as described above. From these measurements, the absolute x- and y-coordinates of the pointing position of the light pen 12 may then be deduced. The cursor may be moved in accordance with the calculated x-position. It should be noted that a scanning light line sweep may be so fast, i.e., the total sweep time (addition of time durations of both sweeps) may be preferably only about 5 to 10 milliseconds that it is unlikely that a user moves the light pen 12 within this short time period. Thus, inaccurate measurements according to user movements during a sweep may be neglected.
The invention has the main advantages that it may be implemented at low costs and is display-independent, i.e., may be used with modern flat panel displays such as LCD, TFT, or plasma displays as well as the older CRT displays.
The functionality of the invention, particularly the detection of the pointing position of the light pen may be performed by hard- or software. In case of an implementation in software, a single or multiple standard microprocessors or microcontrollers may be used to process a single or multiple algorithms implementing the invention.
It should be noted that the word “comprise” does not exclude other elements or steps, and that the word “a” or “an” does not exclude a plurality. Furthermore, any reference signs in the claims shall not be construed as limiting the scope of the invention.
| Number | Date | Country | Kind |
|---|---|---|---|
| 05111499.9 | Nov 2005 | EP | regional |
| Filing Document | Filing Date | Country | Kind | 371c Date |
|---|---|---|---|---|
| PCT/IB06/54359 | 11/21/2006 | WO | 00 | 5/27/2008 |
