Patent Application
20130009914
The present invention relates to an optical coordinate input device comprising an operational input detection plane in front of a display surface or projection surface, as well as to a program that controls the detection process thereof.
Interactive whiteboards (IWBs) have become increasingly popular in recent years. Interactive whiteboards comprise a combination of a display device or projection device and a coordinate input device. It is noted that for the display device that displays an operation screen, a plasma display device, a liquid crystal display device, or some other flat display device is used, for example. For the projection device that projects an operation screen onto a screen or a whiteboard, a front projector or a rear projector is used. With interactive whiteboards, it is possible to draw objects (text and images) with one's finger or an electronic pen as if drawing objects on a blackboard with a chalk.
For coordinate input devices of this type, tablets, touch panels, etc., have conventionally been used. Tablets and touch panels employ such technologies as electromagnetic induction, ultrasound, etc. On the other hand, in recent years, coordinate input devices employing image sensors have shown a steady increase. Coordinate input devices that employ image sensors are advantageous over conventional types in terms of drawing responsiveness, as well as their resistance to infrared light, sunlight, temperature change, and other types of external noise. Patent Literature 1 is a document relating to such a coordinate input device employing image sensors.
Patent Literature 1: JP Patent No. 3931030
Patent Literature 2: JP Patent No. 3986710
Patent Literature 3: JP Patent No. 4043178
Patent Literature 4: JP Patent Application Publication (Kokai) No. 2001-290583 A
With a coordinate input device that uses an image sensor, a light beam is emitted parallel to an operation screen from a light source, and the presence/absence of operational input is detected by detecting whether or not there is an object that blocks the light beam. However, while this detection method may be adequate for detecting simple operations such as pressing a button displayed on the screen, etc., it is inadequate for writing a text string, etc. By way of example, the strokes of a given drawing object may become merged, or trailing may occur with each stroke like a tail.
In this regard, Patent Literatures 1 and 4 attempt to solve these problems by placing the axis of a light beam from a light source to a retroreflective member and the axis of the reflected light beam from the retroreflective member to the image sensor near the surface of the operation screen.
In addition, Patent Literature 3 attempts to solve these problems by measuring the time elapsed from when an object blocking the axis of a light beam appears up to when actual contact with the surface of an electronic board is made.
However, there is a problem with conventional techniques in that natural text string input still cannot be performed.
With respect to coordinate input devices of the type where blockage of a light beam is detected using an image sensor, not only a dedicated electronic pen, but one's finger or a pointer may also be used for operation. The tips of electronic pens and pointers in this case, as well as one's fingers, needless to say, are round. This is to prevent electronic pens or pointers from causing damage to the surface of the display surface or projection surface.
By way of example, when an input object comes into contact with an operation screen, the tip of the input object first blocks detection light over a small area, and, following a subsequent and gradual increase in blocking area (size), the tip of the input object comes into contact with the operation screen. On the other hand, when an input object falls out of contact with the operation screen, the blocking area (size) changes in such a manner as to gradually become smaller.
As such, the present inventors propose a contact/no-contact detection technique focusing on this roundness of the tip portion. Specifically, there is proposed a coordinate input device that comprises an operational input detection plane at a position in front of a display surface or projection surface, and that optically detects coordinates at which an input object with a round tip traverses the coordinate detection plane, the coordinate input device comprising: (a) a processing function that detects the size of the input object blocking detection light which travels along the coordinate detection plane; (b) a processing function that generates a down event of the input object when, after initially detecting blockage of the detection light, it is detected that the size has become greater than a first threshold; and (c) a processing function that generates an up event of the input object when, after detecting a maximum value for the size, it is detected that the size has become smaller than a second threshold.
With the present invention, a change in the size of an input object blocking detection light may be regarded as a precursor to a change in the contact/no-contact state of the input object. Thus, the contact/no-contact state may be detected at more natural timings as compared to when merely the presence/absence of a blocking object is determined It is thus possible to better prevent strokes from becoming merged when writing a text string, or trailing from occurring with each stroke like a tail.
Examples of embodiments of the invention are described below based on the drawings. It is noted that all of the embodiments that follow are examples, and that the present invention encompasses systems that are realized by combining any of the functions described in the present specification, systems that are realized by replacing some of the functions described in the present specification with well-known techniques, and systems in which well-known techniques are incorporated in addition to the functions described in the present specification. In addition, the functions performed in the later-described examples are realized as programs executed on a computer. However, the programs may also be realized via hardware in part or in whole.
In the case of this example, the coordinate input device has a structure where the light sources 102A, the image sensors 102B and the retroreflective member 108 are attached to a rectangular frame (device main body). It is noted that a finger 103 is used for operational input, although an electronic pen or a stylus pen (pointer) may also be used for operational input.
The coordinate input device is disposed at a position in front of a screen or whiteboard onto which an operation screen is projected. In other words, a detection plane for operational input is formed at a position in front of the screen or whiteboard. Although the operation screen is projected in this example, other conceivable configurations include ones where it is disposed at a position in front of a display device such as a flat display, etc., and ones where such display devices are integrated with the frame. In addition, the input area intended for coordinate input of an input object need not be a large area as in screens and whiteboards, and may instead be a small area as in mobile phones, electronic books, and other portable terminals.
The connective relationship among electronic circuits forming the coordinate input device is shown in
The operation screen projection device 104 is used to project onto the screen or whiteboard the operation screen, as well as text and objects that have been inputted with an input object. It is assumed that the screen projected by the operation screen projection device 104 and the screen displayed on the display device 107 are the same.
The control computer 105 has functions comparable to a general-purpose personal computer, and on its internal memory is stored a coordinate calculation program 1051 that calculates the coordinates pointed at by the finger 103 based on the images captured by the image sensors 102B and on the principles of triangulation. In addition to the above, there are also stored on the internal memory a display content control program that processes text objects and image objects, and an event generation program that detects operational input by an input object and generates an event corresponding to the detected state.
Although the coordinate calculation program 1051 is run on the control computer 105 in this example, it may instead by executed by the CPU 704 within the coordinate input device, or it may also be executed within the operation screen projection device 104. The implementation of this function may be in the form of hardware (e.g., semiconductor integrated circuits, processing boards), or in the form of programs (e.g., firmware, applications).
In the case of this example, the coordinate input device used is of a type that uses light (e.g., infrared light) that is emitted parallel to the surface onto which the operation screen is projected, and that detects the position at which an input object (e.g., the finger 103) blocks the light through the principles of triangulation. By way of example, the two light sources 102A (e.g., infrared light sources) and the image sensors (imaging devices) 102B are disposed at both ends of the upper side of the rectangular frame or near the center of the upper side. By way of example, if the two light sources 102A are disposed at both ends on the left and right of the upper side, each of the light sources 102A emits a light beam towards, or scans therewith, the entire length of the side opposite where it is located as well as the entire length of the lower side. In this case, the view angle of the image sensors 102B is approximately 90°. It is noted that, if the two light sources 102A are disposed near the center of the upper side, the emission angle of each of the light sources 102A and the view angle of the image sensors 102B are both set to approximately 180°.
The retroreflective member 108 is disposed on the inner sides (the surfaces facing the light beam) of the frame at the three sides other than the upper side. Thus, the light that is incident on the retroreflective member 108 is reflected in the same direction as the incident direction. This reflected light is imaged with the image sensors 102B disposed near the light sources. When an input object blocks the light beam, a shadow is created in the images captured by the image sensors 102B. Based on the positional information of the shadow imaged by the left and right pair of image sensors 102B, the coordinate position of the input object is calculated according to the principles of triangulation.
The calculating of coordinates itself is carried out by the coordinate calculation program 1051 of the control computer 105. Accordingly, in the case of this example, imaging data is outputted to the control computer 105 from the image sensors 102B. It is noted that, when this type of coordinate input device is used, the control computer 105 is able to simultaneously calculate the coordinates of a plurality of input objects.
A cross-sectional structure of a coordinate input device that detects an operation position of an input object is shown in
The relationship between the distance from the tip of a finger to the surface of the operation screen and light blockage is illustrated in
The diagram on the left in
The graph shown in the lower part of
A processing operation for reliably detecting down operations and up operations is described below. A flowchart for determining contact/no-contact by a finger, etc., with respect to an operation screen is shown in
First, in an imaging result analysis process that is repeatedly executed at short intervals, the processor determines whether or not there is a blocking object (shadow) in the coordinate detection plane (step 601).
If there is no blocking object (if step 601 returns an affirmative result), the processor sets an initial value for maxSize, which holds the maximum value of the thickness of the shadow (step 602). This initial value is used as a flag indicating that no blocking object is present.
If the presence of a blocking object is detected (if step 601 returns a negative result), the processor determines whether or not the value of maxSize is the initial value and whether or not the current shadow thickness is equal to or less than a threshold for generating a down event (step 603). The second determination condition is used for the purpose of preventing the process from proceeding to step 604 when a shadow that is too large is detected.
If the two conditions in step 603 are simultaneously satisfied (if step 603 returns an affirmative result), the processor stores the current shadow thickness in both maxSize and minSize (step 604). This process corresponds to registering the initial detection value for the thickness of the shadow that has actually been detected.
On the other hand, if either of the two conditions in step 603 is not satisfied (if step 603 returns a negative result), the processor further executes the following determination process (step 605). Specifically, the processor determines whether or not the current shadow thickness is equal to or greater than twice the value of maxSize, or whether or not the current shadow thickness is equal to or greater than the threshold for generating a down event (step 605).
If either of the two conditions in step 605 is satisfied (if step 605 returns an affirmative result), the processor generates a down event (step 606), sets minSize to the maximum value (step 607), and executes a shadow thickness updating process (step 608).
This maximum value is used as a flag for making a determination regarding the occurrence of a down event. It is noted that in the shadow thickness updating process, the processor determines whether or not the current shadow size is thicker than maxSize (step 6081). If an affirmative result is obtained, the processor substitutes the current shadow thickness into maxSize (step 6082), whereas if a negative result is obtained, maxSize is left unchanged. In other words, maxSize is updated only when the previous detection value is exceeded.
If a negative result is obtained in step 605, the processor determines whether or not the current shadow thickness is equal to or less than half the value of maxSize (step 609).
If an affirmative result is obtained in step 609, the processor generates an up event (step 610), and maxSize is set to an initial value (step 611). It is noted that in this case, both a case where the shadow has become smaller and a case where the shadow itself has disappeared are included. The initial value in this case signifies that an up event has occurred.
If a negative result is obtained in step 609, the processor determines both whether minSize is a maximum value and whether maxSize is not an initial value (step 612). In other words, the processor determines whether or not a down event has occurred while an up event has not yet occurred.
If an affirmative result is obtained in step 612, the processor generates a move event (step 613), and updates shadow thickness to the current value (step 614). Through this process, the change in thickness that occurred between a down event and an up event is recorded.
If a negative result is obtained in step 612, the processor terminates one cycle of the process.
In the case of the example above, it is determined in step 605 whether or not the current shadow thickness is equal to or greater than twice the value of minSize. However, to allow for cases where an object with a square tip is used or cases where a quick pressing action is performed with a finger, and so forth, it is preferable that the processes of step 606 and onward be performed if, in addition to the conditions in step 605, the shadow is equal to or greater than a given size.
In addition, although an up event is generated in step 610, it is preferable that it be determined whether or not a down event has occurred in step 606, and that an up event not be generated unless it has been confirmed that a down event has occurred.
Further, although maxSize is set to an initial value in step 611, just a determination result as to whether or not an up event has been generated may be stored in this step instead, and the processes of step 606 and onward may be executed if, in addition to the conditions in step 605, an up event has already occurred, and the current shadow thickness is equal to or greater than half the value of maxSize. By providing such a process, it is possible to make a down process be performed when the shadow becomes thick again after having become thin once. Thus, it is possible to draw a text string without having to consciously move the finger away from the surface of the operation screen.
Further, with systems in which image sensors are employed, the thickness of the shadow varies depending on where on the surface of the operation screen a blocking object is placed. By way of example, at a position near the image sensors, the thickness of the shadow of the blocking object increases, whereas at a position further away, the thickness of the shadow of the blocking object decreases. Thus, when an obstacle is moved from a position near the image sensors towards a position further away, the thickness of the shadow will appear to decrease. In this case, if an up event were to be generated based solely on changes in the thickness of the shadow, it would give rise to results the operator may not have anticipated.
In order to avoid the above, it is preferable that the contact/no-contact determination process described in the example not be used if the input object (blocking object) has moved a given distance or more. A strict contact/no-contact determination is only necessary when writing a text string or when delicate operations are called for, and since moving a blocking object by a given distance or more may be deemed not to be the drawing of text, or any delicate operation, it does not become an issue. As for what kind of distance should be defined for the given distance, one may define how big the size of each stroke of a text string is to be, and apply that value.
In addition, in the above-discussed example, descriptions have been provided with respect to a case where the down event determination threshold was defined as twice the initial size, and the up event determination threshold as one half of the maximum value. However, it is preferable that each determination threshold be individually adjustable. By having each determination threshold be adjustable, it is possible to adjust the “feel of writing.”
In addition, by setting determination threshold A for down events to a low value and determination threshold B for up events to a high value (>A), contact states become more readily identifiable, while making determinations of a no-contact state less likely. Consequently, it creates the impression of being able to write text strings with a lighter touch.
In addition, by setting determination threshold A for down events to a high value and determination threshold B for up events to a low value (<A), it creates the impression of being able to write text strings with a strong touch.
| Number | Date | Country | Kind |
|---|---|---|---|
| 2010-068770 | Mar 2010 | JP | national |
| Filing Document | Filing Date | Country | Kind | 371c Date |
|---|---|---|---|---|
| PCT/JP2011/053758 | 2/22/2011 | WO | 00 | 9/14/2012 |
