Patent Application
20130241837
The present invention relates to an input apparatus of an information processing apparatus and its control method, especially relates to an input apparatus using touch sensor and its control method.
In an information processing apparatus, a touch sensor is used as an input apparatus, which detects the contact of an object. Some touch sensors can detect the proximity. In addition, a touch panel composed of a touch sensor and a display device is widely used by the reasons that an input operation can be done with visually checking the input position that it can be flexibly changed to set the input position, input method (zero dimension information input such as button which does not include distance information, one-dimension information input such as a slider and two-dimension information input such as handwriting of a character and a figure)because the control can be set by the program of the computer, and that an input operation is easy with indication the information related to the input position and the input method.
On the other hand, the operability of the conventional keyboard which has been used so far has been refined by the long time improvement and there are many users mastering by the operation method. Accordingly, by setting a virtual key board on a touch panel with the combination of those strong points, the operability in accordance with the conventional keyboard is realized.
For example, in the patent document 1 (Japanese Unexamined Patent 2008-508601), it is disclosed, as shown in
However, the virtual keyboard disclosed by this patent document 1 has several problems.
It is the most serious problem (the first problem) is that the virtual keyboard cannot be operated without visual checking of the indication on the touch panel since the sense of touch cannot be used for grasping the key position.
The virtual keyboard in the patent document 1 has a configuration to display a virtual keyboard with a predetermined size at a predetermined position by the trigger of finger touching to the touch sensor.
An operator, who watches this display, recognizes the position of the key to be operated first. However, in a small terminal or the like, there may be the condition that the display cannot be visually checked. For example, it is the case that a key is operated keeping the terminal in a pocket. Then, a keyboard, in which each conventional key has been mechanically implemented (conventional keyboard), has enabled to grasp the position of the key to be operated first, through the recognition of the keyboard shape by the sense of touch. However, it is difficult to grasp the key position by the sense of touch in a case of a touch panel.
Even in the case that a plurality of key operations following to the first key operation, the visual check for the positions of the plurality of keys to be operated is needed in a touch panel. The scheme that is realized in a conventional keyboard and can be used, such as checking the position of the key to be operated subsequently by using the sense of touch similar to the first key operation, and making it easy for an operator to grasp the relative position relation between the keyboard and the operator hand by setting a home position key whose sense of touch is made different from other key, cannot be used.
In case of a conventional keyboard, by changing the key shape such as making the key top concave, it has been enabled to detect the depressing point of the key, for example whether the center of the key has been pressed, using the sense of touch when the operator has depressed, and to correct the depressing point the next time and the after time if it has been judged to have the gap from the center. However, this correction is impossible in a virtual keyboard.
Next, the second problem is that a virtual keyboard cannot be installed in the position that cannot be visually recognized by an operator of a terminal.
A virtual keyboard premises that it is visually recognized by the user as already stated. On the other hand, when a display device is allocated on a surface facing an operator of a terminal case, there is a case that the keyboard is wished to be allocated on the back surface of a terminal case in consideration of the visibility of the display device. When a virtual keyboard using a touch sensor is used in this case, the method is needed for recognizing the key position and the depressing point of the key on a virtual keyboard without depending on the sense of sight of the operator.
The third problem is the point that, in the case of a small size keyboard such as a numerical keypad, it can not be displayed at the optional position based on an operator will since the position is fixed in spite that there exists a wide area on a touch panel where the keyboard can be displayed. In the case of the patent document 1, a keyboard is displayed at the predetermined position by finger touching to a touch sensor, and it cannot be displayed at the optional position, which the operator desires, on the touch panel.
In order to settle such problem, the present invention is proposed, and the aim is to offer an input apparatus with the excellent operability of the keyboard constituted on the touch sensor.
An input apparatus of an information processing apparatus, which has a touch sensor, of the present invention has a recognition means which detects that a detection object, which the touch sensor has detected, has touched and recognizes the position information of the detection object and a control means which sets a virtual keyboard to the position based on the position information.
An input apparatus as the first embodiment of the present invention will be described using
As shown in a block diagram of
The detection (sampling) of the touch signal in the touch sensor 22 is repeatedly executed in a predetermined time interval, and the touch signal is obtained as a discrete time series value concerning to the discrete sampling time as a result. By narrowing the time interval of the touch signal detection, the time corresponding to the touch signal, which shows that a touch has been detected, can be approximately handled as the time information (time information) of the touch occurrence. The touch signal will include this time information.
The touch sensor 22 outputs a signal that indicates the non-detection of the touch in the state that the touch is not detected (touch non-detection state). When it moves from the touch non-detection state to the state that the touch is detected (touch detection state), the first touch signal is made the touch start signal and the position information and the time information that are included there are made the touch start position information and the touch start time information respectively. When it moves from the touch detection state to the touch non-detection state, the last touch signal is made the touch end signal and the position information and the time information are made the touch end position information and the touch end time information respectively.
The display device 23 inputs a display signal outputted from the processor 24.
A memory 25 has a control program 26, a configuration parameter 27, image data 28 and an application program 29 as the stored information, inputs the stored information from the processor 24 and outputs the stored information to a processor 24.
A storage apparatus 30 has the memory information, inputs the memory information from the processor 24 and outputs the memory information to the processor 24. A memory card and a hard disk are used in order to store the large volume information which the memory 25 cannot store generally. There may be a case of interconnection via a network (not shown).
A timer 31 inputs a setup signal and a trigger signal from the processor 24 and outputs a time count signal and an interrupt signal to the processor 24.
A position sensor 32 outputs a position signal to the processor 24.
The processor 24 inputs a touch signal from the touch sensor 22, outputs a display signal to the display device 23, inputs the stored information from the memory 25 and outputs the stored information to the memory 25. The memory information is inputted from the storage apparatus 30, and the memory information is outputted to the storage apparatus 30. The time counting signal and the interrupt signal are inputted from a timer 31, the setup signal and the trigger signal are outputted to the timer 31, and the position signal is inputted from the position sensor 32.
Next, the operation of the input apparatus will be described using
In the first embodiment, in the condition that a virtual key arrangement cannot be visually checked on the touch sensor 22, the key input operation becomes possible by the similar operating to the conventional blind touch from the second touch.
As shown in
As a virtual keyboard, a numerical keypad example will be described in the first embodiment. However, the number, the shape and the arrangement of the keys are not limited to this in the present invention, and when it is a keyboard having a plurality of keys, it can be applied to everything including a keyboard in which the keys has been grouped and arranged for the right hand operation and for left hand operation as well as a keyboard with the usual QWERTY arrangement. A new key arrangement may be defined, of course.
The operation which arranges a virtual keyboard in the optional position on the touch sensor 22 and inputs by operator finger, will be described in detail with reference to
First, the input mode (the virtual keyboard input mode) using the virtual keyboard is set. (Step 1 of
As a case that the virtual keyboard input mode is set, there exist the case of setting by the initialization just after the power on and the case of switching by a mode switching request signal from other operation modes including the input mode. Depending on the operation mode, the case is also assumed that the application works which does not relate to the input.
In the case that it is set by the initialization, the processor 23 executes the control program 26, which performs a virtual keyboard input processing, after the initialization. In the case that it is switched from the other operation mode, the processor 24 executes the control program 26 when detecting a mode switching request signal.
The mode switching request signal can be generated based on the signals that other sensors and input devices output as well as the case that it is generated when the virtual keyboard input mode is selected by the input from the touch sensor 22.
For example, positioning of the operator hand for the touch panel 21 and the touch sensor 22 as well as the mode switching operation is easy when installing the switch 33 (it may be small touch sensor or a key) that can be recognized the position by the sense of touch in a frame portion surrounding the touch panel 21 and the touch sensor 22, a side face and a back face of the terminal (
It is possible to have an acceleration sensor 34 (a vibration sensor and a gravity sensor are also included) and generate a mode switching request signal using the output.
For example, if an operator prefers to arrange the touch panel 21 horizontally and operate it while looking down usually, and if, according to the detection result of the gravity direction, it is judged that the touch panel 21 has the large angle (for example not less than 60 degrees) to the horizontal plane, a high possibility is presumed that the operator operates the terminal without watching the terminal while keeping to put the terminal in a bag or a pocket. And the input mode may be changed based on the possibility. Similarly, it is presumed that the operator operates the virtual keyboard without watching in the case that the touch panel 21 and the touch sensor 22 direct downwards (the touch panel 21 and touch sensor 22 are arranged on the back face of the terminal to the operator who looks down from the top) according to the detection result of the gravity. And it can be set to switch the input mode based on the presumption. Further, the vibration is detected when the operator walks with carrying the terminal in hand or drives a car, and it is presumed that the operator operates it under the condition that he cannot turn his eye to the virtual keyboard or that the visual recognition of the virtual keyboard is difficult for the operator because of the vibration. And it can be set to switch the input mode based on the presumption.
It is possible to generate the mode switching request signal by using the output from an illuminance sensor 35 equipped (a light receiving element of a camera is also included).
When the surrounding is dark such as in a movie theater, it is presumed that the virtual keyboard operation of groping is requested under the condition where the terminal illumination is not available. And it can be set to switch the input mode based on the presumption.
The processor 24 has the function to mask the input of the mode switching request signal and enables the configuration in order to reject the switching of the operation mode in the case that the virtual key board input mode is unnecessary or is prohibited.
When the touch sensor 22 enters in the virtual keyboard input mode, the processor 24 executes the control program 26 and performs the following processing. At this time, the setting information (included in a configuration parameter 27) is transmitted from the memory 25 to the processor 24 and referred. Hereinafter, although the case of the contact is described as a touch, the case of the proximity is the same.
The virtual keyboard initial position decision processing is performed at first. (Step 2 of
The processor 24 is waiting for the first touch signal from the touch sensor 22 in the initial state of the virtual keyboard input mode. When the operator finger as a detection target touches the touch sensor 22, the touch sensor 22 detects the contact (touch) of the finger and outputs the first touch signal to the processor 24. The time information on the contact (touch) (the first time information) and the position information on the contact (touch) (the first position information) are included in the first touch signal. While the finger keeps contacting (touching) the touch sensor 22, the touch sensor 22 continues to capture the contact (touch) of the finger and keeps outputting as the first touch signal.
When the touch sensor 22 detects that the operator finger is away from the touch sensor 22, the processor 24 acquires the touch end position information concerning to the first touch signal as the first position information. Further, the first position information is not limited to the touch end position information. For example, although touch starting position information can be used, a case when touch end position information is used will be described here.
A processor 24 carries out virtual keyboard setting processing based on the first position information. (Step 3 of
The processor 24 correlates the input information of a virtual keyboard stored in the configuration parameter 27 to each area of the matrix.
As an example of the input information of a virtual keyboard, a character code of A,B, . . . and 1,2, . . . and a control code of a Return key and an Escape key are pick up. These are stored in the configuration parameter 27 with a table format, for example. The information whose order is fixed in advance, such as character codes, may be correlated by calculation. In this case, the information concerning to an operational expression of calculation (the coefficient and the invariable, for example) is stored in the configuration parameter 27.
Next, the virtual keyboard operation processing is carried out. (Step 4 of
The processor 24 waits for the second touch signal from the touch sensor 22. When the operator finger touches the touch sensor 22 again as the second contact (touch), and the touch sensor 22 outputs the second touch signal to the processor 24, the processor 24 performs the calculation to compare the position information included in the second touch signal with the threshold value information, and detects in which area of said matrix the second contact (touch) has occurred.
And the input information of a virtual keyboard corresponding to the detected area is delivered to the processing of an application program 29 executed as documentary creation, a spread sheet and a game.
The processor 24 repeats the above-mentioned processing, from the state waiting for the second touch signal to the delivery of the input information, during the processing of the virtual keyboard operation. Further, in the state that the second touch signal or after one is waited for, if the upper limit time which is predetermined has passed without the next touch signal input, the virtual keyboard operation processing is terminated. The information on the upper limit time stored in the configuration parameter 27 is set to a timer 31 by the processor 24. The processor 24 outputs a trigger signal for starting the time count to the timer 31 when it enters the waiting state for the touch signal, and the timer 31 begins to count the time. When counting the time beyond the upper limit time, the timer 31 outputs an interrupt signal to the processor 24, and the processor 24 terminates the virtual keyboard operation processing. When the touch signal is detected before the interrupt signal is received from the timer 31, the processor 24 sends the setup signal to stop the time count to the timer 31, and the timer 31 stops the time counting. And when the input information delivery is completed, it returns in the waiting state for the touch signal again.
According to the description mentioned above, the virtual keyboard to be touched at the second touch and after is constructed (virtual keyboard initial position determination processing, virtual keyboard configuration processing) based on the first touch signal which includes the first position information indicating the position of the operator hand (here, it means that a plurality of fingers whose relative position relation is predetermined are included as a group). Accordingly, even if a virtual key arrangement cannot be indicated on the touch sensor 22 or the indication cannot be watched, a key input operation is possible by the operation similar to the conventional blind touch on the second touch and after.
As an application, an operational conceptual diagram when a virtual keyboard is built on the terminal back face is shown in
On the other hand, there is a case that it is wanted to change the position of the virtual keyboard on the touch sensor 22 during the virtual keyboard operation. Although it is able to switch into a new mode where the position of the virtual keyboard can be changed or to change by depressing a key for the moving, the operation time for it is needed and there is a defect in the operability. The operator hand (a plurality of fingers whose relative position relation is predetermined are included as a group) may move from the position set initially in the first touch during the virtual keyboard operation.
Accordingly, the first embodiment has the composition that the virtual keyboard position can be changed, according to the movement of the operator hand (a group of fingers) under the operation, with no special operation.
That is, the first virtual keyboard movement processing indicated below is performed. (
While a finger contacts (touches) the touch sensor 22, the position information of contact (touch) is repeatedly detected in a predetermined time interval and is input to the processor 24 as a discrete time series value. The processor 24 monitors the position information and performs the processing to move the virtual keyboard by the direction and the distance of the movement vector, which is obtained according to the arithmetic operation for the movement vector whose start point is set to be the position information at a certain time and whose end point is set to be the position information at the next detection time, every time interval mentioned above. (The first virtual keyboard movement processing)
In the processing mentioned above, the processor 24 can also performs the processing on the arithmetic operation of the movement vector and after based on the position information which has been thinned out in the predetermined sample interval. In other words, the arithmetic operation frequency of the movement vector can be reduced for the detection frequency of the touch. As a result, the amount of the arithmetic operation can be reduced.
As other methods, the second virtual keyboard movement processing is indicated. (
The processor 24 monitors the position information. And if the processor 24 judges that the duration of the touch (touch time) is more than or equal to the predetermined upper limit value and the operator finger has moved keeping the contact (touch) on the touch sensor 22, the processor 24 performs the processing to obtain the movement vector whose start point is set to be the touch start position information and whose end point is set to be the touch end position information and to move the virtual keyboard by the direction and the distance of the movement vector.
As the processing to measure the touch time, the processor 24 outputs a trigger signal which directs the start of time counting to the timer 31 when the touch start time information is inputted. If the trigger signal from the processor 24 is inputted, the timer 31 counts the time which is given in advance as the configuration information from the processor 24 and outputs an interrupt signal to the processor 24 when the time has passed. When then interrupt signal is received, the processor 24 sets the internal interruption flag (not shown). (In initialization, an interruption flag is cleared.) After that, when the touch ending time information is inputted, a movement vector, whose start point is set to be the touch start position information and whose end point is set to be the touch end position information, is obtained by calculation. When the size of the movement vector is smaller than a lower limit value set in advance, the processor 24 judges that the same key continues to be depressed. On the other hand, when the size of the movement vector is larger than or equal to said lower limit value, it is judged to have moved with keeping to contact (touch) and the processing is performed to move the virtual keyboard by the direction and the size (distance) of the movement vector. When the touch ending time information is inputted, the processor 24 clears the interruption flag. Further, if the interruption flag is not set when the touch ending time information is inputted, the processor 24 does not perform the processing of the calculation for the movement vector and after.
According to the first embodiment, a virtual keyboard to be touched from the second touch is constructed based on the position information on the first touch signal (the first position information), as shown above. Accordingly, even in the condition that the virtual key arrangement cannot be watched, a key input operation becomes possible by the similar operation to the conventional blind touch from the second touch.
The keyboard position can be changed without performing the special operation.
Moreover, because the virtual keyboard follows the movement even when the operator hand (a plurality of fingers are included as a group) has moved from the position set initially by the first touch during the virtual keyboard operation, the misoperation occurrence probability of the next touch input can be reduced and the operatability can be improved.
Further, when a touch panel 21 is used with the replacement of the touch sensor 22, the processor 24 can indicate an image, by which an operator can visually distinguish each area of the matrix mentioned above, to a display device 23. In
Next, the formation and the operation of the second embodiment of the present invention will be described using
This embodiment has the characteristics that the virtual keyboard performs the input operation by the second finger with continuously keeping to detect the touch position of the first finger, which is the reference, in order to recognizes the position of the operator hand continuously.
As the formation of the embodiment, an input apparatus is described according to
In this input apparatus, the virtual keyboard input mode is set by the same procedure as the first embodiment. (Step 1 of
First, the virtual keyboard initial position decision processing is carried out. (Step 2 of
In
Next, when the first touch signal (by a thumb in
Further, the following operation described below is terminated when the operator finger, which is an origin of the first touch signal, ends the contact (touch).
The processor 24 divides a contact detecting face of the touch sensor 22 into a plurality of areas based on the position information on the first touch signal (the first position information). For example, when the position is indicated by the XY two-dimensional coordinate system, the threshold values are set up to each of the x-direction and the y-direction, and it is divided into matrix shapes. The basic threshold value is stored in a configuration parameter 27. The information of relative arrangement and size in the matrix is included in the basic threshold value. The processor 24 generates the threshold value information by adding the position information of the first touch signal (the first position information) as an offset to the basic threshold value. The processor 24 correlates the input information of a virtual keyboard stored in the configuration parameter 27 to each area of the matrix. In
Next, the virtual keyboard operation processing is carried out. (Step 4 of
The processor 24 waits for the second touch signal (by an index finger in
While the virtual keyboard operation processing is carried out, the processor 24 repeats the above-mentioned processing, from the state waiting for the second touch signal to the delivery of the input information.
The virtual keyboard operation processing is terminated when the operator finger, which is an origin of the first touch signal, ends the contact (touch). This operation does not depend on the presence of the touch signal on the second touch signal and after.
Further, even in the case of the continuation of the first touch signal, similarly to the first embodiment, the virtual keyboard operation processing is terminated when there is no input of the next touch signal in spite of the condition that the touch of the second touch and after is waited for and the predetermined time has passed.
In order to resume the virtual keyboard initial position decision processing, all of the contacts (touches) are released once.
On the other hand, when the first contact (touch) point has moved with maintaining the contact (touch), the position of the virtual keyboard is updated according to it.
That is, the virtual keyboard movement processing indicated below is performed. (
During the contact (touch) of a thumb to the touch sensor 22 (touch), the first position information is repeatedly detected in a predetermined time interval and is input to the processor 24 as a discrete time series value. The processor 24 monitors the first position information, obtains the movement vector, whose start point is set to be the first position information at a certain time and whose end point is set to be the first position information at the next detection time, every said time interval by an arithmetic operation, and performs the processing to move the virtual keyboard by the direction and the distance of the movement vector.
The processor 24 can perform the processing on the arithmetic operation of the movement vector and after based on the first position information that has been thinned out in the predetermined sample interval in the processing mentioned above. In other words, the arithmetic operation frequency of the movement vector can be reduced for the touch detection frequency. As a result, the amount of operations can be reduced.
After the virtual keyboard operation processing or the virtual keyboard movement processing ends, and all of the contacts (touches) are released, the virtual keyboard initial position decision processing restarts when the touch sensor 22 detects that a thumb has contacted (touched) again.
In the description of the second embodiment mentioned above, a case that a thumb is used for the first touch which is a position reference has been indicated. But the other fingers may be used, of course. In the case, the basic threshold value according to the used finger is used.
According to the second embodiment, as shown above, the position of the operator hand is continuously recognized based on the position information (the first position information) on the first touch by the first finger that is the reference. And the virtual keyboard is constructed by applying that the relative position relation is limited between the first finger and a different finger, which shares the same hand with the first finger and is used for the second touch and after. As a result, the input operation can be easily performed even in the condition that the virtual key arrangement cannot be watched.
An operational conceptual diagram in the case that the virtual keyboard is built on the back face of the terminal is shown in
In addition, the effect is obtained that the misoperation occurrence probability of the touch input can be reduced and the input operation is improved more since the virtual keyboard follows concerning to the movement of the operator hand that has taken place between touch and touch especially in the case that the hand or finger of the operator has moved on the first contact (touch) and after.
Although the detection of the finger which is the reference is performed by the touch sensor 22 that composes the virtual keyboard in the embodiment mentioned above, it may be detected with the implementation of the second position sensor 32 as shown in
In the embodiment described above, the operation for setting a virtual keyboard with the size suitable for each operator (initial calibration) will be described using
The virtual keyboard with the most suitable key arrangement and size for a user, can be selected and constructed, by inputting the information on the hand size (the positional relationship between the finger and the finger) for each user and registering (storing) the information in the memory 25 and a storage apparatus 30 in the initial calibration.
First, on touching the touch sensor 22 by opening a hand and extending a finger as shown in
the positions (setting as a point O, a point A, a point B, a point C and a point D concerning to the thumb, the index finger, the middle finger, the ring finger and the little finger respectively), that the tip of the finger touches, are read-out and stored in the memory 25. (Step 1 of
The distances between each of points are obtained from the coordinate of the acquired point O, point A, point B, point C and point D. If it is supposed that the hands of various operators are in a homothetic relation, these correspond to the size of the operator hand. Accordingly, for example, the mean value of each point-to-point distance, and the weighted mean value, that has been weighted depending on the finger to which each point-to-point distance is related, are stored in the parameter 27 as the size information of the hand. The shape data of a standard virtual keyboard such as a numerical keypad and a QWERTY keyboard is registered in the configuration parameter 27. And a basic threshold value of the virtual keyboard which fits to the size of the operator hand, can be calculated by performing the arithmetic operation such as multiplying the shape data by the size information on the hand. (Step 2 of
Next, a further device is indicated concerning to the shape of the virtual keyboard.
The point O, the point A, the point B, the point C and the point D shown in
An example of these points on the touch sensor 22 is shown in
The information on the positions and the boundaries of these virtual keys is obtained by the arithmetic operation based on the coordinates of the point A, the point B, the point C, the point D, the point A′, the point B′, the point C′ and the point D′ which are stored in memory 25, (Step 2 of
The mentioned above has been described based on the second embodiment, in which the position O of the thumb is a reference, but can be also utilized the first embodiment.
For example, it has been predetermined that the first touch is carried out by “the extended middle finger”. And when the first touch signal is detected, it is supposed that the position information corresponds to the point B and the virtual keyboard is set. (In this case, the position information when bending and extending is also acquired in advance concerning to a thumb.)
In addition, as a modification of the first embodiment, if it has been predetermined that the first touch is performed with “an extended middle finger”, the second touch is done with “an extended thumb” and the virtual keyboard input is carried out after the third touch, a gap in the rotation direction,
whose center is made by the first touch (middle finger), can be corrected.
The present invention relates to an input apparatus using a touch sensor and its control method.
A Description of Codes
21 Touch panel.
22 Touch sensor.
23 Display device.
24 Processor.
25 Memory.
26 Control program.
27 Configuration parameter.
28 Image data.
29 Application program.
30 Memory storage.
31 Timer.
32 Position sensor.
33 Switch.
34 Acceleration sensor.
35 Illuminance sensor.
| Number | Date | Country | Kind |
|---|---|---|---|
| 2010-261060 | Nov 2010 | JP | national |
| Filing Document | Filing Date | Country | Kind | 371c Date |
|---|---|---|---|---|
| PCT/JP2011/077440 | 11/22/2011 | WO | 00 | 5/20/2013 |
