Patent Application
20120113028
The invention relates to a smooth, solid touch- and vibration-sensitive surface that is easy to clean and that allows the user to rest their hands or fingers on the surface without causing an event actuation. More specifically, the surface may be used as a computer keyboard for inputting text and commands.
The origin of the modern keyboard as the primary method for inputting text and data from a human to a machine dates back to early typewriters in the 19th century. As computers were developed, it was a natural evolution to adapt the typewriter keyboard to be used as the primary method for inputting text and data. While the implementation of the keys on a typewriter and subsequently computer keyboards has evolved from mechanical to electrical and finally to electronic, the size, placement, and mechanical nature of the keys themselves have remained largely unchanged.
Computers, and their accompanying keyboards, have become pervasive in environments across numerous industries, many of which have harsh conditions not originally accounted for in the computer and keyboard designs. For example, computers are now used in the kitchens of restaurants, on production floors of manufacturing facilities, and on oil drilling rigs. These are environments where a traditional keyboard will not remain operational for very long without cleaning, due to extreme contamination conditions.
To overcome the problem of cleanability of the keyboard, it seems intuitive that if the keyboard surface itself could be a flat, or nearly flat, surface, then wiping the keyboard to clean it would be much easier. This means, however, that an alternative to the physical mechanical or membrane keys of the keyboard would need to be found.
In partial response, new computer form factors have evolved to eliminate external keyboards entirely, consisting solely of a touch-sensitive flat display screen with a software-based “virtual” keyboard for data entry. Touch screen virtual keyboards are difficult to use at high speed for typists who are trained to rest their hands on the keyboard, as the act of resting results in unwanted key activations from the keyboard.
Therefore, there is a need to improve on the above methods for keyboard entry in a way which is easy to clean, allows the user to feel the keys, allows the user to rest their fingers on the keys, requires the same or less force to press a key as on a standard keyboard, is responsive to human touch, and allows the user to type as fast as or faster than on a conventional mechanical keyboard.
The present invention provides systems and methods for enabling use of vibration sensors attached to the touch-sensitive surface to both detect and locate finger contact events on the surface. The invention specifically discriminates between intentional typing events and casual or unwanted contacts resulting from normal typing actions. This approach makes it possible for the user to rest their fingers on the keys, allowing them to type as they would on a regular keyboard.
As the user places their fingers on the surface, the touch sensors (one or more per key) and vibration sensors are simultaneously activated. Signals from both the touch and vibration sensors are translated into a series of input events. Input events are then temporally correlated to determine the location of the finger contact and activation of the corresponding key. Touch events without a corresponding “tap” (i.e., vibration) are ignored. Correlated events are then filtered to remove unwanted events and resolve ambiguous or contradictory results. For example, the present invention is able to detect the difference between an intentional key press and when a user has set their hands down on the keyboard in preparation for typing.
The present invention has significant advantages over traditional touch sensitive input devices. One such advantage is that the user can rest their fingers on the keys without causing a key actuation to occur. Another is that the user can type by touch without having to look at the keyboard.
Preferred and alternative examples of the present invention are described in detail below with reference to the following drawings:
Memory 170 is in data communication with the CPU 110. The memory 170 includes program memory 180 and data memory 190. The program memory 180 includes operating system software 181, tap/touch detection software 182 and other application software 183. The data memory 190 includes a touch capacitive sensor history array 191, user options/preferences 192, and other data 193.
As the user's fingers come into contact with the flat planar surface, the capacitive touch sensors 130 are asserted. Periodically, the CPU 110, executing the keyboard operating system software 181, collects the raw sensor data from the touch 130 and tap 140 sensors and stores the raw sensor date in the data memory 191.
In a separate thread of execution, the CPU 110 continuously executes the tap and touch detection and location software (algorithm) 182 described herein to process the sensor data produced by the keyboard into a sequence of key “up” and “down” states. Each execution of the algorithm constitutes a “cycle”, which is the basic timing unit for the algorithm. When a valid key activation is detected, the CPU 110, supported by the touch/tap detection software 182, performs an algorithmic analysis of the sensor data contained in the memory 191 to determine which area of the planar surface was touched and tapped on. When a valid tap/touch location is calculated by the algorithm 182, it is passed to the keyboard operating system software 181 where it is mapped into a specific keyboard function code. Typical keyboard functions include standard keyboard alphanumeric keys, function and navigation keys. The mapped function code is then sent to a connected host computer terminal 194 through a standard peripheral/host interface like USB or PS/2.
At Stage 1 (
A Tap SensorChannel invoked by the SC_Tap_CaptureData method 220 identifies the temporal occurrence of a finger initiated tap on the surface.
At stage 2 (
A Touch InputChannel process invoked by the IC_Touch_GetEvents method 310 looks for user touch input events. The CPU 110 executing the Touch InputChannel process analyzes stored touch capacitive sensor data, creating a Touch input event for each signal that exceeds a threshold value.
A Tap multilateration InputChannel invoked by the IC_TapMultilateration_GetEvents method 330 uses the relative time difference of arrival (TDOA) of a tap event at each vibration sensor to calculate the coordinate of the tap location on the keyboard and create an input event. The CPU 110 uses the technique of multilateration to triangulate the source location of a signal given three or more detectors of that signal at a fixed known location. The CPU 110 using multilateration takes the relative arrival time to each accelerometer stored in the tap event record and calculates the most likely location on the keyboard that the tap has occurred, based on the experimentally measured speed of propagation of the vibration wave on the surface. The keys that fall near the calculated tap location are chosen as candidate keys in the generated input event.
In one embodiment, the Tap multilateration algorithm includes a method for detecting and eliminating external (off keyboard) vibrations from consideration as a tap event. A common problem occurs when a user is moving their fingers on the surface of the keyboard, but not tapping, at the same time as an external vibration source is activating the vibration sensors. Unless the external tap is filtered, this leads to a false positive as the vibration is correlated to a change in the touch sensors. It is therefore important to be able to detect external vibrations and filter them out. The Tap Multilateration algorithm uses the characteristic of the physical structure of the surface to detect an external tap. Any external tap causes both the left and right accelerometer to fire before the center accelerometer because the external vibration is carried through the left and right feet of the keyboard to the left and right accelerometers before they propagate to the center detector—the center detector is last. If the conditions for both approaches are met, then the signal has a high probability of originating as an external vibration and can be eliminated as a tap event.
A Tap Amplitude InputChannel process invoked by the IC_TapAmplitude_GetEvents method 330 uses the relative differences in tap signal amplitude to calculate the coordinate of the tap location on the keyboard and create an input event locate. An amplitude variance algorithm takes the relative amplitudes recorded by each of the accelerometers to triangulate and calculate the coordinates of the tap location on the keyboard, based on an experimentally measured linear force response approximation of the vibration wave in the surface material. The keys that fall near the calculated amplitude tap location are chosen as candidate output keys.
In one embodiment, the Tap amplitude differential process 330 includes an approach for detecting and disqualifying external vibrations as tap events. When a tap occurs on the surface of the keyboard, except for a few known coordinates on the surface, there is usually a large differential in the amplitudes detected by each accelerometer a characteristic that is the basis for the tap amplitude differential process 330. However, when an external tap occurs, the amplitudes detected by each sensor are often very close in amplitude and can be used to identify the tap as a potential external tap and disqualifying it from further consideration.
A Press InputChannel process invoked by the IC_Press_GetEvents method 340 detects input events that occur when a resting finger is pressed hard onto the keyboard surface. It recognizes and remembers the touch signal strength of the resting finger and measures the difference between the resting finger and the pressed finger. If the signal strength difference exceeds a threshold value then an input event is generated.
A Tap Waveform InputChannel process invoked by the IC_TapWaveform_GetEvents method 350 compares the shape of the tap signal waveform to recognize known shapes and thus to calculate the coordinate of the tap location on the keyboard and create an input event. Exemplary vibration waveform are recorded and stored for each location on the surface in multiple use environments. In one embodiment, each of the recorded waveforms is analyzed and a number of unique characteristics (a “fingerprint”) of the waveforms are stored rather than the complete waveform. The characteristics of each user-initiated tap occurrence are compared with stored characteristics for each key in the database and the best match is found. Characteristics of the waveform that can contribute to uniquely identifying each tap location include, but are not limited to the following: the minimum peak of the waveform; the maximum peak of the waveform; the rate of decay of the waveform; the standard deviation of the waveform, the Fast Fourier Transform of the waveform; the average frequency of the waveform, the average absolute amplitude of the waveform; and others.
At stage 3 (
Correlation Phase 1, shown in block 410, analyzes the input events to determine how many events are available in history and what their relative time difference is from each other;
Correlation Phase 2 shown in block 420, generates pairs of event (duples) that are possible combinations;
Correlation Phase 3, shown in block 430, generates tuples (three or more events), from the set of calculated duples;
Correlation Phase 4, shown in block 440, reduces the sets of candidate tuples and duples, eliminating any of the combinations that are not fully reflexively supporting; and
Correlation Phase 5, shown in block 450, generates new correlated input events from the set of tuples, replacing the individual input events that make up the tuple with a single correlated input event.
The InputCorrelationManager process 400 requests historical input event data from the InputEventManager, redundant events are eliminated from the input event history, and new correlated input events are created. All input events that contributed to a correlated event are removed from the input event history database.
At stage 4 (
The embodiment implements a rule execution engine for sequentially applying filter rules to a correlated input event set. Each filter is defined as a rule that operates on a specific aspect of the input event set, changing scores and updating the long term state of the InputManager system. Filters have access to the complete set of input events and are allowed to either remove the event from processing consideration and/or reduce the set of candidate keys within the event. Filters are also allowed to access and update the long term (multi-cycle) state of the input manager in support of long-term trend and behavioral analysis. The long-term state feeds back into the other stages of input event processing.
A set of correlated input events calculated by the InputCorrelationManager is passed to the InputFilterManager through the IFM_FilterEvents (block 500) method. The rule engine applies filter rules to each element of the input event set at block 520 in rule registration order. The result of rules is a set of modifications that are applied to the (filtered) input events in block 530, and which are output at block 540 to the next stage of processing. The embodiment implements a number of rules that address special cases for key input.
The embodiment includes a vertical touch filter rule. The vertical touch filter adjusts key probabilities for events with candidate keys that are vertically adjacent. As the user types on the keys above the home row, the finger extends and “lies out” on the keyboard, often activating both the intended key above the home row and the key immediately below it on the home row. The filter detects the signature of that situation and boosts the score of the topmost candidate key in the vertical adjacency as the one most likely typed. The boost factor is appropriately scaled such that a mistype between the vertically adjacent keys will not overcome a strong signal on the lower key. Thus the boost is small enough to favor the higher key, but not preclude selection of the lower key on a partial mistype onto the higher key boundary.
The embodiment includes a next key filter. The next key filter adjusts key probabilities for events with candidate keys that are ambiguous (equally scored). The filter uses a simple probability database that defines, for any given character, the most likely character to follow that character in the current target language. The current language is specified by target national language key layout of the keyboard. The next character probability has no relationship to words or the grammatical structure of the target language. It is the probability distribution of character pairs in the target language.
In one embodiment, a set down filter detects the signature of input events resulting from the user setting their hands into a rest position on the home row of the keyboard. “Setting down” can occur after a period of nonuse of the keyboard or during a pause in active typing. The filter eliminates the unwanted key activations that occur when the fingers make contact with the home row keys during the set down.
The set down filter is a multicycle filter that updates and relies on the long-term state of the input manager and input event queues. The set down filter processes in two distinct phases. Phase 1 is the detection phase, analyzing the correlated input event set looking for two or more simultaneous home row events that include multiple touch activations on the home row with a close temporal proximity. If a set down is detected, then the long term set down state is asserted for subsequent processing cycles and event translation to key activation. Once set down state is asserted, all input events are deferred until the set down is completed. Phase 2 is the completion phase, analyzing the deferred and new events and either qualifying or disqualifying events from participating in the set down. Set down termination is determined by any of: exceeding the maximum time duration for a set down, exceeding the maximum time duration between individual events within the set down (gap threshold) or detecting a non-home row input event. When any of the set down termination conditions are met, set down state is cleared by the filter. Any deferred events are either removed as part of the set down or released for processing. A set down detection does not always result in events being removed as set down completion may detect a termination that disqualifies home row events from participating in the set down.
In one embodiment, a typing style filter analyses the input events and long-term state of the InputManager to determine what the typing style of the current user is. It then sets various control parameters and long-term state values that feedback (are used by) other filters including set down and special case.
In one embodiment, a multiple modifier filter prevents the accidental activation of two or more modifier keys due to mistyping. The modifier keys typically occupy the periphery of the keyboard and are difficult to activate properly, particularly for a touch typist. The multiple modifier filter adjusts key probabilities for events with modifier keys, favouring the shift key as the most commonly used modifier, and lowering the score for the caps lock key as a rarely used key. The adjusted scores eliminate many of the inadvertent caps lock activations when reaching for a shift key.
At stage 5 (
While the focus of the embodiment described herein is for a keyboard application, someone skilled in the art will see that the system could also be successfully applied to any type of touch-screen device.
While the preferred embodiment of the invention has been illustrated and described, as stated above, many changes can be made without departing from the spirit and scope of the invention. Accordingly, the scope of the invention is not limited by the disclosure of the preferred embodiment. Instead, the invention should be determined entirely by reference to the claims that follow.
This application claims the benefit of U.S. Provisional Application Ser. No. 61/359,235, filed Jun. 28, 2010, which is incorporated by reference.
| Number | Date | Country | |
|---|---|---|---|
| 61359235 | Jun 2010 | US |
