Classifying the intent of user input

Abstract

Different types of user inputs can be input by a user via a keyboard of an input device. These different types of user inputs include, for example, key strikes, multi-touch interactions, single finger motions, and/or mouse clicks. Touch information regarding the pressure applied to the keys of a pressure sensitive keyboard over time (or the contact area of the user input for other types of keyboards over time) is used to classify the intent of the user input as one of the various types of user inputs.

Description

BACKGROUND

Computing devices oftentimes have multiple different types of input mechanisms. These input mechanisms can include, for example, keyboards, virtual keyboards, mice, track pads, and so forth. Although these different types of mechanisms provide multiple input options for the user, they are not without their problems. One such problem is that given the number and different types of input mechanisms, it can be difficult for a user to manage the different input mechanisms, particularly in a mobile setting.

SUMMARY

Classifying the intent of user input techniques are described.

In one or more implementations, touch information regarding a user input to an input device is obtained. Based on this touch information, an intent of the user input is classified as being either a key strike or one or more other types of input.

In one or more implementations, an input device is configured to provide an output that indicates touch information regarding a user input. This touch information is usable by one or more modules to determine a user intent in providing the user input.

This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter.

BRIEF DESCRIPTION OF THE DRAWINGS

The detailed description is described with reference to the accompanying figures. In the figures, the left-most digit(s) of a reference number identifies the figure in which the reference number first appears. The use of the same reference numbers in different instances in the description and the figures may indicate similar or identical items. Entities represented in the figures may be indicative of one or more entities and thus reference may be made interchangeably to single or plural forms of the entities in the discussion.

FIG. 1 is an illustration of an example input device implementing the techniques described herein.

FIG. 2 is an illustration of an example computing device employing the techniques described herein.

FIG. 3 is an illustration of the computing device of FIG. 2 displaying a virtual keyboard.

FIGS. 4A and 4B illustrate an example input device with example user inputs.

FIG. 5 is an illustration of a system in an example implementation that is operable to employ the techniques described herein.

FIG. 6 depicts an example of a graph of pressure at a particular location over time.

FIG. 7 depicts another example of a graph of pressure at a particular location over time.

FIG. 8 is a flowchart illustrating an example process for implementing the techniques described herein in accordance with one or more embodiments.

FIG. 9 is a flowchart illustrating another example process for implementing the techniques described herein in accordance with one or more embodiments.

FIG. 10 illustrates an example system including various components of an example device that can be implemented as any type of computing device as described with reference to FIGS. 1-9 to implement embodiments of the techniques described herein.

DETAILED DESCRIPTION

Overview

Classifying the intent of user input techniques are described. Different types of user inputs can be provided by a user via a keyboard of an input device. These different types of user inputs include, for example, key strikes, multi-touch interactions, and/or mouse clicks. Force information regarding the pressure applied to a pressure sensitive keyboard, as well as one or more locations where that pressure is applied, is used to classify the intent of the user input as one of the various types of user inputs. Contact information regarding the contact area of the user input for other types of keyboards (e.g., capacitive and/or resistive systems), as well as one or more locations where that contact area occurs, may similarly be used to classify the intent of the user input as one of the various types of user inputs.

In the following discussion, an example environment is first described that may employ the techniques described herein. Example procedures are then described which may be performed in the example environment as well as other environments. Consequently, performance of the example procedures is not limited to the example environment and the example environment is not limited to performance of the example procedures.

Example Environment and Procedures

FIG. 1 is an illustration of an example input device 100 implementing the techniques described herein. In the illustrated example, the input device 100 is configured as a keyboard having a QWERTY arrangement of keys although other arrangements of keys are also contemplated. Further, other non-conventional configurations are also contemplated, such as a game controller, a remote control device, a configuration to mimic a musical instrument, and so forth. Thus, the input device 100 and keys incorporated by the input device 100 may assume a variety of different configurations to support a variety of different functionality.

The input device 100 is a multi-use device, supporting various types of user inputs. A user input can have various different intents, such as a multi-touch interaction, a key strike, a mouse click, and so forth. The input device 100 supports these different types of user inputs in a common input area, such as the keyboard. For example, the user may touch the keys “q”, “w”, and “e” on the keyboard, and the intent of those touches may be determined to be key strikes selecting the letters “q”, “w”, and “e”. The user may also swipe his or her finger across the keys “q”, “w”, and “e” on the keyboard, and the intent of that swipe may be determined to be a multi-touch interaction or single finger motion. This determination of the intent of the user input is discussed in more detail below.

The input device 100 may be communicatively coupled to a computing device 102. The input device 100 may be physically separate from the computing device 102 and communicate with the computing device 102 via any of a variety of conventional communication mechanisms. For example, the input device 100 may communicate with the computing device via a wireless connection, via a wired connection, via communication contacts of the devices 100 and 102 in contact with one another, and so forth.

The computing device 102 may range from full resource devices with substantial memory and processor resources to a low-resource device with limited memory and/or processing resources. The computing device 102 may also relate to software that causes the computing device 102 to perform one or more operations. In various implementations, the computing device 102 may assume a variety of different configurations, such as for computer 104, mobile 106, and television 108 uses. Each of these configurations includes devices that may have generally different constructs and capabilities, and thus the computing device 102 may be configured according to one or more of the different device classes.

For instance, the computing device 102 may be implemented as the computer 104 class of a device that includes a personal computer, desktop computer, a multi-screen computer, laptop computer, netbook, and so on. The computing device 102 may also be implemented as the mobile 106 class of device that includes mobile devices, such as a mobile phone, portable music player, portable gaming device, a tablet computer, a multi-screen computer, and so on. The computing device 102 may also be implemented as the television 108 class of device that includes devices having or connected to generally larger screens in casual viewing environments. These devices include televisions, set-top boxes, gaming consoles, and so on.

The computing device 102 also includes an input/output module 110. The input/output module 110 is representative of functionality relating to processing of inputs and rendering outputs of the computing device 102. A variety of different inputs may be processed by the input/output module 110, such as inputs relating to functions that correspond to keys of the input device 100, inputs that are multi-touch interactions or single finger motions recognized through the input device 100 and that cause operations to be performed that correspond to the multi-touch interactions or single finger motions, and so forth. Thus, the input/output module 110 may support a variety of different input techniques by recognizing and leveraging a division between types of inputs including key presses, multi-touch interactions, single finger motions, and so on.

Alternatively, the input device 100 may be included as part of a computing device. FIG. 2 is an illustration of an example computing device 202 employing the techniques described herein. The computing device 202 may assume a variety of different configurations, analogous to computing device 102 of FIG. 1. For example, the computing device 202 may be configured for mobile use, such as a mobile phone, a tablet computer, and so on. The computing device 202 may also include an input/output module 204, analogous to input/output module 110 of FIG. 1, and may also relate to software that causes the computing device 202 to perform one or more operations.

The computing device 202 includes a display device 206 via which, in a display mode, various data and information can be displayed. The display device 206 may use a variety of display technologies. These display technologies may include, for example, liquid crystal (LCD) display technologies, light-emitting diode (LED) display technologies, organic light-emitting diode (OLED) display technologies, plasma display technologies, and so forth. Although examples of display technologies are discussed herein, other display technologies are also contemplated.

The display device 206 can be a touchscreen display, and various user inputs can be provided by the user touching the display device 206. The computing device 202 also supports a keyboard mode in which a virtual keyboard is displayed. FIG. 3 is an illustration of computing device 202 displaying a virtual keyboard 302. Virtual keyboard 302 is a multi-use device, supporting various types of user inputs analogous to the keyboard of input device 100 of FIG. 1. However, rather than being a physically separate device, the keyboard 302 is a virtual keyboard that is part of the computing device 202. Thus, the display device 206 also serves as the input device in computing device 202.

User inputs to the input device (e.g., the input device 100 of FIG. 1 and/or the display device 206 of FIG. 2) are sensed by an input sensing component of the input device. The input device can use a variety of different input sensing technologies. These input sensing technologies may include pressure sensitive systems that sense pressure or force. These input sensing technologies may also include capacitive systems and/or resistive systems that sense touch. These input sensing technologies may also include optical based images that sense reflection or disruption of light from objects touching (or close to) the surface of the display device, such as Sensor in Pixel (SIP) systems, Infrared systems, optical imaging systems, and so forth. Other types of input sensing technologies can also be used, such as surface acoustic wave systems, acoustic pulse recognition systems, dispersive signal systems, and so forth. Although examples of input sensing technologies are discussed herein, other input sensing technologies are also contemplated.

In response to a user touching the keyboard at a particular time, the input sensing component determines a location that was touched at that particular time. Various modules of the input sensing component may use different algorithms or techniques to identify the location that was touched, and these algorithms or techniques can vary based on the particular input sensing technology being used. For pressure sensitive input sensing components, these various modules also identify the amount of pressure applied at the location that was touched. For other types of input sensing components, these various modules also identify the area that was touched (the contact area).

The determination of a location that is touched can be determined at various frequencies, such as 1000 times per second although other sampling frequencies are contemplated. It should be noted that the frequency at which the determination of a location of a touch is made is sufficient to identify whether the user input conforms to the characteristics of different user intents. For example, a sampling frequency of 1000 times per second may be sufficient to determine whether the user inputs conform to the characteristics of a key strike intent, whereas lower sampling frequencies (such as 100 times per second) may not be sufficient to identify whether the user inputs conform to the characteristics of a key strike intent. Thus, in contrast to the input sensing components discussed herein, many input sensing components that determine a location that is touched at a low frequency would be unable to determine whether the user input conforms to the characteristics of some intents.

The input sensing component detects or senses the touch of an object, such as a finger of a user's hand, a stylus, a pen, and so forth. The discussions of the classifying the intent of user input techniques herein refer to the user input being provided by a user's finger (and thus the touch is also referred to as a finger impact), although this user input can alternatively be provided by a stylus or other object controlled by the user.

FIGS. 4A and 4B illustrate an example input device 400 with example user inputs. Input device 400 can be, for example, an input device 100 of FIG. 1 and/or a display device 206 of FIG. 3. The input device 400 is a multi-use device, supporting various types of user inputs. These multiple types of user inputs are supported concurrently. For example, the input device 400 can include a keyboard and the user can provide different types of inputs on that keyboard, with the techniques discussed herein being used to automatically determine the intent of that user input.

Thus, the touching of the input device 400 can have different user intents, such as a key strike, a mouse click, a single finger motion, or a multi-touch interaction, although other intents or types of user input are also contemplated. A key strike refers to user selection of a particular key on the keypad, with different locations of the keyboard being associated with different keys. A mouse click refers to the pressing of a button typically associated with a mouse or button of a track pad. A multi-touch interaction refers to a user touching multiple locations (e.g., with multiple ones of the user's fingers) concurrently to initiate one or functions of a computing device. The multi-touch interaction can also include a motion or path taken by one or more of the user's fingers. For example, a multi-touch interaction may be sliding of the user's fingers in one or more particular directions, the user's fingers tracing one or more particular characters or symbol, and so forth. A single finger motion refers to motion or path taken by the user's finger to move a cursor, pointer, or other object (e.g., an icon, file, etc. being dragged), or to initiate one or more functions of the computing device. A single finger motion intent and a multi-touch interaction intent can also be referred to as movement intents because they typically (although not always for multi-touch interactions) include movement by the user's finger.

FIG. 4A illustrates an example user input the intent of which is a key strike of the letter “d” or a mouse click. Whether the intent of the user input was a key strike or a mouse click can be determined in different manners, such as based on the characteristics of a key strike and the characteristics of a mouse click as discussed below.

FIG. 4B illustrates an example user input the intent of which is a single finger motion moving from left to right. The ending position of the user's finger is illustrated using a dashed outline of a hand. Whether the intent of the user input was a single finger motion can be determined in different manners, such as based on the distance the touch travels and/or a velocity of the touch, as discussed in more detail below.

As illustrated in FIGS. 4A and 4B, both user inputs begin by touching a location of the keyboard corresponding to the letter “d”. However, the intent of the user (e.g., a key strike or a multi-touch interaction) can be determined using the techniques discussed herein. The user need not press any button or select any menu option to indicate his or her intent (e.g., whether the user desires to input key strikes or multi-touch interactions). Rather, the user can simply provide his or her input and the techniques discussed herein automatically determine the intent of that input.

The touch, as well as one or more locations of the touch as sensed by one or more sensors as discussed above, is used to classify the intent of a user input, such as classifying the user input as a key strike or other type of input. The user input refers to the touching by the user of one or more portions of the input device. The touch may be sensed using a dense array of pressure sensors that are sampled at a sufficient frequency to characterize the touch, as discussed above. The touch may alternatively be sensed using a capacitive sensor, with the impact being sensed indirectly by noting the rapid change in coupling as the finger conforms to the surface and then rebounds. Regardless of the type of sensors use, the location of the touch can be readily determined based on (e.g., by interpolating) data from the sensors in the region where the touch occurred.

The touch can have associated force information (in situations in which the input sensing technologies used include pressure sensitive systems) or contact information (in situations in which other input sensing technologies are used). The force information refers to the pressure applied by the user when touching the keyboard. The contact information refers to the area that is touched (the portions of the touchpad or touchscreen that were touched by the user's finger or other object, the amount of light reflected by the user's finger or other object, etc.) by the user when touching the keyboard.

In the following discussions, the classifying the intent of user input techniques are discussed with reference to force information and a user input device that is a pressure sensitive device. However, the classifying the intent of user input techniques can alternatively be implemented using other types of sensors as discussed above. In such situations, rather than using force information to classify the intent of the user input, the contact information is used to classify the intent of the user input. When using the contact information, the contact area is treated analogously to the pressure discussed below.

FIG. 5 is an illustration of a system 500 in an example implementation that is operable to employ the techniques described herein. The system 500 includes a touch information collection module 502 and an input classification module 504. System 500 may be implemented, for example, in the input device 100 and/or the computing device 102 of FIG. 1, or the computing device 202 of FIGS. 2 and 3. Thus, for example, module 502 may be implemented in input device 100 and module 504 may be implemented in computing device 102, both modules 502 and 504 may be implemented in input device 100, and so forth.

Touch information collection module 502 obtains an indication of the amount of pressure applied by a user's finger over time (or alternatively contact information over time, as discussed above), as well as the locations of the touch over time, as sensed by the input sensing component as discussed above. Module 502 obtains (e.g., receives or generates) touch information 506 regarding user inputs to the keyboard of the input device. The touch information 506 identifies the characteristics of and locations of touches. These characteristics and locations of a touch can include, for example, the size of the touch (e.g., the amount of area touched), changes in the size of the touch over time, the shape of the touch (e.g., a geographic shape or outline of the area touched), changes in the shape of the touch over time, the location of the touch over time, the change in pressure of the touch over time, the movement of the touch (directions and locations that are touched), a velocity of the touch, an acceleration of the touch, a distance the touch travels, and so forth.

Based on the touch information 506, input classification module 504 classifies the intent of the user input, and outputs or otherwise makes available the input intent classification 508. Additional information, such as the touch information 506, can also be output or otherwise made available along with the input intent classification 508. The intent of the user input can be, for example, a key strike, a multi-touch interaction, a mouse click, and so forth as discussed below. The input intent classification 508 can subsequently be used by one or more other modules to take the appropriate action based on the user input. For example, if the intent is a key strike, then the key associated with the location pressed by the user as the user input is recorded as user selection of that key. By way of another example, if the intent is a multi-touch interaction, then the touch information is used to determine which multi-touch interaction was input by the user and the corresponding action taken.

Generally, the intent of the user input is classified based on the touch information 506. The force applied by the user's finger over time, and optionally the one or more locations where that force is applied, is analyzed in view of characteristics of the different intents supported by input classification module 504. A user input that conforms to the characteristics of a particular intent is classified as that particular intent, whereas a user input that does not conform to the characteristics of a particular intent is not classified as that particular intent.

In one or more embodiments, the intent of a user input can be classified as a key strike, a single finger motion, a multi-touch interaction, or other input. A user input is compared to the characteristics of one or more of a key strike, a single finger motion, a multi-touch interaction. If the user input conforms to the key strike characteristics then the intent of the user input is classified as a key strike, if the user input conforms to the single finger motion characteristics then the intent of the user input is classified as a single finger motion, and if the user input conforms to the multi-touch interaction characteristics then the intent of the user input is classified as a multi-touch interaction.

Furthermore, if the user input conforms to the characteristics of no particular intent (e.g., a key strike, a single finger motion, a multi-touch interaction, etc.), then the intent of the user input is classified as an “other” intent. The other intent refers to some other type of intent, typically an intent that is filtered out by system 500. Examples of such user inputs that can be classified as an “other” intent can be the user's hands resting on the keyboard, sensor noise, the device being bumped, and so forth. Thus, not only can input classification module 504 distinguish between different intents of user inputs to the keyboard, but module 504 can also filter out user inputs to that same keyboard that are not intended to be user inputs to the input device that are acted upon by the computing device.

A user input intent may be characterized by a force graph shape indicating the force applied by the user's finger over time for the intent. Different user input intents can have different associated force graph shapes, and input classification module 504 can maintain these different force graph shapes and compare user inputs to these different force graph shapes. A user input (as characterized by touch information 506) that conforms to the force graph shape for an intent is classified as being that particular intent. Whether a user input conforms to a particular force graph shape can be determined in a variety of different manners. For example, a force graph can be generated based on the touch information obtained for the user input, and the generated force graph can be compared to the one or more maintained force graph shapes. If the generated force graph shape matches a particular maintained force graph shape, then the user input conforms to the intent associated with that particular maintained force graph shape. Whether two graph shapes match can be determined using a variety of public and/or proprietary graph or shape matching techniques.

The behavior of a particular touching of the keyboard over a lifetime of the touch may be used to classify a user input as a key strike or other type of input (e.g., a multi-touch interaction or single finger motion). The lifetime of the touch refers to a time duration that begins when the user's finger touching the surface is sensed and ends when the user's finger is no longer sensed as touching the surface. The user's finger can remain approximately stationary for the lifetime of the touch (e.g., which may be typical for a key strike), or can move across the keypad during the lifetime of the touch (e.g., which may be typical for multi-touch interactions and single finger motions).

The intent of a user input may be classified as a movement in response to the touch travelling at least a threshold distance. This threshold distance can be a fixed distance (e.g., 1.5 inches) or a relative distance (e.g., 10% of the width of the keyboard). The travelling of a touch refers to the distance moved by the user's finger while being moved along some path during the lifetime of the touch. If multiple touches are sensed concurrently then the intent of the user input may be classified as a multi-touch interaction, whereas if only a single touch is sensed at a particular time then the intent of the user input at that particular time may be classified as a single finger motion.

The intent of a user input may also be classified as a movement in response to the touch having at least a threshold velocity and a short (e.g., less than a threshold value) lifetime. The threshold value of the lifetime of the touch may be 0.25 seconds, although other threshold values are contemplated. The velocity of a touch refers to the distance moved by the user's finger while being moved along some path during the lifetime of the touch divided by the time duration of the lifetime of the touch. For example, the velocity may be 4 inches/second, although other velocities are contemplated. If multiple touches are sensed concurrently then the intent of the user input may be classified as a multi-touch interaction, whereas if only a single touch is sensed at a particular time then the intent of the user input at that particular time may be classified as a single finger motion.

Generally, a key strike (also referred to as a tap) is characterized by a sharp rise in force followed by a sharp decrease in force as the finger rebounds from the surface, and optionally followed by a rise in force again as the momentum of the remainder of the finger continues towards the surface of the keyboard. The pressure of a touch is analyzed over time, and based on these characteristics of a key strike the user input is classified as either a key strike or some other type of input (e.g., a multi-touch interaction). User inputs that conform to these characteristics are classified as key strikes, and user inputs that do not conform to these characteristics are classified as other types of user inputs. If the user inputs include multiple touches concurrently, then the user input may be classified as multiple key strikes if each touch conforms to the characteristics of a key strike and the touches are for particular locations (e.g., locations corresponding to a modifier key on the keyboard, such as a “shift” key, a “ctrl” key, an “alt” key, and so forth). Alternatively, if the user inputs include multiple touches concurrently, then the user input is classified as some other type of input rather than a key strike.

It should be noted that these characteristics of a key strike allow the intent of a user input to be distinguished between a key strike and an “other” intent (e.g., the user resting his or her hands on the keyboard). The key strike is characterized by a particular force graph shape indicating the force applied by the user's finger over time. If a user intends to strike a key then the input conforms to this particular force graph shape, and if the user intends to rest his or her hands on the keyboard then the input does not conform to this particular force graph shape. Thus a key strike intent or an “other” intent can be readily determined based on whether the user input conforms to this particular force graph shape.

The characteristics of a key strike can also include temporal information. For example, the characteristics of a key strike may include the touch being at the same location (not moving) or having a location that moves less than a threshold amount (e.g., less than the width or height of a key on the keyboard, although other threshold amounts are contemplated).

FIG. 6 depicts an example 600 of a graph of pressure of a touch over time. The vertical axis is pressure or force (e.g., in grams), and the horizontal axis is time (e.g., in milliseconds). A line 602 represents the pressure applied as the touch over time. The pressure applied as the touch over time can be analyzed to determine whether the pressure applied conforms to the characteristics of a key strike.

In analyzing the pressure an initial impact time duration may be determined, which is a duration of time that begins when the pressure applied as the touch rises to (e.g., is equal to or greater than) a start threshold amount. This start threshold amount may be 200 grams, although other start threshold amounts are contemplated. This duration of time ends when the pressure applied as the touch reaches a local minimum before rising again (e.g., as the momentum of the remainder of the finger continues towards the surface of the keyboard) or the pressure applied as the touch drops to (e.g., is equal to or less than) a stop threshold amount. This stop threshold amount may be the same as the start threshold amount (e.g., 200 grams) or a lesser amount (e.g., 100 grams), although other values for the stop threshold amount are contemplated. Thus, the initial impact time duration can be different for different user inputs.

In example 600, an initial impact time duration 604 is illustrated. The initial impact time duration begins at point 606 where the pressure applied as the touch rises to the start threshold amount, and ends at point 608 where the pressure applied as the touch drops to a local minimum before rising again.

Also in analyzing the pressure a maximum force may be identified. This maximum force refers to a local maximum of the pressure applied as the touch during the initial impact time duration 604 and after the pressure applied as the touch rises to the start threshold amount. In example 600, the maximum force is at point 610.

Given the initial impact time duration and/or the maximum force, various rules or criteria are applied to determine whether the pressure applied conforms to the characteristics of a key strike. These rules or criteria may include an initial impact criteria, a maximum force criteria, and/or an initial impact ending force criteria.

The initial impact criteria refers to the duration of the initial impact time duration, indicating that the initial impact time duration is to be within a particular time range. This particular time range can be, for example, between 5 milliseconds (ms) and 25 ms, although other time ranges are contemplated. Thus, for example, the initial impact time duration is to be at least 5 ms and no more than 25 ms in order for the pressure applied as the touch to conform to the characteristics of a key strike.

The maximum force criteria refers to when, within the initial impact time duration, the maximum force occurs. The maximum force criteria indicates that the maximum force is to occur within a threshold amount of time, such as 12 ms although other amounts of time are also contemplated. The maximum force criteria also indicates that the maximum force is not to occur in a last portion of the initial impact time duration. This last portion may be the last third of the initial impact time duration, although other portions are contemplated. Thus, for example, the maximum force is to occur within the first 12 ms of the initial impact time duration and is not to occur within the last ⅓ of the initial impact time duration in order for the pressure applied as the touch to conform to the characteristics of a key strike.

The initial impact ending force criteria refers to the pressure of the touch at the end of the initial impact time duration. The initial impact ending force criteria indicates that the pressure of the touch at the end of the initial impact time duration is to be less than a particular percentage of the maximum force. This particular percentage may be 80%, although other percentages are contemplated. Alternatively, the initial impact ending force criteria can indicate that the pressure of the touch is to be less than another particular percentage of the maximum force in response to the maximum force being reached quickly (within a threshold amount of time). This other particular percentage may be 90% and the threshold amount of time may be 5 ms, although other percentages and threshold amounts of time are contemplated. Thus, for example, at the end of the initial impact time duration the pressure of the touch is to be less than 80% of the maximum force, or less than 90% of the maximum force if the maximum force was reached within 5 ms, in order for the pressure applied as the touch to conform to the characteristics of a key strike.

If the initial impact criteria, the maximum force criteria, and the initial impact ending force criteria all conform to the characteristics of a key strike, then the pressure applied as the touch conforms to the characteristics of a key strike and the user input is classified as a key strike. However, if one or more of the initial impact criteria, the maximum force criteria, and the initial impact ending force criteria does not conform to the characteristics of a key strike, then the pressure applied as the touch does not conform to the characteristics of a key strike and the user input is not classified as a key strike.

Characteristics of a key strike are discussed above. Characteristics of other types of inputs can also be maintained. The pressure of a touch may be analyzed over time and a determination made as to whether the user input conforms to the characteristics of another type of input. User inputs that conform to the characteristics of another type of input are classified as that other type of input, while user inputs that do not conform to the characteristics of that other type input are not classified as that other type of input.

For example, a mouse click may be characterized by a slow rise in force followed by a sharp decrease in force as the finger rebounds from the surface. The pressure of a touch is analyzed over time, and based on these characteristics of a mouse click the user input is classified as either a mouse click or some other type of input (e.g., a key strike). User inputs that conform to these characteristics are classified as mouse clicks, and user inputs that do not conform to these characteristics are classified as other types of user inputs.

The characteristics of a mouse click can also include temporal information. For example, the characteristics of a mouse click may include the touch being at the same location (not moving) or having a location that moves less than a threshold amount (e.g., 0.25 inches, although other threshold amounts are contemplated).

FIG. 7 depicts an example 700 of a graph of pressure of a touch over time. The vertical axis is pressure or force (e.g., in grams), and the horizontal axis is time (e.g., in milliseconds). A line 702 represents the pressure applied as the touch over time. The pressure applied as the touch over time can be analyzed to determine whether the pressure applied conforms to the characteristics of a mouse click.

In analyzing the pressure a touch time duration may be determined, which is a duration of time that begins when the pressure applied as the touch rises to (e.g., is equal to or greater than) a start threshold amount. This start threshold amount may be 200 grams, although other start threshold amounts are contemplated. This duration of time ends when the pressure applied as the touch drops to (e.g., is equal to or less than) a stop threshold amount. This stop threshold amount may be the same as the start threshold amount (e.g., 200 grams) or a lesser amount (e.g., 100 grams), although other values for the stop threshold amount are contemplated.

In example 700, a touch time duration 704 is illustrated. The touch time duration begins at point 706 where the pressure applied as the touch rises to the start threshold amount, and ends at point 708 where the pressure applied as the touch drops to the stop threshold amount.

Also in analyzing the pressure a maximum force may be identified. This maximum force refers to a local maximum of the pressure applied as the touch during the touch time duration 704 and after the pressure applied as the touch rises to the start threshold amount. In example 700, the maximum force is at point 710.

Given the touch time duration and/or the maximum force, various rules or criteria are applied to determine whether the pressure applied conforms to the characteristics of a mouse click. These rules or criteria may include rules or criteria referring to when the maximum force occurs. The rules or criteria may indicate that the maximum force is to occur at least a threshold amount of time (e.g., 25 ms, although other amounts of time are also contemplated) after the pressure applied as the touch rises to the start threshold amount. The rules or criteria may indicate that the maximum force is not to occur in a beginning portion of the touch time duration. This beginning portion may be the beginning half of the touch time duration, although other portions are contemplated. Thus, for example, the rules or criteria may indicate that the maximum force is to occur at least 25 ms after the touch time duration beginning, and is not to occur within the first half of the touch time duration in order for the pressure applied as the touch to correspond to the characteristics of a mouse click.

As indicated above, although discussed herein with reference to force information and a user input device that is a pressure sensitive input device, the classifying the intent of user input techniques can alternatively be implemented using other types of sensors and input sensing technologies. In such situations, rather than using the pressure applied by a user input to classify the intent of the user input, the contact area of the user input (the portions of the touchpad or touchscreen that were touched by the user's finger or other object, the amount of light reflected by the user's finger or other object, etc.) may be used to classify the intent of a user input. The contact area (e.g., measured in millimeters or other units) is treated analogously to the pressure discussed above. For example, referring to FIG. 5, a key strike is characterized by a sharp rise in contact area followed by a sharp decrease in contact area as the finger or other object touching the surface rebounds from the surface, and optionally followed by a rise in contact area again as the momentum of the remainder of the finger or other object touching the surface of the keyboard continues towards the surface.

FIG. 8 is a flowchart illustrating an example process 800 for implementing the techniques described herein in accordance with one or more embodiments. Process 800 is carried out by an input classification module, such as input classification module 504 of FIG. 5, and can be implemented in software, firmware, hardware, or combinations thereof. Process 800 is shown as a set of acts and is not limited to the order shown for performing the operations of the various acts. Process 800 is an example process for implementing the techniques described herein; additional discussions of implementing the techniques described herein are included herein with reference to different figures.

In process 800, touch information regarding a user input to an input device is obtained (act 802). This touch information can be obtained from a pressure sensitive input device, or other types of input devices (e.g., capacitive or resistive) as discussed above.

Based on the obtained touch information, an intent of the user input is classified as being a key strike or other type of input (act 804). The classification is performed based on characteristics of a key strike or other type of input, as discussed above. Various other types of inputs can be received as user inputs, such as multi-touch interactions, a single finger motions, and mouse clicks as discussed above.

An indication of the classification of the user input is output (act 806). This classification can be used by one or more other modules (e.g., of computing device 102 of FIG. 1) to take the appropriate action based on the user input. This classification can be output to, for example, one or more modules of the input device 104 and/or the computing device 102 of FIG. 1.

FIG. 9 is a flowchart illustrating another example process 900 for implementing the techniques described herein in accordance with one or more embodiments. Process 900 is carried out by an input classification module, such as input classification module 504 of FIG. 5, and can be implemented in software, firmware, hardware, or combinations thereof. Process 900 is shown as a set of acts and is not limited to the order shown for performing the operations of the various acts. Process 900 is an example process for implementing the techniques described herein; additional discussions of implementing the techniques described herein are included herein with reference to different figures.

In process 900, when a user input is received (e.g., the user touches the keyboard), a check is made as to whether the touch travels a threshold distance (act 902). This threshold distance can be a fixed or relative distance, such as 1.5 inches or 10% of the width of the keyboard, as discussed above.

If the touch travels a threshold distance (e.g., at least the threshold distance, equal to or greater than the threshold distance, etc.), then the touch is classified as a movement (act 904). A movement can be, for example, a single finger motion or a multi-touch interaction, as discussed above. Whether the movement is a single finger motion or a multi-touch interaction can be determined in different manners. For example, if during the time of the touch multiple touches (e.g., multiple fingers) are sensed concurrently then the movement is a multi-touch interaction, whereas if during the time of the touch just a single touch (e.g., a single finger) is sensed then the movement is a single finger motion. By way of another example, if the input device does not support (or is operating in a mode that does not support) multi-touch interactions, then the movement is a single finger motion. By way of another example, if the input device does not support (or is operating in a mode that does not support) single finger motions, then the movement is a multi-touch interaction.

However, if the touch does not travel the threshold distance, then a check is made as to whether the lifetime of the touch is less than a threshold amount (act 906). If the lifetime of the touch is less than a threshold amount (e.g., less than 0.25 seconds), then a check is made as to whether the touch has a threshold velocity (act 908). This threshold velocity can be, for example, 4 inches/second or other velocities as discussed above. If the touch has a threshold velocity (e.g., at least the threshold velocity, equal to or greater than the threshold velocity, etc.), then the touch is classified as a movement (act 904). A movement can be, for example, a single finger motion or a multi-touch interaction, as discussed above.

However, if the lifetime of the touch is not less than the threshold amount, or if the touch does not have the threshold velocity, then a check is made as to whether initial impact criteria are satisfied (act 910). The initial impact criteria are satisfied if the initial impact time duration is within a particular time range (e.g., at least 5 ms and no more than 25 ms), as discussed above.

If the initial impact criteria are not satisfied, then the touch is classified as neither a movement nor a key strike (act 912). In such situations, the touch may be interpreted as some default input (e.g., the user resting his or her fingers on the keyboard), or alternatively may be further analyzed to determine the intent of the user input (e.g., a mouse click, as discussed above).

However, if the initial impact criteria are satisfied, then a check is made as to whether maximum force criteria are satisfied (act 914). The maximum force criteria are satisfied if the maximum force occurs during a particular portion of the initial impact time duration (e.g., within the first 12 ms of the initial impact time duration, or during the first ⅔ of the initial impact time duration), as discussed above.

If the maximum force criteria are not satisfied, then the touch is classified as neither a movement nor a key strike (act 912).

However, if the maximum force criteria are satisfied, then a check is made as to whether initial impact ending force criteria are satisfied (act 916). The initial impact ending force criteria are satisfied if the pressure on the key at the end of the initial impact time duration is less than a particular percentage of the maximum force (e.g., less than 80% of the maximum force, or less than 90% of the maximum force if the maximum force occurred in the first 5 ms of the initial impact time duration), as discussed above.

If the initial impact ending force criteria are not satisfied, then the touch is classified as neither a movement nor a key strike (act 912). However, if the initial impact ending force criteria are satisfied, then the touch is classified as a key strike (act 918).

Example System and Device

FIG. 10 illustrates an example system generally at 1000 that includes an example computing device 1002 that is representative of one or more computing systems and/or devices that may implement the various techniques described herein. The computing device 1002 may, for example, be configured to assume a mobile configuration through use of a housing formed and size to be grasped and carried by one or more hands of a user, illustrated examples of which include a mobile phone, mobile game and music device, and tablet computer although other examples are also contemplated.

The example computing device 1002 as illustrated includes a processing system 1004, one or more computer-readable media 1006, and one or more I/O interfaces 1008 that are communicatively coupled, one to another. Although not shown, the computing device 1002 may further include a system bus or other data and command transfer system that couples the various components, one to another. A system bus can include any one or combination of different bus structures, such as a memory bus or memory controller, a peripheral bus, a universal serial bus, and/or a processor or local bus that utilizes any of a variety of bus architectures. A variety of other examples are also contemplated, such as control and data lines.

The processing system 1004 is representative of functionality to perform one or more operations using hardware. Accordingly, the processing system 1004 is illustrated as including hardware elements 1010 that may be configured as processors, functional blocks, and so forth. This may include implementation in hardware as an application specific integrated circuit or other logic device formed using one or more semiconductors. The hardware elements 1010 are not limited by the materials from which they are formed or the processing mechanisms employed therein. For example, processors may be comprised of semiconductor(s) and/or transistors (e.g., electronic integrated circuits (ICs)). In such a context, processor-executable instructions may be electronically-executable instructions.

The computer-readable storage media 1006 is illustrated as including memory/storage 1012. The memory/storage 1012 represents memory/storage capacity associated with one or more computer-readable media. The memory/storage component 1012 may include volatile media (such as random access memory (RAM)) and/or nonvolatile media (such as read only memory (ROM), Flash memory, optical disks, magnetic disks, and so forth). The memory/storage component 1012 may include fixed media (e.g., RAM, ROM, a fixed hard drive, and so on) as well as removable media (e.g., Flash memory, a removable hard drive, an optical disc, and so forth). The computer-readable media 1006 may be configured in a variety of other ways as further described below.

Input/output interface(s) 1008 are representative of functionality to allow a user to enter commands and information to computing device 1002, and also allow information to be presented to the user and/or other components or devices using various input/output devices. Examples of input devices include a keyboard, a cursor control device (e.g., a mouse), a microphone, a scanner, touch functionality (e.g., capacitive or other sensors that are configured to detect physical touch), a camera (e.g., which may employ visible or non-visible wavelengths such as infrared frequencies to recognize movement as gestures that do not involve touch), and so forth. Examples of output devices include a display device (e.g., a monitor or projector), speakers, a printer, a network card, tactile-response device, and so forth. Thus, the computing device 1002 may be configured in a variety of ways to support user interaction.

The computing device 1002 is further illustrated as being communicatively and physically coupled to an input device 1014 that is physically and communicatively removable from the computing device 1002. In this way, a variety of different input devices may be coupled to the computing device 1002 having a wide variety of configurations to support a wide variety of functionality. In this example, the input device 1014 includes one or more keys 1016, which may be configured as pressure sensitive keys, keys on a touchpad or touchscreen, mechanically switched keys, and so forth.

The input device 1014 is further illustrated as including one or more modules 1018 that may be configured to support a variety of functionality. The one or more modules 1018, for instance, may be configured to process analog and/or digital signals received from the keys 1016 to determine whether a keystroke was intended, determine whether an input is indicative of resting pressure, support authentication of the input device 1014 for operation with the computing device 1002, classify the intent of user input, and so on. Modules 1018 may include, for example, the input classification module 504 of FIG. 5.

Although illustrated as separate from the computing device 1002, the input device 1014 can alternatively be included as part of the computing device 1002 as discussed above. In such situations, the keys 1016 and the modules 1018 are included as part of the computing device 1002. Additionally, in such situations the keys 1016 may be keys of a virtual keyboard and/or keys of a non-virtual keyboard (e.g., a pressure sensitive input device).

Various techniques may be described herein in the general context of software, hardware elements, or program modules. Generally, such modules include routines, programs, objects, elements, components, data structures, and so forth that perform particular tasks or implement particular abstract data types. The terms “module,” “functionality,” and “component” as used herein generally represent software, firmware, hardware, or a combination thereof. The features of the techniques described herein are platform-independent, meaning that the techniques may be implemented on a variety of commercial computing platforms having a variety of processors.

An implementation of the described modules and techniques may be stored on or transmitted across some form of computer-readable media. The computer-readable media may include a variety of media that may be accessed by the computing device 1002. By way of example, and not limitation, computer-readable media may include “computer-readable storage media” and “computer-readable signal media.”

“Computer-readable storage media” may refer to media and/or devices that enable persistent and/or non-transitory storage of information in contrast to mere signal transmission, carrier waves, or signals per se. Thus, computer-readable storage media refers to non-signal bearing media. The computer-readable storage media includes hardware such as volatile and nonvolatile, removable and non-removable media and/or storage devices implemented in a method or technology suitable for storage of information such as computer readable instructions, data structures, program modules, logic elements/circuits, or other data. Examples of computer-readable storage media may include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical storage, hard disks, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or other storage device, tangible media, or article of manufacture suitable to store the desired information and which may be accessed by a computer.

“Computer-readable signal media” may refer to a signal-bearing medium that is configured to transmit instructions to the hardware of the computing device 1002, such as via a network. Signal media typically may embody computer readable instructions, data structures, program modules, or other data in a modulated data signal, such as carrier waves, data signals, or other transport mechanism. Signal media also include any information delivery media. The term “modulated data signal” means a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal. By way of example, and not limitation, communication media include wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, RF, infrared, and other wireless media.

As previously described, hardware elements 1010 and computer-readable media 1006 are representative of modules, programmable device logic and/or fixed device logic implemented in a hardware form that may be employed in some embodiments to implement at least some aspects of the techniques described herein, such as to perform one or more instructions. Hardware may include components of an integrated circuit or on-chip system, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a complex programmable logic device (CPLD), and other implementations in silicon or other hardware. In this context, hardware may operate as a processing device that performs program tasks defined by instructions and/or logic embodied by the hardware as well as a hardware utilized to store instructions for execution, e.g., the computer-readable storage media described previously.

Combinations of the foregoing may also be employed to implement various techniques described herein. Accordingly, software, hardware, or executable modules may be implemented as one or more instructions and/or logic embodied on some form of computer-readable storage media and/or by one or more hardware elements 1010. The computing device 1002 may be configured to implement particular instructions and/or functions corresponding to the software and/or hardware modules. Accordingly, implementation of a module that is executable by the computing device 1002 as software may be achieved at least partially in hardware, e.g., through use of computer-readable storage media and/or hardware elements 1010 of the processing system 1004. The instructions and/or functions may be executable/operable by one or more articles of manufacture (for example, one or more computing devices 1002 and/or processing systems 1004) to implement techniques, modules, and examples described herein.

CONCLUSION

Although the example implementations have been described in language specific to structural features and/or methodological acts, it is to be understood that the implementations defined in the appended claims is not necessarily limited to the specific features or acts described. Rather, the specific features and acts are disclosed as example forms of implementing the claimed features.

Claims

1. A method comprising: obtaining touch information regarding a single finger touching a keyboard of an input device, the input device supporting multiple different user intents for the single finger touching the keyboard, the obtaining comprising obtaining force information; andautomatically determining, based on the touch information, which of the multiple different user intents is intended for the single finger touching the keyboard, the multiple different user intents including a key strike, a mouse click, and a single finger motion, the determining including: determining that the user intent intended for the single finger touching the keyboard is the key strike in response to the touch information conforming to characteristics of the key strike,determining that the user intent intended for the single finger touching the keyboard is the mouse click in response to the touch information conforming to characteristics of the mouse click, the characteristics of the key strike indicating that the force applied by the single finger touching the keyboard rises faster than the characteristics of the mouse click indicate,determining that the user intent intended for the single finger touching the keyboard is the single finger motion in response to the touch information conforming to characteristics of the single finger motion.

2. The method as recited in claim 1, the input device comprising multiple pressure sensors that sense the single finger touching the keyboard, and the obtaining comprising obtaining force information.

3. The method as recited in claim 1, the input device comprising a capacitive system that senses the single finger touching the keyboard, and the obtaining comprising obtaining contact information.

4. The method as recited in claim 1, the determining that the user intent is the single finger motion comprising determining that the user intent is the single finger motion in response to the single finger travelling a threshold distance across the keyboard.

5. The method as recited in claim 1, the determining that the user intent is the single finger motion comprising determining that the user intent is the single finger motion in response to the single finger having a threshold velocity across the keyboard.

6. The method as recited in claim 1, the input device comprising an input device that is physically separate from and communicatively removable from a computing device.

7. The method as recited in claim 1, the determining comprising determining which of the multiple different user intents is intended by comparing the touch information to one or more graphs representing user input intents, and determining whether the key strike or the mouse click is intended based on an intent associated with the one of the one or more graphs with which the touch information conforms.

8. The method as recited in claim 1, the determining comprising determining which of the multiple different user intents is intended for the single finger touching the keyboard in the absence of another option or button selection by a user to indicate which of the multiple different user intents is intended.

9. The method as recited in claim 1, further comprising: using the touch information to identify a location of the keyboard touched by the single finger touching the keyboard; andrecording a user selection of a key associated with the location in response to determining that the user intent intended for the single finger touching the keyboard is the key strike, but not recording the user selection of the key associated with the location in response to determining that the user intent intended for the single finger touching the keyboard is the mouse click or the single finger motion.

10. An apparatus comprising an input device configured to provide an output that indicates touch information regarding a single finger touching a keyboard of the input device, the touch information including force information, the input device supporting multiple different user intents for the single finger touching the keyboard of the input device, the multiple different user intents including a key strike, a mouse click, and a single finger motion, the output being usable by one or more modules to automatically classify the user intent for the single finger touching the keyboard as the key strike in response to the touch information conforming to characteristics of the key strike, classify the user intent for the single finger touching the keyboard as the mouse click in response to the touch information conforming to characteristics of the mouse click, and classify the user intent for the single finger touching the keyboard as the single finger motion in response to the touch information conforming to characteristics of the single finger motion, the characteristics of the key strike indicating that the force applied by the single finger touching the keyboard rises faster than the characteristics of the mouse click indicate.

11. The apparatus as recited in claim 10, the input device comprising a pressure sensitive input device.

12. The apparatus as recited in claim 11, the touch information comprising information indicating changes in locations where pressure is applied to the keyboard of the pressure sensitive input device over time as well as changes in pressure applied to the keyboard over time.

13. The apparatus as recited in claim 10, the apparatus including the one or more modules.

14. The apparatus as recited in claim 10, the output being usable by the one or more modules to automatically classify the user intent for the single finger touching keyboard in the absence of another option or button selection by a user to indicate which of the multiple different user intents is intended.

15. The apparatus as recited in claim 10, the input device comprising an input device that is physically separate from and communicatively removable from a computing device.

16. The apparatus as recited in claim 10, further comprising: the output being usable by the one or more modules to record a user selection of a key associated with a location of the keyboard touched by the single finger in response to classifying the user intent for the single finger touching the keyboard as the key strike, but not record the user selection of the key associated with the location in response to classifying the user intent for the single finger touching the keyboard as the mouse click or the single finger motion.

17. An apparatus comprising: a hardware processor; andcomputer-readable storage devices having stored thereon multiple instructions that, responsive to execution by the processor, cause the processor to perform operations comprising: obtaining touch information regarding a single finger touching a keyboard of an input device, the touch information comprising force information, the input device supporting multiple different user intents for the single finger touching the keyboard; andautomatically determining, based on the touch information, which of the multiple different user intents is intended for the single finger touching the keyboard, the multiple different user intents including a key strike, a mouse click, and a single finger motion, and the determining including: determining that the user intent intended for the single finger touching the keyboard is the key strike in response to the touch information conforming to characteristics of a key strike,determining that the user intent intended for the single finger touching the keyboard is the mouse click in response to the touch information conforming to characteristics of the mouse click, the characteristics of the key strike indicating that the force applied by the single finger touching the keyboard rises faster than the characteristics of the mouse click indicate, anddetermining that the user intent intended for the single finger touching the keyboard is the single finger motion in response to the touch information conforming to characteristics of the single finger motion.

18. The apparatus as recited in claim 17, the apparatus including the input device.

19. The apparatus as recited in claim 17, the single finger touching the keyboard comprising a single touch user input, and the touch information comprising force information indicating pressure applied to the keyboard by the single touch user input.

20. The apparatus as recited in claim 17, the single finger touching the keyboard comprising a single touch user input, and the touch information comprising contact information indicating an area of the keyboard that is touched by the single touch user input.