Patent Application
20100265182
Input devices such as keyboards are often used with computers. Keyboards typically provide alpha-numeric inputs arranged in a familiar QWERTY pattern, a number pad, and/or function keys. Some keyboards include media buttons, volume controls, and/or quick launch buttons. In some cases the quick launch buttons may be assigned a user-specified functionality by opening a keyboard control program and associating a specific function with the quick launch button. One drawback with such an approach is that it is difficult for a user to ascertain the function of a user assignable key upon visual inspection, since the key itself it typically labeled with a non-descript label. In this case, the user relies upon his or her memory to recall the assigned function. Further, the position of these assignable keys is fixed, and the fixed position may not be suitable for the assignment of certain functions. As a result, assignable keys have not achieved universal popularity.
Various embodiments of systems and methods to implement a context-based state change for an adaptive input device are provided. For example, a method is disclosed that may include receiving a state input indicating a change in a system state, changing adaptive input device data in response to the state input, the adaptive input device data including one or more of image data or adaptive input device mapping data, and adjusting an adaptive input device display state using the adaptive input device data. Adjusting the adaptive input device display state may include one or more of displaying the image data on an adaptive input device display or adjusting an adaptive input device mapping state according to the adaptive input device mapping data.
This Summary is provided to introduce concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter. Furthermore, the claimed subject matter is not limited to implementations that solve any or all disadvantages noted in any part of this disclosure.
Returning to
Via either route, application 112 may send a message, an input, or other communication to the keyboard 150, which message includes the application state 114, or the associated keyboard state 116 or keyboard data 118. However, other embodiments may not be so limited and the application state 114 may be mapped to keyboard state 116 or keyboard data 118 and otherwise communicated to keyboard 150.
In operating system space 120, computing device 105 includes an adaptive keyboard application 130 storing a computing device state 132, a keyboard state 116 and keyboard data 118. Computing device state 132 may receive input from operating system components 127 which may further receive inputs from hardware 126 of computing device 105 or attached devices. Adaptive keyboard application 130 communicates with keyboard 150 through a private application programming interface 128 and bus driver 124. For example, adaptive keyboard application 130 may send a message, an input, or other communication to keyboard 150 that includes the computing device state 132, or the associated keyboard state 116 or keyboard data 118. However, other embodiments may not be so limited and the computing device state 132 may be mapped to keyboard state 116 or keyboard data 118 and otherwise communicated to keyboard 150.
Turning now to keyboard 150, which includes a controller 160 coupled with a plurality of keys 158 and a keyboard display 154 and including an adaptive keyboard program 162. In some embodiments keyboard 150 may include a touch sensor 152 to detect a change in user state. For example, touch sensor 152 may be an optical touch sensor configured to optically detect a user touch of a region of the keyboard, a capacitive touch sensor configured to detect an electrical change from a touch by a user, or an accelerometer touch sensor configured to detect movement of a key due to touch, etc. In some embodiments, keyboard 150 may include a key display 156 on one or more of the keys 158, or a display 155 on the keyboard, projected from the keyboard, attached to the keyboard, etc. Additionally, keyboard display 154 may include a plurality of displays, wherein adaptive keyboard program 162 may update each of the plurality of displays according to respective system state changes, user selected display content, system options, etc.
Keyboard 150 sends and receives communications from computing device 105 through bus driver 124 in the computing device. Adaptive keyboard program 162 further includes an adaptive keyboard module 180, a keyboard mapping state 164, a keyboard display state 166 and a user state service 170. User state service 170 further includes a user state 172 as sensed by touch sensor 152, and an associated keyboard state 116 and keyboard data 118.
Adaptive keyboard module 180 includes one or more inputs indicating a system state 182, including on or more of an application state 114, the computing device state 132 and/or a user state 172. Alternately, adaptive keyboard module 180 may receive an already associated keyboard state 116 and/or keyboard data 118, wherein the keyboard state 116 and keyboard data is associated with a system state by the adaptive keyboard application 130 running in the application space 110 or operating system space 120, as non-limiting examples.
In one embodiment, keyboard 150 may be configured to provide a context-based state change. For example, adaptive keyboard program 162 may be configured to receive a state input indicating a change in a system state and change keyboard data 118 in response to the state input. In an example embodiment, the keyboard data 118 includes one or more of image data or keyboard mapping data, and the controller is configured to adjust the keyboard display using the image data and further configured to adjust a keyboard mapping state according to the keyboard mapping data. Image data may be icons, menu items, etc. from application 112, operating system components 127, configuration options for hardware 126, etc.
In some examples, image data that may be displayed on one or more keys can come from an application 112 running on the computing device 105, can come from the operating system of the computing device 105, it could be saved in memory on the keyboard 150, etc. For example, to display image data on a specific key or to a specific display portion on the keyboard, the adaptive keyboard program 162 may receive keyboard state 116 from the adaptive keyboard application 130 running in the application 112 or the operating system space 120 on computing device 105, or from the user state service 170 on the keyboard 150, as will be explained in the following paragraphs in more detail.
In one example embodiment, the change in system state may be a change in application state 114 in computing device 105 in communication with the keyboard 150. For example, the keyboard 150 may be configured to receive the change in the application state 114 through a public application programming interface 122 and being further configured to display on the keyboard display 154 user options related to the change in application state 114.
In some embodiments, a change in application state 114 may be determined based on if the application 112 is an active application, whether the application 112 is operating in the computing background, whether there are different states within the application while the application 112 is active, etc. In one example, in a word processing program if text is selected, a change in application state 114 may be detected. In response to the change in application state 114, functionality may be displayed and exposed to a user off the keyboard through a keyboard display, key mapping, etc.
As an example, if a user in the word processing application had previously selected text and changed the text to blue text, the change in application state 114 may be detected when the text is selected and a keyboard display may show multiple colors to change the text to. In a particular example, the previous actions of the user may also be ranked according to frequency, a recent change, in relation to the change in application state 114, etc., and the keyboard display can order a list of options for the user to select. In this way, if the user most recently selected text and converted the text to blue text, but had previously selected the text and converted it to red text, a first option may be displayed of converting the text to blue text, a second option may be displayed to convert the text to red text, etc.
In another example change in application state 114, when a slide show presentation program is first launched, there may be a state where the program queries a user if they are creating a new slide show or if the user wants to open a recent slide show. If a user selects to open a recent slide show, the adaptive keyboard program 162 may detect this change in application state 114 and change the keyboard display 154 to show the ten most recent slide show presentations that have been opened. Furthermore, once a slide show is selected, the adaptive keyboard program 162 could detect another state change and display on the keyboard display 154 options within the opened slide show. For example, as a user navigates the opened slide show, or as a user selects an animation to include in the slide show, functionality associated with the navigation or the animation may be exposed to the user through the keyboard display 154 and the user could select the functionality through the keyboard without keystroke sequences that take the user into the menu options.
In another example embodiment, the change in system state may be a change in a computing device state 132 in communication with the keyboard 150. For example, the keyboard may be configured to receive the change in the computing device state 132 through a private application programming interface 128 and may be further configured to display on the keyboard display 154 user options related to the change in computing device state 132. For example, a change in a computing device state 132 may include a computer turning on, turning off, going to sleep, being placed in a standby state, turning on a screen saver, as non-limiting examples. In this way, a detectable change in a computing device state 132 may be displayed to a user through keyboard display 154 using adaptive keyboard program 162.
In some embodiments, the change in system state may be detected by touch sensor 152. For example, the change in system state may be a change in a user state 172, where the keyboard is configured to display on the keyboard display 154 user options related to the change in user state 172. For example, a change in a user state 172 may be detected when user touches the keyboard, when a keyboard is moved or picked up, when a user is approaching a keyboard, when a user presses one or more keys, or a key sequence, when a user interacts with an interactive display on the keyboard that is displaying one or more user options, etc. In this way, when a user state changes, the keyboard 150 may be configured to change a display state in response to the user state change.
In some embodiments, the keyboard 150 may be configured to change the keyboard mapping state 164 of the plurality of keys 158 in response to the state input indicating a change in application state 114, computing device state 132, or user state 172. As an example, adaptive keyboard program 162 may comprise a look up table (LUT) configured to map a key code from application 112 to a particular key and provide the key code in response to a subsequent push of the key. In a game application example, a “Q” button press on a keyboard may fire a weapon within the game. The adaptive keyboard program 162 may then be configured to communicate with the game application, map one or more key codes from the game application to respective keys, and in response to a button press on a mapped key, in this example a “Q”, the adaptive keyboard program 162 may send to the game the respective codes, in this example the key code for firing the weapon.
In some embodiments, adaptive keyboard program 162 may adjust a keyboard display or a keyboard mapping state in response to a combination of state changes. For example, a change in user state 172 can be detected when a user approaches the keyboard 150 while an attached computing device is in a locked state. In this way, the adaptive keyboard program can illuminate keyboard keys 158 when the user gets within range and also display instructions to the user to type in a password to unlock the computing device.
In an example use case scenario, display 310 may display a standby computing device state 132 as received from computing device 132 through a private application programming interface 128 and a bus driver 124. Then, display 310 may prompt a user for a password to unlock the computing device 132 if it is locked, as an example. As another example use case scenario, once the computer is unlocked, the display 320 may provide menu options for media player application as received through public application programming interface 122 and bus driver 124. In this way, display 320 may display the menu options for the media player for recently played audio files in response to sensing a user approaching the keyboard.
Continuing with the Figures,
Method 400 also comprises changing adaptive input device data in response to the state input, the adaptive input device data including one or more of image data or adaptive input device mapping data, as indicated in block 420. Next, method 400 comprises adjusting the adaptive input device display state using the adaptive input device data, wherein adjusting the adaptive input device display state includes one or more of displaying the image data on a adaptive input device display or adjusting a adaptive input device mapping state according to the adaptive input device mapping data, as indicated at 430. In some embodiments, the image data or adaptive input device mapping data may be received from a computing device, an application, and/or a user input, as shown in block 435.
In an example application state change embodiment, method 400 includes detecting a change in an application state, and generating the state input indicating a change in an application state. For example, the state input indicating a change in an application state may be received through an application programming interface, which may be public, of a computing device, as indicated in block 417. In another embodiment, the state input may be received from an application via an interprocess communication mechanism such as a named pipe, at an adaptive input device application, which in turn is configured to forward the state input via a bus driver to the adaptive input device. In this way, a keyboard may adjust a keyboard display, a key mapping, etc., in response to the change in application state.
In an example computing device state change embodiment, method 400 may further comprise detecting a change in a computing device state, and generating the state input indicating a change in computing device state. As an example, in response to receiving a state input indicating a change in a computing device state through an application programming interface, which may be private, as indicated in block 417, a keyboard may adjust a keyboard display, key mapping, etc., in response to one or more of a change in an operating system component in the computing device or a hardware component in the computing device.
In an example user state change embodiment, method 400 may further comprise detecting a change in a user state using a touch sensor on a keyboard, and generating the state input indicating a change in a user state. In some embodiments, method 400 may further comprise changing a keyboard mapping state in response to a change in application state, computing device state, or user state as illustrated in block 437.
As described above, the adaptive input device may include a keyboard, such as a mechanical keyboard with mechanically depressible keys, and/or a touch screen device configured to present a virtual keyboard. Other adaptive input devices are also contemplated, which include both touch sensors and displays as described above.
It will be appreciated that the embodiments described herein may be implemented, for example, via computer-executable instructions or code, such as programs, stored on a computer-readable storage medium and executed by a computing device. Generally, programs include routines, objects, components, data structures, and the like that perform particular tasks or implement particular abstract data types. As used herein, the term “program” may connote a single program or multiple programs acting in concert, and may be used to denote applications, services, or any other type or class of program. Likewise, the terms “computer” and “computing device” as used herein include any device that electronically executes one or more programs, including, but not limited to, a keyboard with computing functionality and other computer input devices.
It will further be understood that the configurations and/or approaches described herein are exemplary in nature, and that these specific embodiments or examples are not to be considered in a limiting sense, because numerous variations are possible. The specific routines or methods described herein may represent one or more of any number of processing strategies. As such, various acts illustrated may be performed in the sequence illustrated, in other sequences, in parallel, or in some cases omitted. Likewise, the order of any of the above-described processes is not necessarily required to achieve the features and/or results of the embodiments described herein, but is provided for ease of illustration and description.
The subject matter of the present disclosure includes all novel and nonobvious combinations and subcombinations of the various processes, systems and configurations, and other features, functions, acts, and/or properties disclosed herein, as well as any and all equivalents thereof.
