Patent Application
20100306683
The present invention relates generally to user interface behaviors for input devices, and more particularly to dynamic and individual control of backlighting for input elements within an input device.
Electronic devices are ubiquitous in society and can be found in everything from wristwatches to computers. While electronic devices such as computers operate in a world of ones and zeros, human beings do not. Thus, many computers include intermediary devices that allow human beings to interface to the computer. One such device is the keyboard where the user interfaces with the computer by pressing certain keys and then watching a display device connected to the computer to determine if the user's desired response was achieved.
While most conventional approaches utilize keyboards and other intermediary devices as strict input devices, some devices may convey output information to the user of the computer via the keyboard. For example, when a user presses the CAPS lock key, a light at the top of the keyboard (or on the CAPS lock key itself) may light up to indicate that such a selection has been made. Alternatively, some conventional approaches may provide a keyboard that associates lights with its keys where the keyboard may be statically configured at boot time.
Methods and apparatuses are disclosed that provide user interface behaviors for input devices with individually controlled illuminated input elements. Some embodiments may take the form of a method, including the operations of: receiving a request for input device lighting from a program executing on a computing device; determining illumination information for one or more light sources coupled to one or more input elements of the input device based on the request; and dynamically controlling illumination of the one or more light sources based on the determined illumination information. The illumination information may include brightness, color, and/or duration of the one or more light sources. The input device may constitute a keyboard with individually controlled illuminated keys.
In some embodiments, the illumination of the light sources may present information related to a program in operation on an associated computing device. For example, the light sources may function as a graphic equalizer on an input device or generally relate to at least a portion of an image on a display device. In other embodiments, the illumination of light sources may indicate or change functionality of a program utilizing the input elements. For example, input elements can be utilized to access functionality of a first-person shooter game. In still other embodiments, the illumination of the light sources may present a notification associated with the program such as an “E” key on a keyboard to indicate that an email has been received by an email application.
In some embodiments, the request for input device lighting from the program may be based on input selections received from the input device. For example, a program may receive keystrokes from a keyboard that constitute a portion of a word and the program may request input device lighting to illuminate keys on the keyboard that are possible next letters in the word. By way of another example, a program may receive a command key keystroke from a keyboard and the program may request input device lighting to illuminate keys on the keyboard that are associated with the command key.
The use of the same reference numerals in different drawings indicates similar or identical items.
Embodiments are disclosed that allow presentation of user interface behaviors on an input device with individually controlled illuminated input elements. Some embodiments may facilitate dynamically controlling illumination of light sources coupled to, or forming a part of, one or more input elements of an input device based on an illumination request from a program executing on a computing device. The program may include system software (such as an operating system) or application software. For example, the program may include an operating system, a word processing program, and/or a first-person shooter game. In some embodiments, the request for input device lighting from the program may be based on input element selections received from the input device.
Although one or more of these embodiments may be described in detail in the context of a computer system, the embodiments disclosed should not be interpreted as limiting, or otherwise used to limit the scope of the disclosure, including the claims. In addition, one skilled in the art will understand that the following description has broad application. Accordingly, the discussion of any embodiment is meant only to be exemplary and is not intended to intimate that the scope of the disclosure, including the claims, is limited to these embodiments.
Embodiments described herein may provide various user experiences by using a keyboard, or other input device, to provide output as well. The individual keys of the keyboard may each be separately illuminated across a range of colors and brightnesses. Depending on the application and/or operation requesting or controlling illumination of the keys, the color and/or brightness of each lit key may convey a variety of information.
For example, the computing system may provide reactive feedback by lighting keys, such as lighting the key that has been or is being pressed. The embodiment may provide predictive feedback, such as illuminating keys that are likely to be pressed given prior keystrokes, the application interfacing with the keyboard, an application currently in use by the user, prior user activities and so on. To provide a concrete example, a word processor may operate to control lighting of the keys on a keyboard such that the next most likely letter to be pressed is illuminated. The word processor may determine this by tracking the keys already pressed and employing a dictionary function. In some embodiments, the word processor may clear its memory and begin tracking keystrokes anew every time the space bar in pressed.
As another example, the system may employ output via the keyboard for purposes of training or education, such as teaching a specific sequence of keystrokes to a user by illuminating them in a pattern. The pattern may be temporal, such that each key in the sequence is illuminated in turn according to the sequence. The previously-illuminated keys may remain lit, may dim, or may be extinguished entirely. The pattern may be related to color, so that each key is illuminated in a shade of the same color, with the shades becoming darker as the sequence progresses. Alternately, the colors may vary with the progression of the sequence. As one illustration, the first key in the sequence may be red, the next orange, the third yellow, the fourth green, and so on such that the color of the keys ranges along the colors of the spectrum from red to indigo as the sequence progresses.
As yet another example, embodiments may create visual effects by lighting the various keys of the keyboard or elements of another output device. “Running lights” may be simulated, keys may have color and/or intensity changed as the embodiment plays music, and so on.
Still another function that may be achieved by embodiments separately illuminating keys of a keyboard, as well as separately adjusting the brightness and color of such keys, is functional key grouping. Keys having similar functions in an application may be colored similarly or may have similar brightness. Different functional key groups may have different colors and/or brightnesses. As a user switches from one application or program to another, the functional key groups may change and thus the color and/or brightness of each key may change.
The embodiment may also contextually change illumination based on the current state of an application, operating system or software. For example, in an image application, a first set of keys may be illuminated when the application is operating in an image management mode. The first set of keys generally provides management functionality, such as saving, resizing, tagging, and so forth. If the user switches the application to run in a second mode, such as a slideshow mode, the functionality of the keys on the keyboard may change. Thus, the application may request the keyboard extinguish backlighting for the first set of keys and illuminate a second set of keys corresponding to the functions available in the slideshow mode. Further, the context of the application function may affect the illumination brightness as well. As an example, keys may be more softly illuminated (or illuminated in a darker color) during a slideshow mode than during a management mode in order to reduce the likelihood of distraction. It should be understood that this contextual illumination may occur when any application experiences or provides a change in functionality. As yet another example, opening a menu on a word processing program may cause the keys corresponding to the functions of that menu to illuminate.
Depending upon the embodiment, the keyboard 125 and mouse 126 may take a variety of forms. For example, in some embodiments the keyboard 125 may be a 101-key US traditional keyboard configured to support the English language while the mouse may be a PS2 style. However, in alternative embodiments the keyboard 125 may be a 102/105-key International keyboard configured to support non-English languages. In still other embodiments, the keyboard 125 may provide multimedia support, with special keys for accessing and controlling multimedia files.
While
Some embodiments may implement the computing device 110 as a Mac™ manufactured by Apple Inc.™. For example, the computing device 110 may be a Mac Mini™ and the OS may be Mac OS™ version 10.
Of course alternative embodiments are possible where the computing system 100 is not a personal computer. For example, the computing system 100 may be a gaming system, such as the X-Box™ manufactured by Microsoft, Inc.™, the Playstation™ manufactured by Sony, Inc.™, and/or the Wii™ manufactured by Nintendo™. By way of another example, the computing system 100 may be a cellular telephone or personal digital assistant, such as the Razr V3™ manufactured by Motorola™ or the Bold™ manufactured by RIM™. As will be appreciated by one of skill in the art, the input devices, such as the keyboard 125 and mouse 126, may take a variety of forms depending upon the actual implementation of the computing system 100. For example, in embodiments where the computing system 100 is a gaming system, the input devices may be game controllers with dynamic illumination operations akin to the keyboard 125 and mouse 126 (which are described in further detail below).
In some embodiments, processor 310 is a microprocessor manufactured by Motorola™, such as the 680XX0 processor, or a microprocessor manufactured by Intel, such as the 80X86 (Pentium®), Core™ 2 Duo, or Xeon™ processors. Any other suitable microprocessor or microcomputer may be utilized, however.
Depending upon the embodiment, the bus 305 may take the form of multiple independent busses. For example, the bus 305 may couple a processor 310 to a Northbridge chip, where the Northbridge chip generally controls functions like the main memory 302 and video memory 300. Another bus 305 may couple the Northbridge chip to a Southbridge chip to control operation of the keyboard 125, mouse 126, and/or other peripheral devices.
During operation, code received by system 100 may be executed by the processor 310 as it is received, and/or stored in the mass storage 303, or other non-volatile storage for later execution. In this manner, the system 100 may access application programs or OS software stored in a variety of forms. Application programs may be embodied in any form of computer program product, such as a medium configured to store or transport computer readable code or data, or in which computer readable code or data may be embedded. Examples of computer program products include CD-ROM discs, ROM cards, floppy disks, magnetic tapes, computer hard drives, servers on a network, and solid state memory devices.
In a first operation 410, an input device lighting driver, executing on a computing device, receives a request for input device lighting from a program executing on the computing device. The program may include any program executing on the computing device, such as an application program and/or the OS. In some embodiments, the request for input device lighting may be based on input element selections received from the input device and provided to the program.
The program may request for input device lighting to accomplish a variety of purposes. These purposes may include, but are not limited to: displaying graphical information related to the program; displaying, mirroring or approximating, at least a portion of an image currently displayed on a display device coupled to the computing device; conveying information regarding functionality of the program related to input elements of the input device; conveying assistance regarding functionality of the program, conveying assistance regarding use of the program, conveying information related to the status of the program; presenting a notification associated with the program; and conveying information regarding certain functionality of input elements of the input device related to the program and already selected input elements of the input device (such as a command key).
In a second operation 420, the input device lighting driver determines illumination information for one or more light sources coupled to one or more input elements of an input device coupled to the computing device. The illumination information may include a brightness of the one or more light sources, a color of the one or more light sources, and/or a duration of the one or more light sources.
In a third operation 430, the input device lighting driver dynamically controls illumination of the one or more light sources based on the determined illumination information. For example, based on the illumination information determined in operation 420, the input device lighting driver may transmit a red-green-blue (RGB) value for the one or more light sources to the input device for a particular duration.
At least a portion of the OS running on the computing device 110 may include a keyboard driver 510 that handles the individual color control of the backlighting source for one or more keys of the keyboard 125. The keyboard driver 510 may receive a request from the OS or an application program to illuminate one or more keys of the keyboard 125 via the associated light source. For example, if the user types all but the last letter of a word in a word processing program, a spell checker function of the word processing program may request the keyboard driver 510 to light the most probable last letter of the word being typed by the user.
As shown in
Although Table 1 illustrates potential signals for two keys, the array generated by the keyboard driver 510 may contain many entries. For example, in the event that the keyboard 125 is a 101-key US traditional keyboard, then the array may contain 101 entries each having a brightness, color, and/or duration. Furthermore, although Table 1 illustrates potential color illumination signals, non-color illumination signals (e.g., luminance only) are also possible.
It should be noted that the duration parameter shown in Table 1 is provided for convenience and is not necessary to control operation of any light source (such as a backlight for a key). An embodiment may instead vary the duration of a light source's activation by adjusting the PWM duty cycle for the light source, or for individual components of the light source such as individual red, green and/or blue LEDs. For example, in order to determine which keys should be illuminated at any given time, the keyboard driver 510 may provide updated values for the red, green and/or blue PWM values at a rate of N times per second, where N is chosen such that the human eye cannot discern any flickering. Thus, in one embodiment, N is greater than or equal to 60 so that the operating rate of the LED(s) is 60 Hz or greater.
The backlight driver 520 may couple to a backlight controller 530. In some embodiments, the backlight controller 530 may exist as a discrete integrated circuit within the keyboard 125, such as in the form of a keyboard controller of the keyboard 125. In other embodiments, the backlight controller 530 may exist as firmware stored in a read only memory (ROM) within a lighting controller of the keyboard 125. Regardless of the implementation of the backlight controller 530, the backlight driver 520 may generate data signals for programming the backlight controller 530.
In some embodiments, the data signals generated by the backlight driver 520 may be in array form as shown in Table 2, which is akin to the array shown in Table 1, yet more rudimentary than the array of data shown in Table 1. The more rudimentary nature of the data signals in Table 2 may be beneficial, for example, in the embodiments where the driver 520 is less complex and unable to directly process the data of Table 1. Again, although Table 2 illustrates potential signals for but a few keys, the array generated by the keyboard driver 510 may contain many entries, such as when the keyboard 125 is a 101-key US traditional keyboard.
As shown in Table 2, each individual key may have customized RGB values, current levels, and/or firing durations each red, green, and/or blue LEDs of each key of the keyboard 125. Notably, these customized values may vary as the keyboard 125 is dynamically controlled based upon user inputs.
The keyboard 125 also may include firmware or circuitry capable of detecting keystrokes and conveying this information back to the computing device 110 to allow requests to illuminate one or more keys of the keyboard 125 to be based on detected keystrokes. As with Table 1, certain embodiments may not require a duration parameter.
Regardless of whether reporting occurs via firmware or via dedicated circuitry, the keyboard driver 510 described above also may process data reported from the firmware or circuitry and report depressed key sequences back to the OS or application programs running on the computing device 110. Reporting the depressed keys and/or key sequences back to the OS and/or application programs running on the computing device 110 may allow generation of requests to illuminate one or more keys of the keyboard 125 that are interactively based upon inputs by the user. Thus, in the event that the user's inputs result in a request to illuminate one or more keys of the keyboard 125, the OS or application program may request dynamic key lighting.
By way of an example, a application program executing on computing device 110 may request illumination of one or more keys of keyboard 125, as illustrated in
By way of another example, the OS or application executing on computing device 110 may request illumination of one or more keys of the keyboard 125 to present a display on the keyboard 125 that corresponds to at least a portion of the image the OS or application is displaying on the display device 105. The OS or application may generate a point-to-point mapping of the portion of the image the OS or application is displaying on the display device 105 to the keyboard 125. The OS or application may generate the point-to-point mapping by overlaying a grid on the image where portions of the grid, defined by grid vertices, correspond to keys of the keyboard 125. The OS or application may sample a value for the area of the portion of the image that maps to a particular key (such as a median value or an average value for that area) based on the overlaid grid. Thus, the OS or application may request illumination of one or more keys based on the values for the areas mapping to the one or more keys. This illumination request may include both color and brightness data, as generally detailed herein.
In a third example, a program executing on computing device 110 may request illumination of one or more keys to present a notification to a user. For example, the program may be an email application program. When the email application program receives an email, the email application may notify the user that an email has been received by, for example, selecting one or more keys to illuminate. Continuing the example, the email application program may select the “E” key to illuminate, as email begins with an “e.”
In other implementations of this example, the program may constitute a first-person shooter game. In the first-person shooter game, the user's in game character may die or become injured. When the user's in-game character dies or becomes injured, the first-person shooter game may request illumination of one or more keys to notify the user that their in-game character has dies or become injured. The first-person shooter game may select a sequence of keys to illuminate in red (such as first illuminating keys in the top row of the keyboard 125, then keys in the next row down, and then keys in the row below that until the keys in the bottom row of the keyboard 125 have been illuminated) to create a visual effect of blood running down the keyboard 125 to notify the user that their in-game character died or was injured. Although this example is discussed in the context of a first-person shooter game any kind of game, such as a role-playing game, could be utilized without departing from the scope of the present disclosure.
In a fourth example, a program executing on computing device 110 may request illumination of one or more keys to indicate functionality of the program associated with the one or more keys. In one or more embodiments of this example, the program may be a first-person shooter game or other kind of game. The first-person shooter game may select keys to illuminate that are operable to influence functions of the first-person shooter game. The keys may be set statically by the first-person shooter game or may be configured by a user (such as short cut keys configured by the user for functions in the first-person shooter game).
Alternatively, the keys associated with the functionality may change periodically and the first-person shooter game may request illumination of the one or more keys to indicate to the user which keys are currently associated with the functionality. Thus, the functionality is associated with the illumination rather than particular keys. The user is able to invoke a particular function by selecting a key that is illuminated in a way associated with a function (such as by color or brightness) rather than selecting a particular key.
In one or more other embodiments of this example, the program may be an image manipulation or classification program, such as Photoshop™. The photographic program may select keys to illuminate that are operable to influence functions of the photographic program. The photographic program may request to illuminate groups of keys associated with different groups of photographic program functions in different colors. The photographic program may determine groups of keys associated with different kinds of photographic program functions, such as image manipulation commands, storage access commands, and help commands. The photographic program may associate a color with each kind of photographic program function. The color associated with each kind of photographic program function may correspond to a color displayed for that kind of photographic program function in a drop down menu for the photographic program. The photographic program may then request to illuminate keys according to the color associated with the kind of photographic program function the keys are associated with. For example, the photographic program may request to illuminate keys associated with image manipulation commands in green, keys associated with storage access commands in blue, keys associated with help commands in red, and etc.
In a fifth example, as illustrated in
Further, the program may request to illuminate groups of keys associated with different groups of functions in the program when pressed simultaneously with the command keys in different colors. The program may determine groups of keys associated with different kinds of functions related to the command keys, such as storage access commands and text editing commands. The program may associate a color with each kind of function. The color associated with each kind of function may correspond to a color displayed for that kind of function in a drop down menu for the program. The program may then request to illuminate keys according to the color associated with the kind of function the keys are associated with. If the command key is the CTRL key, the program may request to illuminate the s key in blue to indicate the s key is associated with storage access commands and the a, z, x, c, v, u, and i keys in green to indicate the a, z, x, c, v, u, and i keys are associated with text editing commands.
In yet another example, the program may have been provided with one or more keystrokes received from keyboard 125 that correspond to a portion of a word. The program may analyze the portion of the word utilizing a database of words and determine possible words a user may be attempting to type based on the portion of the word. The program may compare the portion of the word to the possible words to determine what might be the next letter in the word the user may be attempting to type. The program may then select to illuminate keys corresponding to what would be the next letter.
The description above includes example systems, methods, techniques, instruction sequences, and/or computer program products that embody techniques of the present disclosure. However, it is understood that the described disclosure may be practiced without these specific details.
In the present disclosure, the methods disclosed may be implemented as sets of instructions or software readable by a device. Further, it is understood that the specific order or hierarchy of steps in the methods disclosed are examples of exemplary approaches. Based upon design preferences, it is understood that the specific order or hierarchy of steps in the method can be rearranged while remaining within the disclosed subject matter. The accompanying method claims present elements of the various steps in a sample order, and are not necessarily meant to be limited to the specific order or hierarchy presented.
The described disclosure may be provided as a computer program product, or software, that may include a machine-readable medium having stored thereon instructions, which may be used to program a computer system (or other electronic devices) to perform a process according to the present disclosure. A machine-readable medium includes any mechanism for storing information in a form (e.g., software, processing application) readably by a machine (e.g., a computer). The machine-readable medium may include, but is not limited to, magnetic storage medium (e.g., floppy diskette), optical storage medium (e.g., CD-ROM); magneto-optical storage medium, read only memory (ROM); random access memory (RAM); erasable programmable memory (e.g., EPROM and EEPROM); flash memory; or other types of medium suitable for storing electronic instructions.
It is believed that the present disclosure and many of its attendant advantages will be understood by the foregoing description, and it will be apparent that various changes may be made in the form, construction and arrangement of the components without departing from the disclosed subject matter or without sacrificing all of its material advantages. The form described is merely explanatory, and it is the intention of the following claims to encompass and include such changes.
While the present disclosure has been described with reference to various examples, it will be understood that these examples are illustrative and that the scope of the disclosure is not limited to them. Many variations, modifications, additions, and improvements are possible. More generally, examples in accordance with the present disclosure have been described in the context or particular embodiments. Functionality may be separated or combined in blocks differently in various embodiments of the disclosure or described with different terminology. These and other variations, modifications, additions, and improvements may fall within the scope of the disclosure as defined in the claims that follow.
The following related patent applications are hereby incorporated by reference in their entirety as if set forth fully herein: U.S. patent application Ser. No. ______ (Attorney Docket No. P7400US1 (191162/US)), titled “Light Source With Light Sensor” and filed concurrently herewith; U.S. patent application Ser. No. ______ (Attorney Docket No. P7332US1 (191159/US)), titled “Keyboard With Increased Control of Backlit Keys” and filed concurrently herewith; and U.S. patent application Ser. No. ______ (Attorney Docket No. P7402US1 (191158/US)), titled “White Point Adjustment For Multicolor Keyboard Backlight” and filed concurrently herewith.
