Patent Application
20060218506
This invention relates generally to user interfaces for electronic devices, and more particularly to menu usage on a user interface, such as is found in a communication device for example.
Electronic systems and their control software can be very complex and therefore benefit from the use of menus to access functions that are not readily known to a particular user. For example, all types of computer software commonly use pull down menus to access various functions. In addition, automatic telephone answering and forwarding systems typically use a multilayered menu approach. Similarly, wireless communication systems, such as portable or mobile cellular telephones for example, have become more complex leading to the incorporation of menus on a user interface to enable a user to access the many available functions.
In these cases, systems may have become complex enough wherein a user will be unaware of all the possible functions available. Therefore, help menus are often provided on a user interface. A problem arises in those situations where users may not be able to focus their time and attention on a menu system, such as when driving a vehicle, wherein using a fully functioned help menu would only serve to distract the driver and the driver may miss information. Similarly, telephone users forced to proceed through long interactive system menus can become frustrated.
Further problems arise when the user interface is relying on a speech recognition system to input commands, as opposed to a keyboard or other means. In today's speech recognition systems, a user when unsure about the list of commands available to navigate the various system menus will invoke the help command. The context sensitive help system will then provide the user with a long help message describing the various functions and commands active at that level in the user interface. The major drawback of this approach is that the user may have to listen to a lengthy help message before being able to proceed with his intended transaction. This can cause the user to become frustrated and impatient with the system, with the induced stress potentially resulting in lower recognition performance and increased task completion time.
One possible solution to the problem is to automatically shorten menus depending upon a user's most often used “favorite” commands. However, this solution is not well suited to the case of help menus where a user is specifically looking for information on available commands (i.e. commands they would not be familiar with). In other words, a user would not be searching a help menu for commands they are already well versed with.
What is needed is a user interface with a menu system that can be automatically adapted, based on usage pattern, to provide efficient assistance and an enhanced user experience. In addition, it would be of benefit to accommodate different users and track how the menu system is used to allow for a dynamic adjustment of the presented information depending on the usage profile of each system user.
The features of the present invention, which are believed to be novel, are set forth with particularity in the appended claims. The invention, together with further objects and advantages thereof, may best be understood by making reference to the following description, taken in conjunction with the accompanying drawings, in the several figures of which like reference numerals identify identical elements, wherein:
The present invention provides an apparatus and method for adapting menus of a user interface in order to provide efficient assistance to meet a user's needs. Different users' habits can be accommodated and tracked to further assist users efficiently. Specifically, the present invention utilizes an adaptive help menu that capitalizes on the user's previous interaction pattern and experience with the system in order to provide a more fluid dialog with a voice activated system in a mobile environment.
The concept of the present invention can be advantageously used on any electronic device with a user interface that can interact with a user using visual, audio, voice, and text signals. In the example provided below, a wireless radio telephone is described using an audio and voice interface. Preferably, the radiotelephone portion of the communication device is a cellular radiotelephone adapted for mobile communication. However, the present invention is equally applicable to a pager, personal digital assistant, computer, cordless radiotelephone, portable cellular radiotelephone, or any other type of electronic or communication device that uses menus on a user interface. The radiotelephone portion in the example given generally includes an existing microphone, speaker, controller and memory that can be utilized in the implementation of the present invention. The electronics incorporated into a mobile cellular phone, are well known in the art, and can be incorporated into the communication device of the present invention. The user interface can include displays, keyboards, audio devices, video devices, and the like.
Many types of digital radio communication devices can use the present invention to advantage. By way of example only, the communication device is embodied in a mobile cellular phone, such as a Telematics unit, having a conventional cellular radiotelephone circuitry, as is known in the art, and will not be described in detail here for simplicity. The mobile telephone, includes conventional cellular phone hardware (also not represented for simplicity) such as processors and user interfaces that are integrated into the vehicle, and further includes memory, analog-to-digital converters and digital signal processors that can be utilized in the present invention. Each particular electronic device will offer opportunities for implementing this concept and the means selected for each application. It is envisioned that the present invention is best utilized in a vehicle with an automotive Telematics radio communication device, as is presented below, but it should be recognized that the present invention is equally applicable to home computers, portable communication devices, control devices, electronic devices, or other devices that have a user interface that utilize a menu system.
The memory 12 typically contains pre-stored menu items or entries characterizing each system function that a user can control 28 and, where appropriate, possible responses enabling for further visual or audio 46 interactions with a user. In the case of a user interface with a display, these menu entries can be text or graphics. In the case of a speech recognition system as in the present example, the pre-stored menu entries will be a set of grammars or rules that control the user's range of options at any point within the speech recognition user interface. Instead of a user pressing a button for placing a call, the user can instead invoke this action through a vocal command such as “dial”. The system responses (46) in this case will be in the form of audio feedback such as “To dial a telephone number, say ‘Dial Number’” or “Dialing 555-1212” that can be played back 40 over the loudspeaker 20 to a user to either prompt the user for input or to provide feedback to a user's speech input. Of course, corresponding visual or text menu responses can be easily substituted on the available user interface. The processor automatically creates a list of menu items 30 from the information in the memory 12, as will be described below.
Upon startup of the electronic device, the processor 10 is operable to create a list of menu items 30 from the memory 12. The user interface 16 is operable to output the list of menu items 30 and input menu selection information 42 from a user. A user can enter or speak a command, such as “Menu”, “Help”, or the like into the user interface 16 (e.g. microphone 22) of the electronic device 11. The microphone transduces the audio signal into an electrical signal. The user interface passes this signal 42 to the processor 10, and particularly an analog-to-digital converter 32, which converts the audio signal to a digital signal that can be used by the processor 10. Further processing can be done on the signal by (digital signal) processing to provide a data representation of the user interface entry, such as a data representation for use in a speech recognition system for example. A comparator 36 compares the data entry to the representations of the list of possible menu entries 28, which are associated to the allowable actions that are active under a given menu, and takes further action thereon.
Referring to
When a user begins to use a newly acquired electronic device, they will probably require some help in operating the device. Therefore, the full range of commands available for a given menu in the user interface will be provided in the corresponding menu, such as is shown in the Help menu of
To accomplish this, and referring back to
Afterwards, the next time the help menu is invoked, the corresponding menu and command statistics are examined from the usage profile 38 of that user from memory. The list of commands 28 associated with the help menu is checked against a predetermined limit to determine the number of times each command was successfully used and if the command was used during a predetermined time period. The most commonly used commands, for the specific menu, are removed from the help message (as demonstrated in
Of course, a user should always be able to obtain information about any command in a menu. Therefore, in the present invention the processor can create an optional menu, which when selected will reinstate any previously removed menu items from the help message. The optional menu item can be provided at the end of the list of menu items (of an adaptively abbreviated menu). In this way, the user is provided with the option to be presented with any removed commands should they need more information. For example, a “More Help” entry can be provided (see
Optionally, an added response 46 such as a user tip or advice can be provided in the menu if repeated failures are detected for completing an action associated with a particular menu item. In other words, if a particular user has selected the same command from the list of menu items a predetermined number of times and unsuccessfully completed that action then the processor can provide further assistance to the user on the user interface. For example, if a user is having problems in the “Dial Number” command stringing together a series of continuous digits in speech recognition mode, the system could ask if user would like advice. The advice could be to “Speak continuously without pausing or articulate in a normal voice.” Advice could be offered based upon collected success statistics in the usage profile 38.
Referring to
In the case of a regular (non-help) menu item, a next step 102 includes using an item from the menu by the user. This can include a user actually selecting the item from a menu, or just invoking the menu item through a voice command without referring to the menu. It is then determined if the task associated with the menu item was successfully accomplished 104. The method keeps track of how many unsuccessful attempts are made. If a user has not completed the task (e.g. successfully used the “Dial Number” command by placing a call) then it is assumed that the user has not learned the menu item. Therefore, unless the task is actually accomplished, this particular event will not be counted towards removal of that particular item from the help menu. For example, if a particular user has unsuccessfully used the same menu item with a voice command from the list of menu items more than a predetermined number of times 126 then the method includes a further step 130 of providing further assistance to the user on the user interface, whereupon the failure count is reset 132 giving the user another predetermined number of times to successfully accomplish a selected task. Otherwise, a task failure counter is incremented 128 and the process returns to the beginning, waiting for the next user input.
Returning to step 104, in the case of a regular (non-help) menu item, if a task is successfully completed, indicating a user's proficiency in invoking such menu item. This is noted by updating menu item statistics 106 for that particular user. The statistics include keeping a statistical usage profile of menu item utilization for particular users. The profile can include a count of how many times the user has successfully used the menu command and completed the intended task, and when the command was used. This statistical usage profile is accessed as part of the criteria 108 in deciding when to remove an item 110. This step 106 can also include the substep of recording a timestamp of when a menu item was removed from a menu.
If the help menu has not been invoked 108 to assist the user with the particular menu item selected, then it is clear that the user is becoming proficient in using the selected command and this menu item can be removed from 110 from the list after a certain number of successful uses 108. The criteria can include counting how many times the user has used the menu item from the list of menu items wherein if the user has successfully used the menu item a predetermined number of times then that selected item can be removed from the list of items in the corresponding help menu next time this menu is invoked. The criteria can also include counting how many times the user has used the menu item from the list of menu items wherein if the user has used the menu item within a predefined time period then that selected item can be removed from the list of items in the help menu. Either or both of these criteria can be used in deciding whether to remove a menu item from the menu.
Once an item has been removed, the providing step 100 can include providing an optional menu item to reinstate any previously removed menu items for presentation to the user. In this way a user may obtain help on using a menu item that they may have forgotten. Further steps can determine when a menu item was removed, wherein the removed item can be reinstated to the list of menu items if the removed menu item has not been used within a predetermined period of time. For example, in regards to a user invoking the help menu 101, it can be determined 112 whether a particular user has selected to optionally reinstate removed menu items in the provided menu list by having the user invoking an additional command, such as “More Help”. If the user asks for such additional assistance, the user will obtain 114 the additional listing of items that had been previously removed.
If an item has not been used recently 118 it can be assumed that a particular user may have become unfamiliar with the use of the menu item and that this item should be reinstated so that the user will not miss help information on this menu item if needed. Therefore, if a menu item has not been used recently 118 the timestamp in the usage statistics can be reset 120 for the menu item for this particular user and the menu item can be reinstated 122 to the help menu list. Thereafter, the menu task completion test can be acted upon 124. If the task is completed successfully then no further action is taken in terms of updating specific statistics as the user has just used the command based on information provided in the help menu and is therefore not familiar yet with this command. If the task is not completed successfully then this will also be counted in the task failure count 126 as explained previously.
Advantageously, the present invention results in improved user experience as it can track the familiarity of a user with a menu-driven speech recognition system over time. The main benefits are lowered user frustration and faster task completion rates, which are essential for eyes-busy, hands-busy environments such as when driving a vehicle. In this way, a driver's cognitive load is applied to the main task (i.e. driving a vehicle) and not on using a voice activated command system. The present invention can best be used for in-vehicle hands-free automatic speech recognition (ASR) systems or hand-held device based ASR.
While the present invention has been particularly shown and described with reference to particular embodiments thereof, it will be understood by those skilled in the art that various changes may be made and equivalents substituted for elements thereof without departing from the broad scope of the invention. In addition, many modifications may be made to adapt a particular situation or material to the teachings of the invention without departing from the essential scope thereof. Therefore, it is intended that the invention not be limited to the particular embodiments disclosed herein, but that the invention will include all embodiments falling within the scope of the appended claims.
