Patent Application
20110144996
The present invention is related generally to computing devices that recognize human speech and, more particularly, to semantic analysis of verbal expressions.
Computing devices, either working alone or linked together in a communication network, are becoming increasingly useful to their human users. In many areas, such as searching and information retrieval, computing devices have become so capable that they seem to be able to perform almost any task given to them. The use of this enormous capability is, however, often stymied by the inefficiency of the means available to present a task to the computing device. That is to say, the user interface for specifying a task is often a chief bottleneck for getting a computing device to perform that task.
The occasion sometimes arises where a user wants to direct a computing device to perform a number of similar tasks. For example, the user wishes to add a number of people to a list (e.g., a list of intended recipients of an e-mail). The user can simply add the people one by one, specifying a separate command to add each person. It would be more efficient for the user, however, if he could specify one command that adds all of the people to the list. For purposes of the present discussion, this one command is called a “multiple-goal” task.
Text interfaces are very common tools for directing a computing device to perform a task. Some text interfaces allow a user to specify a multiple-goal task. However, the user may have to correctly handle an intimidating amount of punctuation (e.g., parentheses) and operators to correctly specify the multiple-goal task.
Voice-search interfaces, which perform tasks by executing commands retrieved from task databases searched with voice queries, are becoming more common and are especially desirable when a user's hands or eyes are otherwise occupied and are therefore not available to manipulate a text interface. As just one example, voice-calling interfaces are most useful when the user is driving or when using a cellular telephone with a limited keypad. A voice-search interface gives the user more freedom to formulate his command by searching a task database by using a voice query. However, voice-search interfaces typically have some significant limitations in comparison with text interfaces, especially when presenting complicated tasks to a computing device. A user communicating through an existing voice interface would probably not be able to correctly specify a multiple-goal task at all, given the limitations of existing voice interfaces and the difficulty in correctly specifying an elaborately punctuated command without the benefit of visual feedback.
The above considerations, and others, are addressed by the present invention, which can be understood by referring to the specification, to the drawings, and to the claims. According to aspects of the present invention, a verbal expression is received from a user. The expression is parsed to determine whether or not it contains a multiple-goal command. Specifically, known techniques are applied to extract terms from the verbal expression. The extracted terms are assigned to categories such as “verb” or “noun.” The temporal distribution of the terms in the expression is also examined. If two or more terms are found in the parsed verbal expression that are in associated categories and that do not overlap one another temporally, then the confidence levels of these terms are compared. If the confidence levels are similar, then the terms may be parallel entries in the verbal expression and may represent multiple goals. If a multiple-goal command is found, then either the command is presented to the user for review and possible editing, or the command is simply executed.
In the present discussion, a “command” can be a command, a search query, or a response.
If the decision is made to present the parsed multiple-goal command to the user for review, then the presentation can be made via any appropriate interface including voice interfaces and text interfaces. Known methods for editing the presented command can be applied to correct any misunderstandings on the part of the parser.
While the appended claims set forth the features of the present invention with particularity, the invention, together with its objects and advantages, may be best understood from the following detailed description taken in conjunction with the accompanying drawings of which:
a and 2b are simplified schematics of a personal electronic device that can receive a verbal expression;
a and 3b together form a flowchart of an exemplary method for analyzing and processing a multiple-goal verbal expression; and
Turning to the drawings, wherein like reference numerals refer to like elements, the invention is illustrated as being implemented in a suitable environment. The following description is based on embodiments of the invention and should not be taken as limiting the invention with regard to alternative embodiments that are not explicitly described herein.
In
If the decision is made to present the parsed multiple-goal command to the user 102 for review, then the presentation can be made via any appropriate interface on the personal electronic device 104 including voice-search interfaces and text interfaces. Known methods for editing the presented command can be applied to correct any misunderstandings on the part of the parser.
a and 2b show a personal electronic device 104 (e.g., a cellular telephone, personal digital assistant, or personal computer) that incorporates an embodiment of the present invention.
The typical user interface of the personal electronic device 104 includes, in addition to the main display 200, a keypad 202 or other user-input devices.
b illustrates some of the more important internal components of the personal electronic device 104. The device 104 includes a communications transceiver 204, a processor 206, and a memory 208. A microphone 210 (or two) receives the verbal expression from the user 102. A speaker 212 is usually present.
a and 3b present an exemplary method embodying the present invention. The method begins at step 300 of
In step 302, the verbal expression 400 is analyzed using any of several possible techniques. For example, lattices of phonemes or words are generated from the waveform of the verbal expression 400, and the phoneme strings or words extracted from the lattices are used as search terms to search a task database containing text items such as commands and names. Because the phoneme lattices contain highly likely phoneme sequences (and the word lattices contain highly likely word sequences) generated from the waveform, allowances are made for variations in speech and for noise in the verbal expression 400, and this database searching may produce some uncertainties including multiple potential interpretations of one or more terms in the verbal expression 400. For purposes of the present discussion, assume that the extraction process produces the following list of possible matches in the database:
(1) Steve Nowlan
(2) Steve Jobs
(3) add
(4) Yan-Ming Cheng
(5) Jackie Chen
Thus, the extraction process confidently recognized the word “add” (that is, only one database match corresponds to this term in the verbal expression 400). However, the extraction process was not too sure about the two name strings, because the extraction process retrieved from the database two possible matches for each of the two name strings.
(In some embodiments, the processing of steps 302 through 316 may be done on a server 106 remote from the personal electronic device 104. The server 106 may have more processing capability than the device 104 and may thus be able to provide a better or a faster analysis.)
In step 304, a database is searched with the potential terms. The results of the search are ranked in terms of possibility of correctness. Some embodiments of the database contain “associational” rules for some of the terms. For example, the database could know that the command word “play” takes the names of media files as its only arguments. Thus, the command word “play” is associated with media-file names as arguments. If the play command were recognized in a verbal expression, then the database search would assign a very low rank to an interpretation of a human's name as an argument to that command.
In step 306 (which in some embodiments is performed in parallel with step 302), the temporal distribution of the potential terms ranked highest in the search results is analyzed. (Only the highest ranked terms are analyzed to prevent the analysis from being cluttered with a number of barely feasible terms.) Note that the analyzed temporal distribution includes all of the potential terms, including alternative database matches for a single term in the original verbal expression 400. The potential terms could overlap in time. In the current example, the analyzed temporal distribution includes all five potential terms listed above. Turning to the exemplary verbal expression 400 of
Step 308 assigns at least some of the potential terms to categories. Different embodiments may use different categories, and different verbal expressions may call for the use of different categories. As discussed above, the database of recognizable terms may also include a categorization of each term, or a different database may be consulted. As one example, the categories can represent parts of speech such as verbs (which may represent command terms), noun phrases (which may represent the arguments of a command), adjectives, logical conjunctions, etc. Noun phrases, for example, may also be sub-categorized based on their meaning: names of human beings, media-file names, etc. Following the example of
In step 310, a set of “logically parallel” potential terms is created. Potential terms in the verbal expression 400 are considered to be “logically parallel” if they (a) do not substantially overlap one another temporally and if they (b) were assigned to associated categories in step 308. In the example of
(Note that step 310 only takes place when the interpretation of the verbal expression 400 calls for it. If no set of logically parallel potential terms can be extracted from a verbal expression, then that expression is not considered to contain multiple goals.)
Step 312 assigns confidence levels to members of the set of logically parallel potential terms created in step 310. (For purposes of the present discussion, step 312 is shown in
In the example of
In step 314 of
To sum up the results of the analysis so far, the verbal expression 400 has been found to contain the command verb “add” 402 which takes in parallel (i.e., takes as multiple-goals) the two noun-phrase arguments 404 and 406. In the embodiment of
There are several possible reasons for presenting the extracted multiple-goal expression for review by the user 102 in step 318 rather than simply acting upon it. For example, the confidence level of the analysis as a whole may be somewhat low, and it makes sense to give the user 102 the opportunity to correct any interpretive mistakes. Also, the user 102 may not yet be confident with verbally expressing multiple-goal commands, so he may appreciate the safety net of a review.
The extracted multiple-goal expression can be presented to the user 102 in any of a number of ways. For example, the expression may be presented as a text string on the main display screen 200 of the personal electronic device 104. This has the advantage that punctuation can be added to clearly show the format of the multiple-goal expression. The user 102 can then use well known text-editing software to amend the expression as needed. Some users may rely on this editing and intentionally treat the spoken expression 400 as merely a draft that he will work into shape with the text editor.
In some embodiments, the extracted verbal expression can be spoken to the user 102 via the speaker 212 of the personal electronic device 104. Modulation of the spoken expression can be exaggerated to clearly delineate the multiple-goal nature of the extracted expression. Again, the user 102 may choose to edit the expression before allowing it to be performed.
In step 320, the multiple-goal command (as edited, if necessary, in step 318) is performed.
While the above analysis is somewhat intricate, the experience of the user 102 is quite straightforward. It is expected that embodiments of the present invention will allow the user 102 to become comfortable with phrasing and executing multiple-goal commands, thus enabling the user to become more efficient at list processing and at other multiple-goal tasks.
In view of the many possible embodiments to which the principles of the present invention may be applied, it should be recognized that the embodiments described herein with respect to the drawing figures are meant to be illustrative only and should not be taken as limiting the scope of the invention. For example, many useful techniques are available for extracting potential terms, for assigning those terms to categories, and for assigning confidence levels to the potential terms. Therefore, the invention as described herein contemplates all such embodiments as may come within the scope of the following claims and of equivalents thereof.
