Patent Grant
11222626
The invention relates to a cooperative conversational model for a human to machine voice user interface.
Advances in technology, particularly within the convergence space, have resulted in an increase in demand for voice recognition software that can exploit technology in ways that are intuitive to humans. While communication between human beings is most often “cooperative,” in that information and/or context is shared to advance mutual conversational goals, existing Human-to-Machine interfaces fail to provide the same level of intuitive interaction. For example, each human participant in a conversation can contribute to an exchange for the benefit of the exchange. This is done through shared assumptions and expectations regarding various aspects of the conversation, such as the topic, participant knowledge about the topic, expectations of the other participant's knowledge about the topic, appropriate word usage for the topic and/or participants, conversational development based on previous utterances, the participants' tone or inflection, the quality and quantity of contribution expected from each participant, and many other factors. Participating in conversations that continually build and draw upon shared information is a natural and intuitive way for humans to converse.
In contrast, complex Human-to-Machine interfaces do not allow users to exploit technology in an intuitive way, which inhibits mass-market adoption for various technologies. Incorporating a speech interface helps to alleviate this burden by making interaction easier and faster, but existing speech interfaces (when they actually work) still require significant learning on the part of the user. That is, existing speech interfaces are unable to bridge the gap between archaic Human-to-Machine interfaces and conversational speech that would make interaction with systems feel normal. Users should be able to directly request what they want from a system in a normal, conversational fashion, without having to memorize exact words or phrases. Alternatively, when users are uncertain of particular needs, they should be able to engage the system in a productive, cooperative dialogue to resolve their requests. Instead, existing speech interfaces force users to dumb down their requests to match simple sets of instructions in simple languages in order to communicate requests in ways that systems can understand. Using existing speech interfaces, there is virtually no option for dialogue between the user and the system to satisfy mutual goals.
Therefore, existing systems lack a conversational speech model that can provide users with the ability to interact with systems in ways that are inherently intuitive to human beings. Existing systems suffer from these and other problems.
According to various embodiments and aspects of the invention, a cooperative conversational voice user interface may understand free form human utterances, freeing users from being restricted to a fixed set of commands and/or requests. Rather, users can engage in cooperative conversations with a machine to complete a request or series of requests using a natural, intuitive, free form manner of expression.
According to an aspect of the invention, an exemplary system architecture for implementing a cooperative conversational voice user interface is provided. The system may receive an input, which may include a human utterance received by an input device, where the utterance may include one or more requests. As used herein, an “utterance” may be words, syllables, phonemes, or any other audible sound made by a human being. As used herein, a “request” may be a command, directive, or other instruction for a device, computer, or other machine to retrieve information, perform a task, or take some other action. In one implementation, the input may be a multi-modal input, where at least part of the multi-modal input is an utterance. The utterance component of the input may be processed by a speech recognition engine (which may alternatively be referred to as an Automatic Speech Recognizer or ASR) to generate one or more preliminary interpretations of the utterance. The one or more preliminary interpretations may then be provided to a conversational speech engine for further processing, where the conversational speech engine may communicate with one or more databases to generate an adaptive conversational response, which may be returned to the user as an output. In one implementation, the output may be a multi-modal output. For example, the utterance may include a request to perform an action, and the output may include a conversational response reporting success or failure, as well as an execution of the action.
According to another aspect of the invention, an exemplary conversational speech engine may generate an adaptive conversational response to a request or series of requests. The conversational speech engine may include a free form voice search module that may understand an utterance made using typical, day-to-day language (i.e., in free form), and may account for variations in how humans normally speak, the vocabulary they use, and the conditions in which they speak. To account for intangible variables of human speech, the free form search module may include models of casual human speech. For example, in one implementation, the free form search module may understand specialized jargon and/or slang, tolerate variations in word order, and tolerate verbalized pauses or stuttered speech. For example, formalized English requests, where a verb precedes a noun, may be treated in an equivalent manner to requests where the noun precedes the verb. In another implementation, compound requests and/or compound tasks with multiple variables may be identified in a single utterance. By identifying all relevant information for completing one or more tasks from a single utterance, advantages may be provided over existing voice user interfaces, such as Command and Control systems that use verbal menus to restrict information that a person can provide at a given point. In another implementation, inferring intended requests from incomplete or ambiguous requests may provide a conversational feel. By modeling what contextual signifiers, qualifiers, or other information may be required to perform a task in an identified context, an adaptive response may be generated, such as prompting a user for missing contextual signifiers, qualifiers, or other information. In one implementation, the response may ask for missing information in a way that most restricts possible interpretations, and the response may be framed to establish a domain for a subsequent user utterance. In another implementation, common alternatives for nouns and verbs may be recognized to reflect variations in usage patterns according to various criteria. Thus, variations in expression may be supported because word order is unimportant or unanticipated, and nouns and/or verbs may be represented in different ways to give simplistic, yet representative, examples. In another implementation, requests may be inferred from contradictory or otherwise inaccurate information, such as when an utterance includes starts and stops, restarts, stutters, run-on sentences, or other imperfect speech. For example, a user may sometimes change their mind, and thus alter the request in mid-utterance, and the imperfect speech feature may nonetheless be able to infer a request based on models of human speech. For example, various models may indicate that a last criterion is most likely to be correct, or intonation, emphasis, stress, use of the word “not,” or other models may indicate which criterion is most likely to be correct.
According to another aspect of the invention, the conversational speech engine may include a noise tolerance module that may discard words or noise which has no meaning in a given context to reduce a likelihood of confusion. Moreover, the noise tolerance module may filter out environmental and non-human noise to further reduce a likelihood of confusion. In one implementation, the noise tolerance module may cooperate with other modules and features to filter out words that do not fit into an identified context. For example, the noise tolerance module may filter other human conversations and/or utterances within a range of one or more microphones. For example, a single device may include multiple microphones, or multiple devices may each include one or more microphones, and the noise tolerance module may collate inputs and cooperatively filter out sound by comparing a speech signal from the various microphones. The noise tolerance module may also filter out non-human environmental noise within range of the microphones, out-of-vocabulary words caused by speaker ambiguity or malapropisms, or other noise that may be unrelated to a target request. Performance benchmarks for the noise tolerance module may be defined by noise models based on human criteria. For example, if a driver of a car is 92% likely to be understood by a passenger when traveling at 65 miles-per-hour with windows cracked, then performance benchmarks for the noise tolerance module may have a similar performance under such conditions.
According to another aspect of the invention, the conversational speech engine may include a context determination process that determines one or more contexts for a request to establish meaning within a conversation. The one or more contexts may be determined by having one or more context domain agents compete to determine a most appropriate domain for a given utterance. Once a given domain agent “wins” the competition, the winning domain agent may be responsible for establishing or inferring further contexts and updating short-term and long-term shared knowledge. If there is a deadlock between context domain agents, an adaptive conversational response may prompt the user to assist in disambiguating between the deadlocked agents. Moreover, the context determination process may infer intended operations and/or context based on previous utterances and/or requests, whereas existing systems consider each utterance independently, potentially making the same errors over and over again. For example, if a given interpretation turns out to be incorrect, the incorrect interpretation may be removed as a potential interpretation from one or more grammars associated with the speech recognition engine and/or from possible interpretations determined by the conversational speech engine, thereby assuring that a mistake will not be repeated for an identical utterance.
The context determination process may provide advantages over existing voice user interfaces by continually updating one or more models of an existing context and establishing context as a by-product of a conversation, which cannot be established a priori. Rather, the context determination process may track conversation topics and attempt to fit a current utterance into recent contexts, including switching between contexts as tasks are completed, partially completed, requested, etc. The context determination process may identify one or more context domains for an utterance by defining a collection of related functions that may be useful for users in various context domains. Moreover, each context domain may have relevant vocabularies and thought collections to model word groupings, which when evaluated together, may disambiguate one context domain from another. Thus, eliminating out-of-context words and noise words when searching for relevant combinations may enhance accuracy of inferences. This provides advantages over existing systems that attempt to assign meaning to every component of an utterance (i.e., including out-of-context words and noise words), which results in nearly infinite possible combinations and greater likelihood of confusion. The context determination process may also be self-aware, assigning degrees of certainty to one or more generated hypotheses, where a hypothesis may be developed to account for variations in environmental conditions, speaker ambiguity, accents, or other factors. By identifying a context, capabilities within the context, vocabularies within the context, what tasks are done most often historically in the context, what task was just completed, etc., the context determination process may establish intent from rather meager phonetic clues. Moreover, just as in human-to-human conversation, users may switch contexts at any time without confusion, enabling various context domains to be rapidly selected, without menu-driven dead ends, when an utterance is unambiguous.
According to another aspect of the invention, an exemplary cooperative conversational model may build upon free form voice search, noise tolerance, and context determination to implement a conversational Human-to-Machine interface that reflects human interaction and normal conversational behavior. That is, the cooperative conversational model enables humans and machines to participant in a conversation with an accepted purpose or direction, with each participant contributing to the conversation for the benefit of the conversation. By taking advantage of human presumptions about utterances that humans rely upon, both as speakers and listeners, a Human-to-Machine interface may be analogous to everyday human-to-human conversation. In one implementation, the exemplary cooperative conversation model may take incoming data (shared knowledge) to inform a decision (intelligent hypothesis building), and then may refine the decision and generate a response (adaptive response building).
According to another aspect of the invention, shared knowledge may include both short-term and long-term knowledge. Short-term knowledge may accumulate during a single conversation, where input received during a single conversation may be retained. The shared knowledge may include cross-modality awareness, where in addition to accumulating input relating to user utterances, requests, locations, etc., the shared knowledge may accumulate a current user interface state relating to other modal inputs to further build shared knowledge models. The shared knowledge may be used to build one or more intelligent hypotheses using current and relevant information, build long-term shared knowledge by identifying information with long-term significance, and generate adaptive responses with relevant state and word usage information. Moreover, because cooperative conversations model human conversations, short-term session data may be expired after a psychologically appropriate amount of time, thereby humanizing system behavior, reducing a likelihood of contextual confusion based on stale data, while also adding relevant information from an expired session context to long-term knowledge models. Long-term shared knowledge may generally be user-centric, rather than session-based, where inputs may be accumulated over time to build user, environmental, cognitive, historical, or other long-term knowledge models. Long-term and short-term shared knowledge may be used simultaneously anytime a user engages in a cooperative conversation. Long-term shared knowledge may include explicit and/or implicit user preferences, a history of recent contexts, requests, tasks, etc., user-specific jargon related to vocabularies and/or capabilities of a context, most often used word choices, or other information. The long-term shared knowledge may be used to build one or more intelligent hypotheses using current and relevant information, generate adaptive responses with appropriate word choices when unavailable via short-term shared knowledge, refine long-term shared knowledge models, identify a frequency of specific tasks, identify tasks a user frequently has difficulty with, or provide other information and/or analysis to generate more accurate conversational responses. Shared knowledge may also be used to adapt a level of unprompted support (e.g., for novices versus experienced users, users who are frequently misrecognized, etc.) Thus, shared knowledge may enable a user and a voice user interface to share assumptions and expectations such as topic knowledge, conversation history, word usage, jargon, tone, or other assumptions and/or expectations that facilitate a cooperative conversation between human users and a system.
According to another aspect of the invention, a conversation type may be identified for any given utterance. Categorizing and developing conceptual models for various types of exchanges may consistently align user expectations and domain capabilities. One or more intelligent hypotheses may be generated as to a conversation type by considering conversational goals, participant roles, and/or an allocation of information among the participants. Based on the conversational goals, participant roles, and allocation of information, the intelligent hypotheses may consider various factors to classify a conversation (or utterance) into general types of conversations that can interact with one another to form many more variations and permutations of conversation types (e.g., a conversation type may change dynamically as information is reallocated from one participant to another, or as conversational goals change based on the reallocation of information).
According to another aspect of the invention, the intelligent hypotheses may include one or more hypotheses of a user's intent in an utterance. In addition, the intelligent hypotheses may use short-term and/or long-term shared knowledge to proactively build and evaluate interaction with a user as a conversation progresses or over time. The hypotheses may model human-to-human interaction to include a varying degree of certainty for each hypothesis. That is, just as humans rely on knowledge shared by participants to examine how much and what kind of information was available, the intelligent hypotheses may leverage the identified conversation type and shared knowledge to generate a degree of certainty for each hypothesis.
According to another aspect of the invention, syntactically, grammatically, and contextually sensitive “intelligent responses” may be generated from the intelligent hypotheses that can be used to generate a conversational experience for a user, while also guiding the user to reply in a manner favorable for recognition. The intelligent responses may create a conversational feel by adapting to a user's manner of speaking, framing responses appropriately, and having natural variation and/or personality (e.g., by varying tone, pace, timing, inflection, word use, jargon, and other variables in a verbal or audible response).
According to another aspect of the invention, the intelligent responses may adapt to a user's manner of speaking by using contextual signifiers and grammatical rules to generate one or more sentences that may cooperate with the user. By taking advantage of shared knowledge about how a user utters a request, the responses may be modeled using similar techniques used to recognize requests. The intelligent responses may rate possible responses statistically and/or randomize responses, which creates an opportunity to build an exchange with natural variation and conversational feel. This provides advantages over existing voice user interfaces where input and output is incongruous, as the input is “conversational” and the output is “computerese.”
According to another aspect of the invention, the intelligent responses may frame responses to influence a user reply utterance for easy recognition. For example, the responses may be modeled to illicit utterances from the user that may be more likely to result in a completed request. Thus, the responses may conform to a cooperative nature of human dialog and a natural human tendency to “parrot” what was just heard as part of a next utterance. Moreover, knowledge of current context may enhance responses to generate more meaningful conversational responses. Framing the responses may also deal with misrecognitions according to human models. For example, humans frequently remember a number of recent utterances, especially when one or more previous utterances were misrecognized or unrecognized. Another participant in the conversation may limit correction to a part of the utterance that was misrecognized or unrecognized, or over subsequent utterances and/or other interactions, clues may be provided to indicate the initial interpretation was incorrect. Thus, by storing and analyzing multiple utterances, utterances from earlier in a conversation may be corrected as the conversation progresses.
According to another aspect of the invention, the intelligent responses may include multi-modal, or cross-modal, responses to a user. In one implementation, responses may be aware of and control one or more devices and/or interfaces, and users may respond by using whichever input method, or combination of input methods, is most convenient.
According to another aspect of the invention, the intelligent responses may correct a course of a conversation without interrupting conversational flow. That is, even though the intelligent responses may be reasonably “sure,” the intelligent responses may nonetheless sometimes be incorrect. While existing voice user interfaces tend to fail on average conversational missteps, normal human interactions may expect missteps and deal with them appropriately. Thus, responses after misrecognitions may be modeled after clarifications, rather than errors, and words may be chosen in subsequent responses to move conversation forward and establish an appropriate domain to be explored with the user.
Other objects and advantages of the invention will be apparent to those skilled in the art based on the following drawings and detailed description.
Referring to
The utterance component of input 105 may be processed by a speech recognition engine 110 (which may alternatively be referred to herein as Automatic Speech Recognizer 110, or as shown in
Referring to
According to one aspect of the invention, free form voice search module 245 may understand an utterance made using typical, day-to-day language (i.e., in free form), and may account for variations in how humans normally speak, the vocabulary they use, and the conditions in which they speak. Because variables such as stress, distraction, and serendipity are always different and infinitely varied, free form search module 245 may be designed with a goal of understanding that no human will come to the same Human-to-Machine interface situation in the same way twice. Thus, free form search module 245 may implement one or more features that model casual human speech. In various implementations, free form search module 245 may include, among other things, a free form utterance feature, a one-step access feature, an inferencing intended operations feature, an alternative expression feature, and/or an imperfect speech feature.
The free form utterance feature may understand specialized jargon and/or slang, tolerate variations in word order (e.g., whether a subject of a request comes before or after a verb may be irrelevant), and tolerate verbalized pauses (e.g., “um,” “ah,” “eh,” and other utterances without meaning). For example, the free form utterance feature may treat formalized English verb-before-noun requests in an equivalent manner to free form requests where a noun may precede a verb. For example, user utterances of “Change it to the Squizz” and “You know, um, that Squizz channel, ah, switch it there” may be treated equivalently (where Squizz is a channel on XM Satellite Radio). In either case, the free form utterance feature is able to identify “Squizz” as a subject of the utterance and “Change it” or “switch it” as a verb or request for the utterance (e.g., by cooperating with context determination process 255, or other features, and identifying a relevant context domain agent 230 and/or vocabulary 235 to interpret the utterance).
The one-step access feature may understand utterances that include compound requests with multiple variables. For example, a user utterance may be “What is the forecast for Boston this weekend?” The one-step access feature may identify “weather” as a context (e.g., by cooperating with context determination process 255, or other features, and identifying “forecast” as a synonym of “weather”), and search for a city equal to “Boston” and a time equal to “weekend.” By identifying all relevant information for completing a task from a single utterance, the one-step access feature may overcome drawbacks of existing voice user interfaces, such as Command and Control systems that use verbal menus to restrict information that a person can provide at a given point (e.g., a Command and Control system for a phone directory service may say: “State please,” . . . “City please,” . . . “What listing,” etc.). Moreover, some utterances may include compound requests, and the one-step access feature may decompose the compound requests into sub-tasks. For example, a user utterance of “I need to be at a meeting tomorrow in San Francisco at 8:00 am” may be decomposed into a set of sub-tasks such as (1) checking availability and reserving a flight on an evening before the meeting, (2) checking availability and reserving a hotel, (3) checking availability and reserving a car, etc., where users may further designate preferences for various tasks (e.g., first check availability on an airline for which the user is a frequent flyer). Depending on a level of shared knowledge about a user's preferences and/or historical patterns, the one-step access feature may infer additional tasks from a request. For example, in the above example, the one-step access feature may also check a weather forecast, and if the weather is “nice” (as defined by the user preferences and/or as inferred from historical patterns), the one-step access feature may schedule a tee-time at a preferred golf course in San Francisco.
The inferencing intended operations feature may identify an intended request from incomplete or ambiguous requests. For example, when a user utters “Route <indecipherable> Chicago <indecipherable> here,” where the user intended to say “Route calculation to Chicago from here,” the inferencing intended operations feature may model what is required to calculate a route (an origination point and a destination point). Because the utterance includes the origination point and the destination point, a request to calculate a route from the user's present location to Chicago may be inferred. Similarly, when the inferencing intended operations feature does not have sufficient information to infer a complete request, an adaptive conversational response may be generated to prompt the user for missing information. For example, when an utterance includes a request for a stock quote but not a company name (e.g., “Get me the stock price for <indecipherable>”), the response may be “What company's stock quote do you want?” The user may then provide an utterance including the company name, and the request may be completed. In one implementation, the response may ask for missing information in a way that most restricts possible interpretations (e.g., in a request for a task that requires both a city and a state, the state may be asked for first because there are fewer states than cities). Moreover, the inferencing intended operations feature may model compound tasks and/or requests by maintaining context and identifying relevant and/or missing information at both a composite and sub-task level.
The alternative expression feature may recognize common alternatives for nouns and verbs to reflect variations in usage patterns according to various criteria. For example, users may vary expression based on age, socio-economics, ethnicity, user whims, or other factors. Thus, the alternative expression feature may support variations in expression where word order is unimportant or unanticipated. Alternatives in expression based on various criteria or demographics may be loaded into context domain agents 230 and/or vocabularies 235, and the alternative expression feature may update context domain agents 230 and/or vocabularies 235 based on inferred or newly discovered variations. In one implementation, conversational speech engine 215 may include a subscription interface to update changes to context domain agents 230 and/or vocabularies 235 (e.g., a repository may aggregate various user utterances and deploy updates system wide). In operation, the alternative expression feature may allow nouns and/or verbs to be represented in different ways to give simplistic, yet representative, examples. For example, a user interested in a weather forecast for Washington, D.C. may provide any of the following utterances, each of which are interpreted equivalently: “What's the weather like in DC,” “Is it raining inside the Beltway,” Gimme the forecast for the capital,” etc. Similarly, utterances of “Go to my home,” “Go home,” “Show route to home,” and “I would like to know my way home” may all be interpreted equivalently, where a user profile may include the user's home address and a navigation route to the home address may be calculated.
The imperfect speech feature may be able to infer requests from contradictory or otherwise inaccurate information, such as when an utterance includes starts and stops, restarts, stutters, run-on sentences, or other imperfect speech. For example, a user may sometimes change their mind, and thus alter the request in mid-utterance, and the imperfect speech feature may nonetheless be able to infer a request based on models of human speech. For example, for an utterance of “Well, I wanna . . . Mexi . . . no, steak restaurant please, I'm hungry,” existing voice user interfaces make no assumptions regarding models of human speech and would be unable to infer whether the user wanted a Mexican or steak restaurant. The imperfect speech feature overcomes these drawbacks by using various models of human understanding that may indicate that a last criterion is most likely to be correct, or intonation, emphasis, stress, use of the word “not,” or other models may indicate which criterion is most likely to be correct. Thus, in the above example, the imperfect speech feature may infer that the user wants a steak restaurant.
According to one aspect of the invention, noise tolerance module 250 may be closely related to the imperfect speech feature, and may operate to discard words or noise that has no meaning in a given context so as not to create confusion. Moreover, noise tolerance module 250 may filter out environmental and non-human noise to further reduce a likelihood of confusion. In one implementation, noise tolerance module 250 may cooperate with other modules and features to filter out words that do not fit into a context. For example, one or more contexts may be identified, and words that have no meaning with respect to system capabilities, random human utterances without meaning and other noise may be filtered out. Thus, noise tolerance module 250 may model real-world conditions to identify meaningful requests. For example, noise tolerance module 250 may filter other human conversations and/or utterances within a range of one or more microphones, For example, a single device may include multiple microphones, or multiple devices may each include one or more microphones, and the noise tolerance module may collate inputs and cooperatively filter out sound by comparing a speech signal from the various microphones. Noise tolerance module 250 may also filter out non-human environmental noise within the range of the microphones, out-of-vocabulary words, which could be a result of speaker ambiguity or malapropisms, or other noise that may be unrelated to a target request. Noise models in noise tolerance module 250 may define performance benchmarks based on human criteria. For example, if a driver of a car, traveling at 65 miles-per-hour, with windows cracked is 92% likely to be understood by a passenger, then noise tolerance module 250 may have a similar performance under those conditions.
According to one aspect of the invention, conversational speech engine 215 may include a context determination process 255 that determines one or more contexts for a request to establish meaning within a conversation. The one or more contexts may be determined by having one or more context domain agents compete to determine a most appropriate domain for a given utterance, as described in U.S. patent application Ser. No. 11/197,504, entitled “Systems and Methods for Responding to Natural Language Speech Utterance,” filed Aug. 5, 2005, which issued as U.S. Pat. No. 7,640,160 on Dec. 29, 2009 and U.S. patent application Ser. No. 11/212,693, entitled “Mobile Systems and Methods of Supporting Natural Language Human-Machine Interactions,” filed Aug. 29, 2005, which issued as U.S. Pat. No. 7,949,529 on May 24, 2011, both of which are hereby incorporated by reference in their entirety. Once a given context domain agent “wins” the competition, the winning agent may be responsible for establishing or inferring further contexts and updating short-term and long-term shared knowledge. If there is a deadlock between context domain agents, an adaptive conversational response may prompt the user to assist in disambiguating between the deadlocked agents. For example, a user utterance of “What about traffic?” may have a distinct meaning in various contexts. That is, “traffic” may have a first meaning when the user is querying a system's media player (i.e., “traffic” would be a Rock and Roll band led by singer/songwriter Steve Winwood), a second meaning when the user is querying a search interface regarding Michael Douglas films (i.e., “traffic” would be a film directed by Steven Soderbergh), a third meaning when the user is querying a navigation device for directions to an airport (i.e., “traffic” would be related to conditions on roads along a route to the airport).
Moreover, context determination process 255 may infer intended operations and/or context based on previous utterances and/or requests, whereas existing systems consider each utterance independently, potentially making the same errors over and over again. For example, if a given interpretation turns out to be incorrect, the incorrect interpretation may be removed as a potential interpretation from one or more grammars associated with the speech recognition engine and/or from possible subsequent interpretations determined by context determination process 255, thereby assuring that a mistake will not be repeated for an identical utterance.
Context determination process 255 may overcome drawbacks of existing systems by continually updating one or more models of an existing context, where establishing context may be a by-product of a conversation, which cannot be established a priori. Context determination process 255 may establish a first context domain, change to a second context domain, change back to the first context domain, and so on, as tasks are completed, partially completed, requested, etc., and a context stack may track conversation topics and attempt to fit a current utterance into a most-recent context, next-most-recent topic, etc., traversing the context stack until a most likely intent can be established. For example, a user may utter “What's the traffic report,” and context determination process 255 may establish Traffic as a context, and return an output including a traffic report, which does not happen to mention traffic on Interstate-5. The user may then utter “What about I-5?” and context determination process 255 may know that the current context is Traffic, a traffic report including information about Interstate-5 may be searched for, and the traffic report indicating that Interstate-5 is crowded may be returned as an output. The user may then utter “Is there a faster way?” and context determination module 255 may know that the current context is still Traffic, and may search for routes to a specified destination with light traffic and avoiding Interstate-5. Moreover, context determination process 255 may build context based on user profiles, environmental profiles, historical profiles, or other information to further refine the context. For example, the profiles may indicate that Interstate-5 is a typical route taken Monday through Friday.
The profiles may be particularly meaningful when attempting to disambiguate between contexts where a word has different meanings in different contexts. For example, a user may utter “What's the weather in Seattle?” and context determination process 255 may establish Weather as a context, as well as establishing Seattle as an environmental context. The user may then utter “and Portland?” and context determination process 255 may return a weather report for Portland, Oreg. based on the Weather and an environmental proximity between Portland, Oreg. and Seattle, Wash. The user may then ask “What time does the game start?” and a search for sports events with teams from Seattle and/or Portland may occur, with results presented conversationally according to methods described in greater detail below in
Context determination process 255 may cooperate with context domain agents 230, where each context domain agent 230 may define a collection of related functions that may be useful for users. Moreover, each context domain agent 230 may include a relevant vocabulary 235 and thought collections that model word groupings, which when evaluated together, may disambiguate one context domain from another (e.g., a Music context domain agent 230 may include a vocabulary 235 for songs, artists, albums, etc., whereas a Stock context domain agent 230 may include a vocabulary 235 for company names, ticker symbols, financial metrics, etc.). Thus, accuracy in identifying meaning may be enhanced by eliminating out-of-context words and noise words when searching for relevant combinations. In contrast, existing systems attempt to assign meaning to every component of an utterance (e.g., including out-of-context words and noise words), which results in nearly infinite possible combinations and greater likelihood of confusion. Moreover, context domain agents 230 may include metadata for each criteria to further assist in interpreting utterances, inferring intent, completing incomplete requests, etc. (e.g., a Space Needle vocabulary word may include metadata for Seattle, landmark, tourism, Sky City restaurant, etc.). Given a disambiguated criterion, context determination process 255 may thus be able to automatically determine other information needed to complete a request, discard importance of word order, and perform other enhancements for conversational speech.
Context domain agents 230 may also be self-aware, assigning degrees of certainty to one or more generated hypotheses, where a hypothesis may be developed to account for variations in environmental conditions, speaker ambiguity, accents, or other factors. Conceptually, context domain agents 230 may be designed to model utterances as a hard-of-hearing person would at a noisy party. By identifying a context, capabilities within the context, vocabularies within the context, what tasks are done most often historically in the context, what task was just completed, etc., a context domain agent 230 may establish intent from rather meager phonetic clues. Moreover, the context stack may be one of a plurality of components for establishing context, and thus not a constraint upon the user. All context domains may be accessible, allowing the user to switch contexts at any time without confusion. Thus, just as in human-to-human conversation, context domains may be rapidly selected, without menu-driven dead ends, when an utterance is unambiguous. For example, a user may utter, “Please call Rich Kennewick on his cell phone,” and a system response of “Do you wish me to call Rich Kennewick on his cell?” may be generated. The user may decide at that point to call Rich Kennewick later, and instead, listen to some music. Thus, the user may then utter, “No, play the Louis Armstrong version of Body and Soul from my iPod,” and a system response of “Playing Body and Soul by Louis Armstrong” may be generated as Body and Soul is played through a media player. In this example, the later utterance has no contextual connection to the first utterance, yet because request criteria in the utterances are unambiguous, contexts can be switched easily without relying on the context stack.
Referring to
According to one aspect of the invention, shared knowledge 305 includes both short-term and long-term knowledge about incoming data. Short-term knowledge may accumulate during a single conversation, while long-term knowledge may accumulate over time to build user profiles, environmental profiles, historical profiles, cognitive profiles, etc.
Input received during a single conversation may be retained in a Session Input Accumulator. The Session Input Accumulator may include cross-modality awareness, where in addition to accumulating input relating to user utterances, requests, locations, etc., the Session Input Accumulator may accumulate a current user interface state relating to other modal inputs to further build shared knowledge models and more accurate adaptive responses (e.g., when a user utters a request relating to a portion of a touch-screen device, as described above). For example, the Session Input Accumulator may accumulate inputs including recognition text for each utterance, a recorded speech file for each utterance, a list-item selection history, a graphical user interface manipulation history, or other input data. Thus, the Session Input Accumulator may populate Intelligent Hypothesis Builder 310 with current and relevant information, build long-term shared knowledge by identifying information with long-term significance, provide Adaptive Response Builder 315 with relevant state and word usage information, retain recent contexts for use with Intelligent Hypothesis Builder 310, and/or retain utterances for reprocessing during multi-pass evaluations. Moreover, because cooperative conversations 300 model human conversations, short-term session data may be expired after a psychologically appropriate amount of time, thereby humanizing system behavior. For example, a human is unlikely to recall a context of a conversation from two years ago, but because the context would be identifiable by a machine, session context is expired after a predetermined amount of time to reduce a likelihood of contextual confusion based on stale data. However, relevant information from an expired session context may nonetheless be added to user, historical, environmental, cognitive, or other long-term knowledge models.
Long-term shared knowledge may generally be user-centric, rather than session-based. That is, inputs may be accumulated over time to build user, environmental, cognitive, historical, or other long-term knowledge models. Long-term and short-term shared knowledge (collectively, shared knowledge 305) may be used simultaneously anytime a user engages in a cooperative conversation 300. Long-term shared knowledge may include explicit and/or implicit user preferences, a history of most recently used agents, contexts, requests, tasks, etc., user-specific jargon related to vocabularies and/or capabilities of an agent and/or context, most often used word choices, or other information. The long-term shared knowledge may be used to populate Intelligent Hypothesis Builder 310 with current and relevant information, provide Adaptive Response Builder 315 with appropriate word choices when the appropriate word choices are unavailable via the Session Input Accumulator, refine long-term shared knowledge models, identify a frequency of specific tasks, identify tasks a user frequently has difficulty with, or provide other information and/or analysis to generate more accurate conversational responses.
As described above, shared knowledge 305 may be used to populate Intelligent Hypothesis Builder 310, such that a user and a voice user interface may share assumptions and expectations such as topic knowledge, conversation history, word usage, jargon, tone (e.g., formal, humorous, terse, etc.), or other assumptions and/or expectations that facilitate interaction at a Human-to-Machine interface.
According to an aspect of the invention, one component of a successful cooperative conversation may be identifying a type of conversation from an utterance. By categorizing and developing conceptual models for various types of exchanges, user expectations and domain capabilities may be consistently aligned. Intelligent Hypothesis Builder 310 may generate a hypothesis as to a conversation type by considering conversational goals, participant roles, and/or an allocation of information among the participants. Conversational goals may broadly include: (1) getting a discrete piece of information or performing a discrete task, (2) gathering related pieces of information to make a decision, and/or (3) disseminating or gathering large amounts of information to build expertise. Participant roles may broadly include: (1) a leader that controls a conversation, (2) a supporter that follows the leader and provides input as requested, and/or (3) a consumer that uses information. Information may be held by one or more of the participants at the outset of a conversation, where a participant may hold most (or all) of the information, little (or none) of the information, or the information may be allocated roughly equally amongst the participants. Based on the conversational goals, participant roles, and allocation of information, Intelligent Hypothesis Builder 310 may consider various factors to classify a conversation (or utterance) into general types of conversations that can interact with one another to form many more variations and permutations of conversation types (e.g., a conversation type may change dynamically as information is reallocated from one participant to another, or as conversational goals change based on the reallocation of information).
For example, in one implementation, a query conversation may include a conversational goal of getting a discrete piece of information or performing a particular task, where a leader of the query conversation may have a specific goal in mind and may lead the conversation toward achieving the goal. The other participant may hold the information and may support the leader by providing the information. In a didactic conversation, a leader of the conversation may control information desired by a supporter of the conversation. The supporter's role may be limited to regulating an overall progression of the conversation and interjecting queries for clarification. In an exploratory conversation, both participants share leader and supporter roles, and the conversation may have no specific goal, or the goal may be improvised as the conversation progresses. Based on this model, Intelligent Hypothesis Builder 310 may broadly categorize a conversation (or utterance) according to the following diagram:
Intelligent Hypothesis Builder 310 may use an identified conversation type to assist in generating a set of hypotheses as to a user's intent in an utterance. In addition, Intelligent Hypothesis Builder 310 may use short-term shared knowledge from the Session Input Accumulator to proactively build and evaluate interaction with a user as a conversation progresses, as well as long-term shared knowledge to proactively build and evaluate interaction with the user over time. Intelligent Hypothesis Builder 310 may thus adaptively arrive at a set of N-best hypotheses about user intent, and the N-best hypotheses may be provided to an Adaptive Response Builder 315. In addition, Intelligent Hypothesis Builder 310 may model human-to-human interaction by calculating a degree of certainty for each of the hypotheses. That is, just as humans rely on knowledge shared by participants to examine how much and what kind of information was available, Intelligent Hypothesis Builder 310 may leverage the identified conversation type and short-term and long-term shared knowledge to generate a degree of certainty for each hypothesis.
According to another aspect of the invention, Intelligent Hypothesis Builder 310 may generate one or more explicit hypotheses of a user's intent when an utterance contains all information (including qualifiers) needed to complete a request or task. Each hypothesis may have a corresponding degree of certainty, which may be used to determine a level of unprompted support to provide in a response. For example, a response may include a confirmation to ensure the utterance was not misunderstood or the response may adaptively prompt a user to provide missing information.
According to another aspect of the invention, Intelligent Hypothesis Builder 310 may use short-term knowledge to generate one or more implicit hypotheses of a user's intent when an utterance may be missing required qualifiers or other information needed to complete a request or task. Each hypothesis may have a corresponding degree of certainty. For instance, when a conversation begins, short-term knowledge stored in the Session Input Accumulator may be empty, and as the conversation progresses, the Session Input Accumulator may build a history of the conversation. Intelligent Hypothesis Builder 310 may use data in the Session Input Accumulator to supplement or infer additional information about a current utterance. For example, Intelligent Hypothesis Builder 310 may evaluate a degree of certainty based on a number of previous requests relevant to the current utterance. In another example, when the current utterance contains insufficient information to complete a request or task, data in the Session Input Accumulator may be used to infer missing information so that a hypothesis can be generated. In still another example, Intelligent Hypothesis Builder 310 may identify syntax and/or grammar to be used by Adaptive Response Builder 315 to formulate personalized and conversational response. In yet another example, when the current utterance contains a threshold amount of information needed to complete a request or task, data in the Session Input Accumulator may be relied upon to tune a degree of certainty.
According to another aspect of the invention, Intelligent Hypothesis Builder 310 may use long-term shared knowledge to generate one or more implicit hypotheses of a user's intent when an utterance is missing qualifiers or other information needed to complete a request or task. Each hypothesis may have a corresponding degree of certainty. Using long-term knowledge may be substantially similar to using short-term shared knowledge, except that information may be unconstrained by a current session, and an input mechanism may include information from additional sources other than conversational sessions. For example, Intelligent Hypothesis Builder 310 may use information from long-term shared knowledge at any time, even when a new conversation is initiated, whereas short-term shared knowledge may be limited to an existing conversation (where no short-term shared knowledge would be available when a new conversation is initiated). Long-term shared knowledge may come from several sources, including user preferences or a plug-in data source (e.g., a subscription interface to a remote database), expertise of a user (e.g., based on a frequency of errors, types of tasks requested, etc., the user may be identified as a novice, intermediate, experienced, or other type of user), agent-specific information and/or language that may also apply to other agents (e.g., by decoupling information from an agent to incorporate the information into other agents), frequently used topics passed in from the Session Input Accumulator, frequently used verbs, nouns, or other parts of speech, and/or other syntax information passed in from the Session Input Accumulator, or other sources of long-term shared knowledge may be used.
According to another aspect of the invention, knowledge-enabled utterances, as generated by Intelligent Hypothesis Builder 310, may include one or more explicit (supplied by a user), and one or more implicit (supplied by Intelligent Hypothesis Builder 310) contextual signifiers, qualifiers, criteria, and other information that can be used to identify and evaluate relevant tasks. At that point, Intelligent Hypothesis Builder 310 may provide an input to Adaptive Response Builder 315. The input received by Adaptive Response Builder 315 may include at least a ranked list of hypotheses, including explicit and/or implicit hypotheses, each of which may have a corresponding degree of certainty. A hypothesis may be assigned one of four degrees of certainty: (1) “sure,” where contextual signifiers and qualifiers relate to one task, context and qualifiers relate to one task, and a confidence level associated with a preliminary interpretation generated at the speech recognition engine exceeds a predetermined threshold; (2) “pretty sure,” where contextual signifiers and qualifiers relate to more than one task (select top-ranked task) and criteria relates to one request, and/or the confidence level associated with the preliminary interpretation generated at the speech recognition engine is below the predetermined threshold; (3) “not sure,” where additional contextual signifiers or qualifiers are needed to indicate or rank a task; and (4) “no hypothesis,” where little or no information can be deciphered. Each degree of certainty may further be classified as explicit or implicit, which may be used to adjust a response. The input received by Adaptive Response Builder 310 may also include a context, user syntax and/or grammar, context domain agent specific information and/or preferences (e.g., a travel context domain agent may know a user frequently requests information about France, which may be shared with a movie context domain agent so that responses may occasionally include French movies).
According to another aspect of the invention, Adaptive Response Builder 315 may build syntactically, grammatically, and contextually sensitive “intelligent responses” that can be used with one or more agents to generate a conversational experience for a user, while also guiding the user to reply in a manner favorable for recognition. In one implementation, the intelligent responses may include a verbal or audible reply played through an output device (e.g., a speaker), and/or an action performed by a device, computer, or machine (e.g., downloading a web page, showing a list, executing an application, etc.). In one implementation, an appropriate response may not require conversational adaptation, and default replies and/or randomly selected response sets for a given task may be used.
According to another aspect of the invention, Adaptive Response Builder 310 may draw on information maintained by Intelligence Hypothesis Builder 310 to generate responses that may be sensitive to context, task recognition of a current utterance, what a user already knows about a topic, what an application already knows about the topic, shared knowledge regarding user preferences and/or related topics, appropriate contextual word usage (e.g., jargon), words uttered by the user in recent utterances, conversational development and/or course correction, conversational tone, type of conversation, natural variation in wording of responses, or other information. As a result, Adaptive Response Builder 315 may generate intelligent responses that create conversational feel, adapt to information that accumulates over a duration of a conversation, maintain cross-modal awareness, and keep the conversation on course.
According to another aspect of the invention, Adaptive Response Builder 315 may create a conversational feel by adapting to a user's manner of speaking, framing responses appropriately, and having natural variation and/or personality (e.g., by varying tone, pace, timing, inflection, word use, jargon, and other variables in a verbal or audible response). Adapting to a user's manner of speaking may include using contextual signifiers and grammatical rules to generate one or more sentences for use as response sets that may cooperate with the user. By taking advantage of short-term (from the Session Input Accumulator) and long-term (from one or more profiles) shared knowledge about how a user utters a request, the responses may be modeled using techniques used to recognize requests. Adaptive Response Builder 315 may rate possible responses statistically and/or randomize responses, which creates an opportunity to build an exchange with natural variation and conversational feel. This may be a significant advantage over existing voice user interfaces with incongruous input and output, where the input is “conversational” and the output is “computerese.” The following examples may demonstrate how a response may adapt to a user's input word choices and manner of speaking:
According to another aspect of the invention, Adaptive Response Builder 315 may frame responses to influence a user to reply with an utterance that may be easily recognized. For example, a user may utter, “Get me the news” and a voice user interface response may be “Which of these categories? Top news stories, international news, political news, or sports news?” The response may be likely to illicit utterances from the user, such as “Top news stories” or “International news,” which are more likely to result in a completed request. Thus, the responses may conform to a cooperative nature of human dialog, and a natural human tendency to “parrot” what was just heard as part of a next utterance. Moreover, knowledge of current context may enhance responses to generate more meaningful conversational responses, such as in the following exchange:
Framing the responses may also deal with misrecognitions according to human models. For example, humans frequently remember a number of recent utterances, especially when one or more previous utterances were misrecognized or unrecognized. Another participant in the conversation may limit correction to a part of the utterance that was misrecognized or unrecognized, or over subsequent utterances and/or other interactions, clues may be provided to indicate the initial interpretation was incorrect. Thus, by storing and analyzing multiple utterances, utterances from earlier in a conversation may be corrected as the conversation progresses.
According to another aspect of the invention, Adaptive Response Builder 315 may generate multi-modal, or cross-modal, responses to a user. In one implementation, responses may be aware of and control one or more devices and/or interfaces, and users may respond by using whichever input method, or combination of input methods, is most convenient. For example, a response asking the user to direct an utterance with a “Yes” or “No” in a multi-modal environment may also display alternatives visually.
According to another aspect of the invention, Adaptive Response Builder 315 may correct a course of a conversation without interrupting conversational flow. Adaptive Response Builder 315 may generate intelligent responses based on the ranked list of hypotheses and corresponding degrees of certainty, which may be used to correct a course of a conversation without interrupting conversational flow. That is, even though the intelligent responses may be reasonably “sure,” the intelligent responses may nonetheless sometimes be incorrect. While existing voice user interfaces tend to fail on average conversational missteps, normal human interactions may expect missteps and deal with them appropriately. Thus, responses after a misrecognition may be modeled after clarifications, rather than errors, and words may be chosen in subsequent responses that move conversation forward and establish an appropriate domain to be explored with the user. For example, course correction may result in the following exchange:
The above disclosure has been described in terms of specific exemplary aspects, implementations, and embodiments of the invention. However, those skilled in the art will recognize various changes and modifications that may be made without departing from the scope and spirit of the invention. Therefore, the specification and drawings are to be regarded as exemplary only, and the scope of the invention is to be determined solely by the appended claims.
This application is a divisional of U.S. patent application Ser. No. 14/691,445, entitled “SYSTEM AND METHOD FOR A COOPERATIVE CONVERSATIONAL VOICE USER INTERFACE,” filed Apr. 20, 2015, which is a continuation of U.S. patent application Ser. No. 13/987,645, entitled “SYSTEM AND METHOD FOR A COOPERATIVE CONVERSATIONAL VOICE USER INTERFACE,” filed Aug. 19, 2013 (which issued as U.S. Pat. No. 9,015,049 on Apr. 21, 2015), which is a divisional of U.S. patent application Ser. No. 13/251,712, entitled “SYSTEM AND METHOD FOR A COOPERATIVE CONVERSATIONAL VOICE USER INTERFACE,” filed Oct. 3, 2011 (which issued as U.S. Pat. No. 8,515,765 on Aug. 20, 2013), which is a continuation of U.S. patent application Ser. No. 11/580,926, entitled “SYSTEM AND METHOD FOR A COOPERATIVE CONVERSATIONAL VOICE USER INTERFACE,” filed Oct. 16, 2006 (which issued as U.S. Pat. No. 8,073,681 on Dec. 6, 2011), each of which are hereby incorporated by reference in its entirety.
| Number | Name | Date | Kind |
|---|---|---|---|
| 4430669 | Cheung | Feb 1984 | A |
| 4821027 | Mallory | Apr 1989 | A |
| 4829423 | Tennant | May 1989 | A |
| 4887212 | Zamora | Dec 1989 | A |
| 4910784 | Doddington | Mar 1990 | A |
| 5027406 | Roberts | Jun 1991 | A |
| 5155743 | Jacobs | Oct 1992 | A |
| 5164904 | Sumner | Nov 1992 | A |
| 5208748 | Flores | May 1993 | A |
| 5265065 | Turtle | Nov 1993 | A |
| 5274560 | LaRue | Dec 1993 | A |
| 5331554 | Graham | Jul 1994 | A |
| 5357596 | Takebayashi | Oct 1994 | A |
| 5369575 | Lamberti | Nov 1994 | A |
| 5377350 | Skinner | Dec 1994 | A |
| 5386556 | Hedin | Jan 1995 | A |
| 5424947 | Nagao | Jun 1995 | A |
| 5471318 | Ahuja | Nov 1995 | A |
| 5475733 | Eisdorfer | Dec 1995 | A |
| 5479563 | Yamaguchi | Dec 1995 | A |
| 5488652 | Bielby | Jan 1996 | A |
| 5499289 | Bruno | Mar 1996 | A |
| 5500920 | Kupiec | Mar 1996 | A |
| 5517560 | Greenspan | May 1996 | A |
| 5533108 | Harris | Jul 1996 | A |
| 5537436 | Bottoms | Jul 1996 | A |
| 5539744 | Chu | Jul 1996 | A |
| 5557667 | Bruno | Sep 1996 | A |
| 5559864 | Kennedy, Jr. | Sep 1996 | A |
| 5563937 | Bruno | Oct 1996 | A |
| 5577165 | Takebayashi | Nov 1996 | A |
| 5590039 | Ikeda | Dec 1996 | A |
| 5608635 | Tamai | Mar 1997 | A |
| 5615296 | Stanford | Mar 1997 | A |
| 5617407 | Bareis | Apr 1997 | A |
| 5633922 | August | May 1997 | A |
| 5634086 | Rtischev | May 1997 | A |
| 5652570 | Lepkofker | Jul 1997 | A |
| 5675629 | Raffel | Oct 1997 | A |
| 5696965 | Dedrick | Dec 1997 | A |
| 5708422 | Blonder | Jan 1998 | A |
| 5721938 | Stuckey | Feb 1998 | A |
| 5722084 | Chakrin | Feb 1998 | A |
| 5729659 | Potter | Mar 1998 | A |
| 5740256 | CastelloDaCosta | Apr 1998 | A |
| 5742763 | Jones | Apr 1998 | A |
| 5748841 | Morin | May 1998 | A |
| 5748974 | Johnson | May 1998 | A |
| 5752052 | Richardson | May 1998 | A |
| 5754784 | Garland | May 1998 | A |
| 5761631 | Nasukawa | Jun 1998 | A |
| 5774841 | Salazar | Jun 1998 | A |
| 5774859 | Houser | Jun 1998 | A |
| 5794050 | Dahlgren | Aug 1998 | A |
| 5794196 | Yegnanarayanan | Aug 1998 | A |
| 5797112 | Komatsu | Aug 1998 | A |
| 5799276 | Komissarchik | Aug 1998 | A |
| 5802510 | Jones | Sep 1998 | A |
| 5829000 | Huang | Oct 1998 | A |
| 5832221 | Jones | Nov 1998 | A |
| 5839107 | Gupta | Nov 1998 | A |
| 5848396 | Gerace | Dec 1998 | A |
| 5855000 | Waibel | Dec 1998 | A |
| 5860059 | Aust | Jan 1999 | A |
| 5867817 | Catallo | Feb 1999 | A |
| 5878385 | Bralich | Mar 1999 | A |
| 5878386 | Coughlin | Mar 1999 | A |
| 5892813 | Morin | Apr 1999 | A |
| 5892900 | Ginter | Apr 1999 | A |
| 5895464 | Bhandari | Apr 1999 | A |
| 5895466 | Goldberg | Apr 1999 | A |
| 5897613 | Chan | Apr 1999 | A |
| 5898760 | Smets | Apr 1999 | A |
| 5899991 | Karch | May 1999 | A |
| 5902347 | Backman | May 1999 | A |
| 5911120 | Jarett | Jun 1999 | A |
| 5918222 | Fukui | Jun 1999 | A |
| 5926784 | Richardson | Jul 1999 | A |
| 5933822 | Braden-Harder | Aug 1999 | A |
| 5950167 | Yaker | Sep 1999 | A |
| 5953393 | Culbreth | Sep 1999 | A |
| 5960384 | Brash | Sep 1999 | A |
| 5960397 | Rahim | Sep 1999 | A |
| 5960399 | Barclay | Sep 1999 | A |
| 5960447 | Holt | Sep 1999 | A |
| 5963894 | Richardson | Oct 1999 | A |
| 5963940 | Liddy | Oct 1999 | A |
| 5982906 | Ono | Nov 1999 | A |
| 5983190 | Trowerll | Nov 1999 | A |
| 5987404 | DellaPietra | Nov 1999 | A |
| 5991721 | Asano | Nov 1999 | A |
| 5995119 | Cosatto | Nov 1999 | A |
| 5995928 | Nguyen | Nov 1999 | A |
| 5995943 | Bull | Nov 1999 | A |
| 6009382 | Martino | Dec 1999 | A |
| 6014559 | Amin | Jan 2000 | A |
| 6018708 | Dahan | Jan 2000 | A |
| 6021384 | Gorin | Feb 2000 | A |
| 6028514 | Lemelson | Feb 2000 | A |
| 6035267 | Watanabe | Mar 2000 | A |
| 6044347 | Abella | Mar 2000 | A |
| 6049602 | Foladare | Apr 2000 | A |
| 6049607 | Marash | Apr 2000 | A |
| 6058187 | Chen | May 2000 | A |
| 6067513 | Ishimitsu | May 2000 | A |
| 6073098 | Buchsbaum | Jun 2000 | A |
| 6076057 | Narayanan | Jun 2000 | A |
| 6076059 | Glickman | Jun 2000 | A |
| 6078886 | Dragosh | Jun 2000 | A |
| 6081774 | deHita | Jun 2000 | A |
| 6081779 | Stefan | Jun 2000 | A |
| 6085186 | Christianson | Jul 2000 | A |
| 6101241 | Boyce | Aug 2000 | A |
| 6108631 | Ruhl | Aug 2000 | A |
| 6119087 | Kuhn | Sep 2000 | A |
| 6119101 | Peckover | Sep 2000 | A |
| 6122613 | Baker | Sep 2000 | A |
| 6134235 | Goldman | Oct 2000 | A |
| 6144667 | Doshi | Nov 2000 | A |
| 6144938 | Surace | Nov 2000 | A |
| 6154526 | Dahlke | Nov 2000 | A |
| 6160883 | Jackson | Dec 2000 | A |
| 6167377 | Gillick | Dec 2000 | A |
| 6173250 | Jong | Jan 2001 | B1 |
| 6173266 | Marx | Jan 2001 | B1 |
| 6173279 | Levin | Jan 2001 | B1 |
| 6175858 | Bulfer | Jan 2001 | B1 |
| 6185535 | Hedin | Feb 2001 | B1 |
| 6188982 | Chiang | Feb 2001 | B1 |
| 6192110 | Abella | Feb 2001 | B1 |
| 6192338 | Haszto | Feb 2001 | B1 |
| 6195634 | Dudemaine | Feb 2001 | B1 |
| 6195651 | Handel | Feb 2001 | B1 |
| 6199043 | Happ | Mar 2001 | B1 |
| 6208964 | Sabourin | Mar 2001 | B1 |
| 6208972 | Grant | Mar 2001 | B1 |
| 6219346 | Maxemchuk | Apr 2001 | B1 |
| 6219643 | Cohen | Apr 2001 | B1 |
| 6219645 | Byers | Apr 2001 | B1 |
| 6226612 | Srenger | May 2001 | B1 |
| 6233556 | Teunen | May 2001 | B1 |
| 6233559 | Balakrishnan | May 2001 | B1 |
| 6233561 | Junqua | May 2001 | B1 |
| 6236968 | Kanevsky | May 2001 | B1 |
| 6243679 | Mohri | Jun 2001 | B1 |
| 6246981 | Papineni | Jun 2001 | B1 |
| 6246990 | Happ | Jun 2001 | B1 |
| 6266636 | Kosaka | Jul 2001 | B1 |
| 6269336 | Ladd | Jul 2001 | B1 |
| 6272455 | Hoshen | Aug 2001 | B1 |
| 6272461 | Meredith | Aug 2001 | B1 |
| 6275231 | Obradovich | Aug 2001 | B1 |
| 6278377 | DeLine | Aug 2001 | B1 |
| 6278968 | Franz | Aug 2001 | B1 |
| 6286002 | Axaopoulos | Sep 2001 | B1 |
| 6288319 | Catona | Sep 2001 | B1 |
| 6292767 | Jackson | Sep 2001 | B1 |
| 6301560 | Masters | Oct 2001 | B1 |
| 6308151 | Smith | Oct 2001 | B1 |
| 6311159 | VanTichelen | Oct 2001 | B1 |
| 6314402 | Monaco | Nov 2001 | B1 |
| 6321196 | Franceschi | Nov 2001 | B1 |
| 6356869 | Chapados | Mar 2002 | B1 |
| 6362748 | Huang | Mar 2002 | B1 |
| 6366882 | Bijl | Apr 2002 | B1 |
| 6366886 | Dragosh | Apr 2002 | B1 |
| 6374214 | Friedland | Apr 2002 | B1 |
| 6374226 | Hunt | Apr 2002 | B1 |
| 6377913 | Coffman | Apr 2002 | B1 |
| 6377919 | Burnett | Apr 2002 | B1 |
| 6381535 | Durocher | Apr 2002 | B1 |
| 6385596 | Wiser | May 2002 | B1 |
| 6385646 | Brown | May 2002 | B1 |
| 6389398 | Lustgarten | May 2002 | B1 |
| 6393403 | Majaniemi | May 2002 | B1 |
| 6393428 | Miller | May 2002 | B1 |
| 6393460 | Gruen | May 2002 | B1 |
| 6397181 | Li | May 2002 | B1 |
| 6404878 | Jackson | Jun 2002 | B1 |
| 6405170 | Phillips | Jun 2002 | B1 |
| 6408272 | White | Jun 2002 | B1 |
| 6411810 | Maxemchuk | Jun 2002 | B1 |
| 6411893 | Ruhl | Jun 2002 | B2 |
| 6415257 | Junqua | Jul 2002 | B1 |
| 6418210 | Sayko | Jul 2002 | B1 |
| 6420975 | DeLine | Jul 2002 | B1 |
| 6429813 | Feigen | Aug 2002 | B2 |
| 6430285 | Bauer | Aug 2002 | B1 |
| 6430531 | Polish | Aug 2002 | B1 |
| 6434523 | Monaco | Aug 2002 | B1 |
| 6434524 | Weber | Aug 2002 | B1 |
| 6434529 | Walker | Aug 2002 | B1 |
| 6442522 | Carberry | Aug 2002 | B1 |
| 6446114 | Bulfer | Sep 2002 | B1 |
| 6453153 | Bowker | Sep 2002 | B1 |
| 6453292 | Ramaswamy | Sep 2002 | B2 |
| 6456711 | Cheung | Sep 2002 | B1 |
| 6456974 | Baker | Sep 2002 | B1 |
| 6466654 | Cooper | Oct 2002 | B1 |
| 6466899 | Yano | Oct 2002 | B1 |
| 6470315 | Netsch | Oct 2002 | B1 |
| 6487494 | Odinak | Nov 2002 | B2 |
| 6487495 | Gale | Nov 2002 | B1 |
| 6498797 | Anerousis | Dec 2002 | B1 |
| 6499013 | Weber | Dec 2002 | B1 |
| 6501833 | Phillips | Dec 2002 | B2 |
| 6501834 | Milewski | Dec 2002 | B1 |
| 6505155 | Vanbuskirk | Jan 2003 | B1 |
| 6510417 | Woods | Jan 2003 | B1 |
| 6513006 | Howard | Jan 2003 | B2 |
| 6522746 | Marchok | Feb 2003 | B1 |
| 6523061 | Halverson | Feb 2003 | B1 |
| 6526335 | Treyz | Feb 2003 | B1 |
| 6532444 | Weber | Mar 2003 | B1 |
| 6539348 | Bond | Mar 2003 | B1 |
| 6549629 | Finn | Apr 2003 | B2 |
| 6553372 | Brassell | Apr 2003 | B1 |
| 6556970 | Sasaki | Apr 2003 | B1 |
| 6556973 | Lewin | Apr 2003 | B1 |
| 6560576 | Cohen | May 2003 | B1 |
| 6560590 | Shwe | May 2003 | B1 |
| 6567778 | ChaoChang | May 2003 | B1 |
| 6567797 | Schuetze | May 2003 | B1 |
| 6567805 | Johnson | May 2003 | B1 |
| 6570555 | Prevost | May 2003 | B1 |
| 6570964 | Murveit | May 2003 | B1 |
| 6571279 | Herz | May 2003 | B1 |
| 6574597 | Mohri | Jun 2003 | B1 |
| 6574624 | Johnson | Jun 2003 | B1 |
| 6578022 | Foulger | Jun 2003 | B1 |
| 6581103 | Dengler | Jun 2003 | B1 |
| 6584439 | Geilhufe | Jun 2003 | B1 |
| 6587858 | Strazza | Jul 2003 | B1 |
| 6591185 | Polidi | Jul 2003 | B1 |
| 6591239 | McCall | Jul 2003 | B1 |
| 6594257 | Doshi | Jul 2003 | B1 |
| 6594367 | Marash | Jul 2003 | B1 |
| 6598018 | Junqua | Jul 2003 | B1 |
| 6601026 | Appelt | Jul 2003 | B2 |
| 6601029 | Pickering | Jul 2003 | B1 |
| 6604075 | Brown | Aug 2003 | B1 |
| 6604077 | Dragosh | Aug 2003 | B2 |
| 6606598 | Holthouse | Aug 2003 | B1 |
| 6611692 | Raffel | Aug 2003 | B2 |
| 6614773 | Maxemchuk | Sep 2003 | B1 |
| 6615172 | Bennett | Sep 2003 | B1 |
| 6622119 | Ramaswamy | Sep 2003 | B1 |
| 6629066 | Jackson | Sep 2003 | B1 |
| 6631346 | Karaorman | Oct 2003 | B1 |
| 6631351 | Ramachandran | Oct 2003 | B1 |
| 6633846 | Bennett | Oct 2003 | B1 |
| 6636790 | Lightner | Oct 2003 | B1 |
| 6643620 | Contolini | Nov 2003 | B1 |
| 6647363 | Claassen | Nov 2003 | B2 |
| 6650747 | Bala | Nov 2003 | B1 |
| 6658388 | Kleindienst | Dec 2003 | B1 |
| 6678680 | Woo | Jan 2004 | B1 |
| 6681206 | Gorin | Jan 2004 | B1 |
| 6691151 | Cheyer | Feb 2004 | B1 |
| 6701294 | Ball | Mar 2004 | B1 |
| 6704396 | Parolkar | Mar 2004 | B2 |
| 6704576 | Brachman | Mar 2004 | B1 |
| 6704708 | Pickering | Mar 2004 | B1 |
| 6707421 | Drury | Mar 2004 | B1 |
| 6708150 | Hirayama | Mar 2004 | B1 |
| 6721001 | Berstis | Apr 2004 | B1 |
| 6721633 | Funk | Apr 2004 | B2 |
| 6721706 | Strubbe | Apr 2004 | B1 |
| 6726636 | DerGhazarian | Apr 2004 | B2 |
| 6732088 | Glance | May 2004 | B1 |
| 6735592 | Neumann | May 2004 | B1 |
| 6739556 | Langston | May 2004 | B1 |
| 6741931 | Kohut | May 2004 | B1 |
| 6742021 | Halverson | May 2004 | B1 |
| 6745161 | Arnold | Jun 2004 | B1 |
| 6751591 | Gorin | Jun 2004 | B1 |
| 6751612 | Schuetze | Jun 2004 | B1 |
| 6754485 | Obradovich | Jun 2004 | B1 |
| 6754627 | Woodward | Jun 2004 | B2 |
| 6754647 | Tackett | Jun 2004 | B1 |
| 6757362 | Cooper | Jun 2004 | B1 |
| 6757544 | Rangarajan | Jun 2004 | B2 |
| 6757718 | Halverson | Jun 2004 | B1 |
| 6785651 | Wang | Aug 2004 | B1 |
| 6795808 | Strubbe | Sep 2004 | B1 |
| 6801604 | Maes | Oct 2004 | B2 |
| 6801893 | Backfried | Oct 2004 | B1 |
| 6804330 | Jones | Oct 2004 | B1 |
| 6810375 | Ejerhed | Oct 2004 | B1 |
| 6813341 | Mahoney | Nov 2004 | B1 |
| 6816830 | Kempe | Nov 2004 | B1 |
| 6823308 | Alexander | Nov 2004 | B2 |
| 6829603 | Chai | Dec 2004 | B1 |
| 6832230 | Zilliacus | Dec 2004 | B1 |
| 6833848 | Wolff | Dec 2004 | B1 |
| 6839896 | Coffman | Jan 2005 | B2 |
| 6850603 | Eberle | Feb 2005 | B1 |
| 6856990 | Barile | Feb 2005 | B2 |
| 6859776 | Cohen | Feb 2005 | B1 |
| 6865481 | Kawazoe | Mar 2005 | B2 |
| 6868380 | Kroeker | Mar 2005 | B2 |
| 6868385 | Gerson | Mar 2005 | B1 |
| 6871179 | Kist | Mar 2005 | B1 |
| 6873837 | Yoshioka | Mar 2005 | B1 |
| 6877001 | Wolf | Apr 2005 | B2 |
| 6877134 | Fuller | Apr 2005 | B1 |
| 6882970 | Garner | Apr 2005 | B1 |
| 6901366 | Kuhn | May 2005 | B1 |
| 6910003 | Arnold | Jun 2005 | B1 |
| 6912498 | Stevens | Jun 2005 | B2 |
| 6915126 | Mazzara, Jr. | Jul 2005 | B2 |
| 6928614 | Everhart | Aug 2005 | B1 |
| 6934756 | Maes | Aug 2005 | B2 |
| 6937977 | Gerson | Aug 2005 | B2 |
| 6937982 | Kitaoka | Aug 2005 | B2 |
| 6941266 | Gorin | Sep 2005 | B1 |
| 6944594 | Busayapongchai | Sep 2005 | B2 |
| 6950821 | Faybishenko | Sep 2005 | B2 |
| 6954755 | Reisman | Oct 2005 | B2 |
| 6959276 | Droppo | Oct 2005 | B2 |
| 6961700 | Mitchell | Nov 2005 | B2 |
| 6963759 | Gerson | Nov 2005 | B1 |
| 6964023 | Maes | Nov 2005 | B2 |
| 6968311 | Knockeart | Nov 2005 | B2 |
| 6973387 | Masclet | Dec 2005 | B2 |
| 6975983 | Fortescue | Dec 2005 | B1 |
| 6975993 | Keiller | Dec 2005 | B1 |
| 6980092 | Turnbull | Dec 2005 | B2 |
| 6983055 | Luo | Jan 2006 | B2 |
| 6990513 | Belfiore | Jan 2006 | B2 |
| 6996531 | Korall | Feb 2006 | B2 |
| 7003463 | Maes | Feb 2006 | B1 |
| 7016849 | Arnold | Mar 2006 | B2 |
| 7020609 | Thrift | Mar 2006 | B2 |
| 7024364 | Guerra | Apr 2006 | B2 |
| 7027586 | Bushey | Apr 2006 | B2 |
| 7027974 | Busch | Apr 2006 | B1 |
| 7027975 | Pazandak | Apr 2006 | B1 |
| 7031908 | Huang | Apr 2006 | B1 |
| 7035415 | Belt | Apr 2006 | B2 |
| 7036128 | Julia | Apr 2006 | B1 |
| 7043425 | Pao | May 2006 | B2 |
| 7054817 | Shao | May 2006 | B2 |
| 7058890 | George | Jun 2006 | B2 |
| 7062488 | Reisman | Jun 2006 | B1 |
| 7069220 | Coffman | Jun 2006 | B2 |
| 7072834 | Zhou | Jul 2006 | B2 |
| 7072888 | Perkins | Jul 2006 | B1 |
| 7076362 | Ohtsuji | Jul 2006 | B2 |
| 7082469 | Gold | Jul 2006 | B2 |
| 7085708 | Manson | Aug 2006 | B2 |
| 7092928 | Elad | Aug 2006 | B1 |
| 7107210 | Deng | Sep 2006 | B2 |
| 7107218 | Preston | Sep 2006 | B1 |
| 7110951 | Lemelson | Sep 2006 | B1 |
| 7127395 | Gorin | Oct 2006 | B1 |
| 7127400 | Koch | Oct 2006 | B2 |
| 7130390 | Abburi | Oct 2006 | B2 |
| 7136875 | Anderson | Nov 2006 | B2 |
| 7137126 | Coffman | Nov 2006 | B1 |
| 7143037 | Chestnut | Nov 2006 | B1 |
| 7143039 | Stifelman | Nov 2006 | B1 |
| 7146315 | Balan | Dec 2006 | B2 |
| 7146319 | Hunt | Dec 2006 | B2 |
| 7149696 | Shimizu | Dec 2006 | B2 |
| 7165028 | Gong | Jan 2007 | B2 |
| 7170993 | Anderson | Jan 2007 | B2 |
| 7171291 | Obradovich | Jan 2007 | B2 |
| 7174300 | Bush | Feb 2007 | B2 |
| 7177798 | Hsu | Feb 2007 | B2 |
| 7184957 | Brookes | Feb 2007 | B2 |
| 7190770 | Ando | Mar 2007 | B2 |
| 7197069 | Agazzi | Mar 2007 | B2 |
| 7197460 | Gupta | Mar 2007 | B1 |
| 7203644 | Anderson | Apr 2007 | B2 |
| 7206418 | Yang | Apr 2007 | B2 |
| 7207011 | Mulvey | Apr 2007 | B2 |
| 7215941 | Beckmann | May 2007 | B2 |
| 7228276 | Omote | Jun 2007 | B2 |
| 7231343 | Treadgold | Jun 2007 | B1 |
| 7236923 | Gupta | Jun 2007 | B1 |
| 7254482 | Kawasaki | Aug 2007 | B2 |
| 7272212 | Eberle | Sep 2007 | B2 |
| 7277854 | Bennett | Oct 2007 | B2 |
| 7283829 | Christenson | Oct 2007 | B2 |
| 7283951 | Marchisio | Oct 2007 | B2 |
| 7289606 | Sibal | Oct 2007 | B2 |
| 7299186 | Kuzunuki | Nov 2007 | B2 |
| 7301093 | Sater | Nov 2007 | B2 |
| 7305381 | Poppink | Dec 2007 | B1 |
| 7321850 | Wakita | Jan 2008 | B2 |
| 7328155 | Endo | Feb 2008 | B2 |
| 7337116 | Charlesworth | Feb 2008 | B2 |
| 7340040 | Saylor | Mar 2008 | B1 |
| 7366285 | Parolkar | Apr 2008 | B2 |
| 7366669 | Nishitani | Apr 2008 | B2 |
| 7376586 | Partovi | May 2008 | B1 |
| 7376645 | Bernard | May 2008 | B2 |
| 7380250 | Schechter | May 2008 | B2 |
| 7386443 | Parthasarathy | Jun 2008 | B1 |
| 7398209 | Kennewick | Jul 2008 | B2 |
| 7406421 | Odinak | Jul 2008 | B2 |
| 7415100 | Cooper | Aug 2008 | B2 |
| 7415414 | Azara | Aug 2008 | B2 |
| 7421393 | DiFabbrizio | Sep 2008 | B1 |
| 7424431 | Greene | Sep 2008 | B2 |
| 7430510 | De Fabbrizio | Sep 2008 | B1 |
| 7447635 | Konopka | Nov 2008 | B1 |
| 7451088 | Ehlen | Nov 2008 | B1 |
| 7454368 | Stillman | Nov 2008 | B2 |
| 7454608 | Gopalakrishnan | Nov 2008 | B2 |
| 7461059 | Richardson | Dec 2008 | B2 |
| 7472020 | Brulle-Drews | Dec 2008 | B2 |
| 7472060 | Gorin | Dec 2008 | B1 |
| 7472075 | Odinak | Dec 2008 | B2 |
| 7477909 | Roth | Jan 2009 | B2 |
| 7478036 | Shen | Jan 2009 | B2 |
| 7487088 | Gorin | Feb 2009 | B1 |
| 7487110 | Bennett | Feb 2009 | B2 |
| 7493259 | Jones | Feb 2009 | B2 |
| 7493559 | Wolff | Feb 2009 | B1 |
| 7502672 | Kolls | Mar 2009 | B1 |
| 7502730 | Wang | Mar 2009 | B2 |
| 7502738 | Kennewick | Mar 2009 | B2 |
| 7512906 | Baier | Mar 2009 | B1 |
| 7516076 | Walker | Apr 2009 | B2 |
| 7529675 | Maes | May 2009 | B2 |
| 7536297 | Byrd | May 2009 | B2 |
| 7536374 | Au | May 2009 | B2 |
| 7542894 | Murata | Jun 2009 | B2 |
| 7546382 | Healey | Jun 2009 | B2 |
| 7548491 | Macfarlane | Jun 2009 | B2 |
| 7552054 | Stifelman | Jun 2009 | B1 |
| 7558730 | Davis | Jul 2009 | B2 |
| 7574362 | Walker | Aug 2009 | B2 |
| 7577244 | Taschereau | Aug 2009 | B2 |
| 7606708 | Hwang | Oct 2009 | B2 |
| 7606712 | Smith | Oct 2009 | B1 |
| 7620549 | DiCristo | Nov 2009 | B2 |
| 7634409 | Kennewick | Dec 2009 | B2 |
| 7640006 | Portman | Dec 2009 | B2 |
| 7640160 | DiCristo | Dec 2009 | B2 |
| 7640272 | Mahajan | Dec 2009 | B2 |
| 7672931 | Hurst-Hiller | Mar 2010 | B2 |
| 7676365 | Hwang | Mar 2010 | B2 |
| 7676369 | Fujimoto | Mar 2010 | B2 |
| 7684977 | Morikawa | Mar 2010 | B2 |
| 7693720 | Kennewick | Apr 2010 | B2 |
| 7697673 | Chiu | Apr 2010 | B2 |
| 7706616 | Kristensson | Apr 2010 | B2 |
| 7729913 | Lee | Jun 2010 | B1 |
| 7729916 | Coffman | Jun 2010 | B2 |
| 7729918 | Walker | Jun 2010 | B2 |
| 7729920 | Chaar | Jun 2010 | B2 |
| 7734287 | Ying | Jun 2010 | B2 |
| 7748021 | Obradovich | Jun 2010 | B2 |
| 7774333 | Colledge | Aug 2010 | B2 |
| 7788084 | Brun | Aug 2010 | B2 |
| 7792257 | Vanier | Sep 2010 | B1 |
| 7801731 | Odinak | Sep 2010 | B2 |
| 7809570 | Kennewick | Oct 2010 | B2 |
| 7818176 | Freeman | Oct 2010 | B2 |
| 7831426 | Bennett | Nov 2010 | B2 |
| 7831433 | Belvin | Nov 2010 | B1 |
| 7856358 | Ho | Dec 2010 | B2 |
| 7873519 | Bennett | Jan 2011 | B2 |
| 7873523 | Potter | Jan 2011 | B2 |
| 7873654 | Bernard | Jan 2011 | B2 |
| 7881936 | Longe | Feb 2011 | B2 |
| 7890324 | Bangalore | Feb 2011 | B2 |
| 7894849 | Kass | Feb 2011 | B2 |
| 7902969 | Obradovich | Mar 2011 | B2 |
| 7917367 | DiCristo | Mar 2011 | B2 |
| 7920682 | Byrne | Apr 2011 | B2 |
| 7949529 | Weider | May 2011 | B2 |
| 7949537 | Walker | May 2011 | B2 |
| 7953732 | Frank | May 2011 | B2 |
| 7974875 | Quilici | Jul 2011 | B1 |
| 7983917 | Kennewick | Jul 2011 | B2 |
| 7984287 | Gopalakrishnan | Jul 2011 | B2 |
| 8005683 | Tessel | Aug 2011 | B2 |
| 8015006 | Kennewick | Sep 2011 | B2 |
| 8024186 | De Bonet | Sep 2011 | B1 |
| 8027965 | Takehara | Sep 2011 | B2 |
| 8032383 | Bhardwaj | Oct 2011 | B1 |
| 8060367 | Keaveney | Nov 2011 | B2 |
| 8069046 | Kennewick | Nov 2011 | B2 |
| 8073681 | Baldwin | Dec 2011 | B2 |
| 8077975 | Ma | Dec 2011 | B2 |
| 8078502 | Hao | Dec 2011 | B2 |
| 8082153 | Coffman | Dec 2011 | B2 |
| 8086463 | Ativanichayaphong | Dec 2011 | B2 |
| 8103510 | Sato | Jan 2012 | B2 |
| 8112275 | Kennewick | Feb 2012 | B2 |
| 8140327 | Kennewick | Mar 2012 | B2 |
| 8140335 | Kennewick | Mar 2012 | B2 |
| 8145489 | Freeman | Mar 2012 | B2 |
| 8150694 | Kennewick | Apr 2012 | B2 |
| 8155962 | Kennewick | Apr 2012 | B2 |
| 8170867 | Germain | May 2012 | B2 |
| 8180037 | Delker | May 2012 | B1 |
| 8190436 | Bangalore | May 2012 | B2 |
| 8195468 | Weider | Jun 2012 | B2 |
| 8200485 | Lee | Jun 2012 | B1 |
| 8204751 | Di Fabbrizio | Jun 2012 | B1 |
| 8219399 | Lutz | Jul 2012 | B2 |
| 8219599 | Tunstall-Pedoe | Jul 2012 | B2 |
| 8224652 | Wang | Jul 2012 | B2 |
| 8255224 | Singleton | Aug 2012 | B2 |
| 8326599 | Tomeh | Dec 2012 | B2 |
| 8326627 | Kennewick | Dec 2012 | B2 |
| 8326634 | DiCristo | Dec 2012 | B2 |
| 8326637 | Baldwin | Dec 2012 | B2 |
| 8332224 | DiCristo | Dec 2012 | B2 |
| 8340975 | Rosenberger | Dec 2012 | B1 |
| 8346563 | Hjelm | Jan 2013 | B1 |
| 8370147 | Kennewick | Feb 2013 | B2 |
| 8447607 | Weider | May 2013 | B2 |
| 8447651 | Scholl | May 2013 | B1 |
| 8452598 | Kennewick | May 2013 | B2 |
| 8503995 | Ramer | Aug 2013 | B2 |
| 8509403 | Chiu | Aug 2013 | B2 |
| 8515765 | Baldwin | Aug 2013 | B2 |
| 8527274 | Freeman | Sep 2013 | B2 |
| 8577671 | Barve | Nov 2013 | B1 |
| 8589161 | Kennewick | Nov 2013 | B2 |
| 8612205 | Hanneman | Dec 2013 | B2 |
| 8612206 | Chalabi | Dec 2013 | B2 |
| 8620659 | DiCristo | Dec 2013 | B2 |
| 8719005 | Lee | May 2014 | B1 |
| 8719009 | Baldwin | May 2014 | B2 |
| 8719026 | Kennewick | May 2014 | B2 |
| 8731929 | Kennewick | May 2014 | B2 |
| 8738380 | Baldwin | May 2014 | B2 |
| 8849652 | Weider | Sep 2014 | B2 |
| 8849670 | DiCristo | Sep 2014 | B2 |
| 8849696 | Pansari | Sep 2014 | B2 |
| 8849791 | Hertschuh | Sep 2014 | B1 |
| 8886536 | Freeman | Nov 2014 | B2 |
| 8972243 | Strom | Mar 2015 | B1 |
| 8972354 | Telang | Mar 2015 | B1 |
| 8983839 | Kennewick | Mar 2015 | B2 |
| 9009046 | Stewart | Apr 2015 | B1 |
| 9015049 | Baldwin | Apr 2015 | B2 |
| 9037455 | Faaborg | May 2015 | B1 |
| 9070366 | Mathias | Jun 2015 | B1 |
| 9070367 | Hoffmeister | Jun 2015 | B1 |
| 9105266 | Baldwin | Aug 2015 | B2 |
| 9171541 | Kennewick | Oct 2015 | B2 |
| 9269097 | Freeman | Feb 2016 | B2 |
| 9305548 | Kennewick | Apr 2016 | B2 |
| 9308445 | Merzenich | Apr 2016 | B1 |
| 9318108 | Gruber | Apr 2016 | B2 |
| 9406078 | Freeman | Aug 2016 | B2 |
| 9443514 | Taubman | Sep 2016 | B1 |
| 9502025 | Kennewick | Nov 2016 | B2 |
| 9626703 | Kennewick, Sr. | Apr 2017 | B2 |
| 9711143 | Kennewick | Jul 2017 | B2 |
| 10089984 | Kennewick | Oct 2018 | B2 |
| 20010011336 | Sitka | Aug 2001 | A1 |
| 20010014857 | Wang | Aug 2001 | A1 |
| 20010021905 | Burnett | Sep 2001 | A1 |
| 20010039492 | Nemoto | Nov 2001 | A1 |
| 20010041977 | Aoyagi | Nov 2001 | A1 |
| 20010041980 | Howard | Nov 2001 | A1 |
| 20010047261 | Kassan | Nov 2001 | A1 |
| 20010049601 | Kroeker | Dec 2001 | A1 |
| 20010054087 | Flom | Dec 2001 | A1 |
| 20020002548 | Roundtree | Jan 2002 | A1 |
| 20020007267 | Batchilo | Jan 2002 | A1 |
| 20020010584 | Schultz | Jan 2002 | A1 |
| 20020015500 | Belt | Feb 2002 | A1 |
| 20020022927 | Lemelson | Feb 2002 | A1 |
| 20020022956 | Ukrainczyk | Feb 2002 | A1 |
| 20020029186 | Roth | Mar 2002 | A1 |
| 20020029261 | Shibata | Mar 2002 | A1 |
| 20020032752 | Gold | Mar 2002 | A1 |
| 20020035501 | Handel | Mar 2002 | A1 |
| 20020040297 | Tsiao | Apr 2002 | A1 |
| 20020049535 | Rigo | Apr 2002 | A1 |
| 20020049805 | Yamada | Apr 2002 | A1 |
| 20020059068 | Rose | May 2002 | A1 |
| 20020065568 | Silfvast | May 2002 | A1 |
| 20020067839 | Heinrich | Jun 2002 | A1 |
| 20020069059 | Smith | Jun 2002 | A1 |
| 20020069071 | Knockeart | Jun 2002 | A1 |
| 20020073176 | Ikeda | Jun 2002 | A1 |
| 20020082911 | Dunn | Jun 2002 | A1 |
| 20020087312 | Lee | Jul 2002 | A1 |
| 20020087326 | Lee | Jul 2002 | A1 |
| 20020087525 | Abbott | Jul 2002 | A1 |
| 20020107694 | Lerg | Aug 2002 | A1 |
| 20020120609 | Lang | Aug 2002 | A1 |
| 20020124050 | Middeljans | Sep 2002 | A1 |
| 20020133347 | Schoneburg | Sep 2002 | A1 |
| 20020133354 | Ross | Sep 2002 | A1 |
| 20020133402 | Faber | Sep 2002 | A1 |
| 20020135618 | Maes | Sep 2002 | A1 |
| 20020138248 | Corston-Oliver | Sep 2002 | A1 |
| 20020143532 | McLean | Oct 2002 | A1 |
| 20020143535 | Kist | Oct 2002 | A1 |
| 20020152260 | Chen | Oct 2002 | A1 |
| 20020161646 | Gailey | Oct 2002 | A1 |
| 20020161647 | Gailey | Oct 2002 | A1 |
| 20020169597 | Fain | Nov 2002 | A1 |
| 20020173333 | Buchholz | Nov 2002 | A1 |
| 20020173961 | Guerra | Nov 2002 | A1 |
| 20020178003 | Gehrke | Nov 2002 | A1 |
| 20020184373 | Maes | Dec 2002 | A1 |
| 20020188455 | Shioda | Dec 2002 | A1 |
| 20020188602 | Stubler | Dec 2002 | A1 |
| 20020198714 | Zhou | Dec 2002 | A1 |
| 20030005033 | Mohan | Jan 2003 | A1 |
| 20030014261 | Kageyama | Jan 2003 | A1 |
| 20030016835 | Elko | Jan 2003 | A1 |
| 20030036903 | Konopka | Feb 2003 | A1 |
| 20030046071 | Wyman | Mar 2003 | A1 |
| 20030046281 | Son | Mar 2003 | A1 |
| 20030046346 | Mumick | Mar 2003 | A1 |
| 20030064709 | Gailey | Apr 2003 | A1 |
| 20030065427 | Funk | Apr 2003 | A1 |
| 20030069734 | Everhart | Apr 2003 | A1 |
| 20030069880 | Harrison | Apr 2003 | A1 |
| 20030088421 | Maes | May 2003 | A1 |
| 20030093419 | Bangalore | May 2003 | A1 |
| 20030097249 | Walker | May 2003 | A1 |
| 20030110037 | Walker | Jun 2003 | A1 |
| 20030112267 | Belrose | Jun 2003 | A1 |
| 20030115062 | Walker | Jun 2003 | A1 |
| 20030120493 | Gupta | Jun 2003 | A1 |
| 20030135488 | Amir | Jul 2003 | A1 |
| 20030144846 | Denenberg | Jul 2003 | A1 |
| 20030154082 | Toguri | Aug 2003 | A1 |
| 20030158731 | Falcon | Aug 2003 | A1 |
| 20030161448 | Parolkar | Aug 2003 | A1 |
| 20030167167 | Gong | Sep 2003 | A1 |
| 20030174155 | Weng | Sep 2003 | A1 |
| 20030182123 | Mitsuyoshi | Sep 2003 | A1 |
| 20030182132 | Niemoeller | Sep 2003 | A1 |
| 20030187643 | VanThong | Oct 2003 | A1 |
| 20030204492 | Wolf | Oct 2003 | A1 |
| 20030206640 | Malvar | Nov 2003 | A1 |
| 20030212550 | Ubale | Nov 2003 | A1 |
| 20030212558 | Matula | Nov 2003 | A1 |
| 20030212562 | Patel | Nov 2003 | A1 |
| 20030216919 | Roushar | Nov 2003 | A1 |
| 20030225825 | Healey | Dec 2003 | A1 |
| 20030233230 | Ammicht | Dec 2003 | A1 |
| 20030236664 | Sharma | Dec 2003 | A1 |
| 20040006475 | Ehlen | Jan 2004 | A1 |
| 20040010358 | Oesterling | Jan 2004 | A1 |
| 20040025115 | Sienel | Feb 2004 | A1 |
| 20040030741 | Wolton | Feb 2004 | A1 |
| 20040036601 | Obradovich | Feb 2004 | A1 |
| 20040044516 | Kennewick | Mar 2004 | A1 |
| 20040093567 | Schabes | May 2004 | A1 |
| 20040098245 | Walker | May 2004 | A1 |
| 20040102977 | Metzler | May 2004 | A1 |
| 20040117179 | Balasuriya | Jun 2004 | A1 |
| 20040117804 | Scahill | Jun 2004 | A1 |
| 20040122673 | Park | Jun 2004 | A1 |
| 20040122674 | Bangalore | Jun 2004 | A1 |
| 20040128514 | Rhoads | Jul 2004 | A1 |
| 20040133793 | Ginter | Jul 2004 | A1 |
| 20040140989 | Papageorge | Jul 2004 | A1 |
| 20040143440 | Prasad | Jul 2004 | A1 |
| 20040148154 | Acero | Jul 2004 | A1 |
| 20040148170 | Acero | Jul 2004 | A1 |
| 20040158555 | Seedman | Aug 2004 | A1 |
| 20040166832 | Portman | Aug 2004 | A1 |
| 20040167771 | Duan | Aug 2004 | A1 |
| 20040172247 | Yoon | Sep 2004 | A1 |
| 20040172258 | Dominach | Sep 2004 | A1 |
| 20040189697 | Fukuoka | Sep 2004 | A1 |
| 20040193408 | Hunt | Sep 2004 | A1 |
| 20040193420 | Kennewick | Sep 2004 | A1 |
| 20040199375 | Ehsani | Oct 2004 | A1 |
| 20040199387 | Wang | Oct 2004 | A1 |
| 20040199389 | Geiger | Oct 2004 | A1 |
| 20040201607 | Mulvey | Oct 2004 | A1 |
| 20040204939 | Daben | Oct 2004 | A1 |
| 20040205671 | Sukehiro | Oct 2004 | A1 |
| 20040220800 | Kong | Nov 2004 | A1 |
| 20040243393 | Wang | Dec 2004 | A1 |
| 20040243417 | Pitts | Dec 2004 | A9 |
| 20040247092 | Timmins | Dec 2004 | A1 |
| 20040249636 | Applebaum | Dec 2004 | A1 |
| 20050015256 | Kargman | Jan 2005 | A1 |
| 20050021331 | Huang | Jan 2005 | A1 |
| 20050021334 | Iwahashi | Jan 2005 | A1 |
| 20050021470 | Martin | Jan 2005 | A1 |
| 20050021826 | Kumar | Jan 2005 | A1 |
| 20050033574 | Kim | Feb 2005 | A1 |
| 20050033582 | Gadd | Feb 2005 | A1 |
| 20050043940 | Elder | Feb 2005 | A1 |
| 20050043956 | Aoyama | Feb 2005 | A1 |
| 20050060142 | Visser | Mar 2005 | A1 |
| 20050080632 | Endo | Apr 2005 | A1 |
| 20050080821 | Breil | Apr 2005 | A1 |
| 20050102282 | Linden | May 2005 | A1 |
| 20050114116 | Fiedler | May 2005 | A1 |
| 20050114781 | Brownholtz | May 2005 | A1 |
| 20050125232 | Gadd | Jun 2005 | A1 |
| 20050131673 | Koizumi | Jun 2005 | A1 |
| 20050137850 | Odell | Jun 2005 | A1 |
| 20050137877 | Oesterling | Jun 2005 | A1 |
| 20050138168 | Hoffman | Jun 2005 | A1 |
| 20050143994 | Mori | Jun 2005 | A1 |
| 20050144013 | Fujimoto | Jun 2005 | A1 |
| 20050144187 | Che | Jun 2005 | A1 |
| 20050149319 | Honda | Jul 2005 | A1 |
| 20050169441 | Yacoub | Aug 2005 | A1 |
| 20050216254 | Gupta | Sep 2005 | A1 |
| 20050222763 | Uyeki | Oct 2005 | A1 |
| 20050234637 | Obradovich | Oct 2005 | A1 |
| 20050234727 | Chiu | Oct 2005 | A1 |
| 20050240412 | Fujita | Oct 2005 | A1 |
| 20050246174 | DeGolia | Nov 2005 | A1 |
| 20050283364 | Longe | Dec 2005 | A1 |
| 20050283532 | Kim | Dec 2005 | A1 |
| 20050283752 | Fruchter | Dec 2005 | A1 |
| 20060041431 | Maes | Feb 2006 | A1 |
| 20060046740 | Johnson | Mar 2006 | A1 |
| 20060047362 | Aoyama | Mar 2006 | A1 |
| 20060047509 | Ding | Mar 2006 | A1 |
| 20060072738 | Louis | Apr 2006 | A1 |
| 20060074670 | Weng | Apr 2006 | A1 |
| 20060074671 | Farmaner | Apr 2006 | A1 |
| 20060080098 | Campbell | Apr 2006 | A1 |
| 20060100851 | Schonebeck | May 2006 | A1 |
| 20060106769 | Gibbs | May 2006 | A1 |
| 20060129409 | Mizutani | Jun 2006 | A1 |
| 20060130002 | Hirayama | Jun 2006 | A1 |
| 20060149633 | Voisin | Jul 2006 | A1 |
| 20060182085 | Sweeney | Aug 2006 | A1 |
| 20060206310 | Ravikumar | Sep 2006 | A1 |
| 20060217133 | Christenson | Sep 2006 | A1 |
| 20060236343 | Chang | Oct 2006 | A1 |
| 20060242017 | Libes | Oct 2006 | A1 |
| 20060247919 | Specht | Nov 2006 | A1 |
| 20060253247 | de Silva | Nov 2006 | A1 |
| 20060253281 | Letzt | Nov 2006 | A1 |
| 20060285662 | Yin | Dec 2006 | A1 |
| 20070011159 | Hillis | Jan 2007 | A1 |
| 20070033005 | Di Cristo | Feb 2007 | A1 |
| 20070033020 | Francois | Feb 2007 | A1 |
| 20070033526 | Thompson | Feb 2007 | A1 |
| 20070038436 | Cristo | Feb 2007 | A1 |
| 20070038445 | Helbing | Feb 2007 | A1 |
| 20070043569 | Potter | Feb 2007 | A1 |
| 20070043574 | Coffman | Feb 2007 | A1 |
| 20070043868 | Kumar | Feb 2007 | A1 |
| 20070050191 | Weider | Mar 2007 | A1 |
| 20070050279 | Huang | Mar 2007 | A1 |
| 20070055525 | Kennewick | Mar 2007 | A1 |
| 20070060114 | Ramer | Mar 2007 | A1 |
| 20070061067 | Zeinstra | Mar 2007 | A1 |
| 20070061735 | Hoffberg | Mar 2007 | A1 |
| 20070067310 | Gupta | Mar 2007 | A1 |
| 20070073544 | Millett | Mar 2007 | A1 |
| 20070078708 | Yu | Apr 2007 | A1 |
| 20070078709 | Rajaram | Apr 2007 | A1 |
| 20070078814 | Flowers | Apr 2007 | A1 |
| 20070094003 | Huang | Apr 2007 | A1 |
| 20070100797 | Thun | May 2007 | A1 |
| 20070106499 | Dahlgren | May 2007 | A1 |
| 20070112555 | Lavi | May 2007 | A1 |
| 20070112630 | Lau | May 2007 | A1 |
| 20070118357 | Kasravi | May 2007 | A1 |
| 20070124057 | Prieto | May 2007 | A1 |
| 20070135101 | Ramati | Jun 2007 | A1 |
| 20070146833 | Satomi | Jun 2007 | A1 |
| 20070162296 | Altberg | Jul 2007 | A1 |
| 20070174258 | Jones | Jul 2007 | A1 |
| 20070179778 | Gong | Aug 2007 | A1 |
| 20070185859 | Flowers | Aug 2007 | A1 |
| 20070186165 | Maislos | Aug 2007 | A1 |
| 20070192309 | Fischer | Aug 2007 | A1 |
| 20070198267 | Jones | Aug 2007 | A1 |
| 20070203699 | Nagashima | Aug 2007 | A1 |
| 20070203736 | Ashton | Aug 2007 | A1 |
| 20070208732 | Flowers | Sep 2007 | A1 |
| 20070214182 | Rosenberg | Sep 2007 | A1 |
| 20070250901 | McIntire | Oct 2007 | A1 |
| 20070265850 | Kennewick | Nov 2007 | A1 |
| 20070266257 | Camaisa | Nov 2007 | A1 |
| 20070276651 | Bliss | Nov 2007 | A1 |
| 20070294615 | Sathe | Dec 2007 | A1 |
| 20070299824 | Pan | Dec 2007 | A1 |
| 20080010135 | Schrock | Jan 2008 | A1 |
| 20080014908 | Vasant | Jan 2008 | A1 |
| 20080032622 | Kopra | Feb 2008 | A1 |
| 20080034032 | Healey | Feb 2008 | A1 |
| 20080046311 | Shahine | Feb 2008 | A1 |
| 20080059188 | Konopka | Mar 2008 | A1 |
| 20080065386 | Cross | Mar 2008 | A1 |
| 20080065389 | Cross | Mar 2008 | A1 |
| 20080065390 | Ativanichayaphong | Mar 2008 | A1 |
| 20080086455 | Meisels | Apr 2008 | A1 |
| 20080091406 | Baldwin | Apr 2008 | A1 |
| 20080103761 | Printz | May 2008 | A1 |
| 20080103781 | Wasson | May 2008 | A1 |
| 20080104071 | Pragada | May 2008 | A1 |
| 20080109285 | Reuther | May 2008 | A1 |
| 20080115163 | Gilboa | May 2008 | A1 |
| 20080126091 | Clark | May 2008 | A1 |
| 20080126284 | Forbes | May 2008 | A1 |
| 20080133215 | Sarukkai | Jun 2008 | A1 |
| 20080140385 | Mahajan | Jun 2008 | A1 |
| 20080147396 | Wang | Jun 2008 | A1 |
| 20080147410 | Odinak | Jun 2008 | A1 |
| 20080147637 | Li | Jun 2008 | A1 |
| 20080154604 | Sathish | Jun 2008 | A1 |
| 20080162471 | Bernard | Jul 2008 | A1 |
| 20080177530 | Cross | Jul 2008 | A1 |
| 20080184164 | Di Fabbrizio | Jul 2008 | A1 |
| 20080189110 | Freeman | Aug 2008 | A1 |
| 20080189187 | Hao | Aug 2008 | A1 |
| 20080228496 | Yu | Sep 2008 | A1 |
| 20080235023 | Kennewick | Sep 2008 | A1 |
| 20080235027 | Cross | Sep 2008 | A1 |
| 20080269958 | Filev | Oct 2008 | A1 |
| 20080270135 | Goel | Oct 2008 | A1 |
| 20080270224 | Portman | Oct 2008 | A1 |
| 20080294437 | Nakano | Nov 2008 | A1 |
| 20080294994 | Kruger | Nov 2008 | A1 |
| 20080306743 | Di Fabbrizio | Dec 2008 | A1 |
| 20080319751 | Kennewick | Dec 2008 | A1 |
| 20090006077 | Keaveney | Jan 2009 | A1 |
| 20090006194 | Sridharan | Jan 2009 | A1 |
| 20090018829 | Kuperstein | Jan 2009 | A1 |
| 20090024476 | Baar | Jan 2009 | A1 |
| 20090030686 | Weng | Jan 2009 | A1 |
| 20090052635 | Jones | Feb 2009 | A1 |
| 20090055176 | Hu | Feb 2009 | A1 |
| 20090067599 | Agarwal | Mar 2009 | A1 |
| 20090076827 | Bulitta | Mar 2009 | A1 |
| 20090106029 | DeLine | Apr 2009 | A1 |
| 20090117885 | Roth | May 2009 | A1 |
| 20090144131 | Chiu | Jun 2009 | A1 |
| 20090144271 | Richardson | Jun 2009 | A1 |
| 20090150156 | Kennewick | Jun 2009 | A1 |
| 20090157382 | Bar | Jun 2009 | A1 |
| 20090164216 | Chengalvarayan | Jun 2009 | A1 |
| 20090171664 | Kennewick | Jul 2009 | A1 |
| 20090171912 | Nash | Jul 2009 | A1 |
| 20090197582 | Lewis | Aug 2009 | A1 |
| 20090216540 | Tessel | Aug 2009 | A1 |
| 20090248565 | Chuang | Oct 2009 | A1 |
| 20090248605 | Mitchell | Oct 2009 | A1 |
| 20090259561 | Boys | Oct 2009 | A1 |
| 20090259646 | Fujita | Oct 2009 | A1 |
| 20090265163 | Li | Oct 2009 | A1 |
| 20090271194 | Davis | Oct 2009 | A1 |
| 20090273563 | Pryor | Nov 2009 | A1 |
| 20090276700 | Anderson | Nov 2009 | A1 |
| 20090287680 | Paek | Nov 2009 | A1 |
| 20090299745 | Kennewick | Dec 2009 | A1 |
| 20090299857 | Brubaker | Dec 2009 | A1 |
| 20090304161 | Pettyjohn | Dec 2009 | A1 |
| 20090307031 | Winkler | Dec 2009 | A1 |
| 20090313026 | Coffman | Dec 2009 | A1 |
| 20090319517 | Guha | Dec 2009 | A1 |
| 20100023320 | Cristo | Jan 2010 | A1 |
| 20100023331 | Duta | Jan 2010 | A1 |
| 20100029261 | Mikkelsen | Feb 2010 | A1 |
| 20100036967 | Caine | Feb 2010 | A1 |
| 20100049501 | Kennewick | Feb 2010 | A1 |
| 20100049514 | Kennewick | Feb 2010 | A1 |
| 20100057443 | Cristo | Mar 2010 | A1 |
| 20100063880 | Atsmon | Mar 2010 | A1 |
| 20100064025 | Nelimarkka | Mar 2010 | A1 |
| 20100076778 | Kondrk | Mar 2010 | A1 |
| 20100094707 | Freer | Apr 2010 | A1 |
| 20100138300 | Wallis | Jun 2010 | A1 |
| 20100145700 | Kennewick | Jun 2010 | A1 |
| 20100185512 | Borger | Jul 2010 | A1 |
| 20100191856 | Gupta | Jul 2010 | A1 |
| 20100204986 | Kennewick | Aug 2010 | A1 |
| 20100204994 | Kennewick | Aug 2010 | A1 |
| 20100217604 | Baldwin | Aug 2010 | A1 |
| 20100268536 | Suendermann | Oct 2010 | A1 |
| 20100286985 | Kennewick | Nov 2010 | A1 |
| 20100299142 | Freeman | Nov 2010 | A1 |
| 20100312547 | Van Os | Dec 2010 | A1 |
| 20100312566 | Odinak | Dec 2010 | A1 |
| 20100318357 | Istvan | Dec 2010 | A1 |
| 20100331064 | Michelstein | Dec 2010 | A1 |
| 20110022393 | Waller | Jan 2011 | A1 |
| 20110106527 | Chiu | May 2011 | A1 |
| 20110112827 | Kennewick | May 2011 | A1 |
| 20110112921 | Kennewick | May 2011 | A1 |
| 20110119049 | Ylonen | May 2011 | A1 |
| 20110131036 | DiCristo | Jun 2011 | A1 |
| 20110131045 | Cristo | Jun 2011 | A1 |
| 20110231182 | Weider | Sep 2011 | A1 |
| 20110231188 | Kennewick | Sep 2011 | A1 |
| 20110238409 | Larcheveque | Sep 2011 | A1 |
| 20110307167 | Taschereau | Dec 2011 | A1 |
| 20120022857 | Baldwin | Jan 2012 | A1 |
| 20120041753 | Dymetman | Feb 2012 | A1 |
| 20120046935 | Nagao | Feb 2012 | A1 |
| 20120101809 | Kennewick | Apr 2012 | A1 |
| 20120101810 | Kennewick | Apr 2012 | A1 |
| 20120109753 | Kennewick | May 2012 | A1 |
| 20120150620 | Mandyam | Jun 2012 | A1 |
| 20120150636 | Freeman | Jun 2012 | A1 |
| 20120239498 | Ramer | Sep 2012 | A1 |
| 20120240060 | Pennington | Sep 2012 | A1 |
| 20120265528 | Gruber | Oct 2012 | A1 |
| 20120278073 | Weider | Nov 2012 | A1 |
| 20130006734 | Ocko | Jan 2013 | A1 |
| 20130054228 | Baldwin | Feb 2013 | A1 |
| 20130060625 | Davis | Mar 2013 | A1 |
| 20130080177 | Chen | Mar 2013 | A1 |
| 20130211710 | Kennewick | Aug 2013 | A1 |
| 20130253929 | Weider | Sep 2013 | A1 |
| 20130254314 | Chow | Sep 2013 | A1 |
| 20130297293 | Cristo | Nov 2013 | A1 |
| 20130304473 | Baldwin | Nov 2013 | A1 |
| 20130311324 | Stoll | Nov 2013 | A1 |
| 20130332454 | Stuhec | Dec 2013 | A1 |
| 20130339022 | Baldwin | Dec 2013 | A1 |
| 20140006951 | Hunter | Jan 2014 | A1 |
| 20140012577 | Freeman | Jan 2014 | A1 |
| 20140025371 | Min | Jan 2014 | A1 |
| 20140108013 | Cristo | Apr 2014 | A1 |
| 20140136187 | Wolverton | May 2014 | A1 |
| 20140136259 | Kinsey, II | May 2014 | A1 |
| 20140156278 | Kennewick | Jun 2014 | A1 |
| 20140195238 | Terao | Jul 2014 | A1 |
| 20140236575 | Tur | Aug 2014 | A1 |
| 20140249821 | Kennewick | Sep 2014 | A1 |
| 20140249822 | Baldwin | Sep 2014 | A1 |
| 20140278413 | Pitschel | Sep 2014 | A1 |
| 20140278416 | Schuster | Sep 2014 | A1 |
| 20140288934 | Kennewick | Sep 2014 | A1 |
| 20140330552 | Bangalore | Nov 2014 | A1 |
| 20140337007 | Waibel | Nov 2014 | A1 |
| 20140365222 | Weider | Dec 2014 | A1 |
| 20150019211 | Simard | Jan 2015 | A1 |
| 20150019217 | Cristo | Jan 2015 | A1 |
| 20150019227 | Anandarajah | Jan 2015 | A1 |
| 20150066479 | Pasupalak | Mar 2015 | A1 |
| 20150066627 | Freeman | Mar 2015 | A1 |
| 20150073910 | Kennewick | Mar 2015 | A1 |
| 20150095159 | Kennewick | Apr 2015 | A1 |
| 20150142447 | Kennewick | May 2015 | A1 |
| 20150170641 | Kennewick | Jun 2015 | A1 |
| 20150193379 | Mehta | Jul 2015 | A1 |
| 20150199339 | Mirkin | Jul 2015 | A1 |
| 20150228276 | Baldwin | Aug 2015 | A1 |
| 20150293917 | Bute | Oct 2015 | A1 |
| 20150348544 | Baldwin | Dec 2015 | A1 |
| 20150348551 | Gruber | Dec 2015 | A1 |
| 20150364133 | Freeman | Dec 2015 | A1 |
| 20160049152 | Kennewick | Feb 2016 | A1 |
| 20160078482 | Kennewick | Mar 2016 | A1 |
| 20160078491 | Kennewick | Mar 2016 | A1 |
| 20160078504 | Kennewick | Mar 2016 | A1 |
| 20160078773 | Carter | Mar 2016 | A1 |
| 20160110347 | Kennewick | Apr 2016 | A1 |
| 20160148610 | Kennewick | May 2016 | A1 |
| 20160148612 | Guo | May 2016 | A1 |
| 20160188292 | Carter | Jun 2016 | A1 |
| 20160188573 | Tang | Jun 2016 | A1 |
| 20160217785 | Kennewick | Jul 2016 | A1 |
| 20160335676 | Freeman | Nov 2016 | A1 |
| 20170004588 | Isaacson | Jan 2017 | A1 |
| Number | Date | Country |
|---|---|---|
| 1433554 | Jul 2003 | CN |
| 1860496 | Nov 2006 | CN |
| 101236635 | Aug 2008 | CN |
| 103229206 | Jul 2013 | CN |
| 1320043 | Jun 2003 | EP |
| 1646037 | Apr 2006 | EP |
| 2867583 | Sep 2005 | FR |
| H08263258 | Oct 1996 | JP |
| H11249773 | Sep 1999 | JP |
| 2001071289 | Mar 2001 | JP |
| 2002297626 | Oct 2002 | JP |
| 2006146881 | Jun 2006 | JP |
| 2008027454 | Feb 2008 | JP |
| 2008058465 | Mar 2008 | JP |
| 2008139928 | Jun 2008 | JP |
| 2011504304 | Feb 2011 | JP |
| 2012518847 | Aug 2012 | JP |
| 9946763 | Sep 1999 | WO |
| 0021232 | Apr 2000 | WO |
| 0046792 | Aug 2000 | WO |
| 0129742 | Apr 2001 | WO |
| 0129742 | Apr 2001 | WO |
| 0171609 | Sep 2001 | WO |
| 0178065 | Oct 2001 | WO |
| 2004072954 | Aug 2004 | WO |
| 2005010702 | Feb 2005 | WO |
| 2007019318 | Jan 2007 | WO |
| 2007021587 | Jan 2007 | WO |
| 2007027546 | Jan 2007 | WO |
| 2007027989 | Jan 2007 | WO |
| 2008098039 | Jan 2008 | WO |
| 2008118195 | Jan 2008 | WO |
| 2009075912 | Jan 2009 | WO |
| 2009145796 | Jan 2009 | WO |
| 2009111721 | Sep 2009 | WO |
| 2010096752 | Jan 2010 | WO |
| 2016044290 | Mar 2016 | WO |
| 2016044316 | Mar 2016 | WO |
| 2016044319 | Mar 2016 | WO |
| 2016044321 | Mar 2016 | WO |
| 2016061309 | Apr 2016 | WO |
| Entry |
|---|
| “Statement in Accordance with the Notice from the European Patent Office” dated Oct. 1, 2007 Concerning Business Methods (OJ EPO Nov. 2007, 592-593), XP002456252. |
| Arrington, Michael, “Google Redefines GPS Navigation Landscape: Google Maps Navigation for Android 2.0”, TechCrunch, printed from the Internet <http://www.techcrunch.com/2009/10/28/google-redefines-car-gps-navigation-google-maps-navigation-android/>, Oct. 28, 2009, 4 pages. |
| Bazzi, Issam et al., “Heterogeneous Lexical Units for Automatic Speech Recognition: Preliminary Investigations”, Processing of the IEEE International Conference on Acoustics, Speech, and Signal Processing, vol. 3, Jun. 5-9, 2000, XP010507574, pp. 1257-1260. |
| Belvin, Robert, et al., “Development of the HRL Route Navigation Dialogue System”, Proceedings of the First International Conference on Human Language Technology Research, San Diego, 2001, pp. 1-5. |
| Chai et al., “MIND: A Semantics-Based Multimodal Interpretation Framework for Conversational System”, Proceedings of the International CLASS Workshop on Natural, Intelligent and Effective Interaction in Multimodal Dialogue Systems, Jun. 2002, pp. 37-46. |
| Cheyer et al., “Multimodal Maps: An Agent-Based Approach”, International Conference on Cooperative Multimodal Communication (CMC/95), May 24-26, 1995, pp. 111-121. |
| Davis, Z., et al., A Personal Handheld Multi-Modal Shopping Assistant, IEEE, 2006, 9 pages. |
| El Meliani et al., “A Syllabic-Filler-Based Continuous Speech Recognizer for Unlimited Vocabulary”, Canadian Conference on Electrical and Computer Engineering, vol. 2, Sep. 5-8, 1995, pp. 1007-1010. |
| Elio et al., “On Abstract Task Models and Conversation Policies” in Workshop on Specifying and Implementing Conversation Policies, Autonomous Agents '99, Seattle, 1999, 10 pages. |
| Kirchhoff, Katrin, “Syllable-Level Desynchronisation of Phonetic Features for Speech Recognition”, Proceedings of the Fourth International Conference on Spoken Language, 1996, ICSLP 96, vol. 4, IEEE, 1996, 3 pages. |
| Kuhn, Thomas, et al., “Hybrid In-Car Speech Recognition for Mobile Multimedia Applications”, Vehicular Technology Conference, IEEE, Jul. 1999, pp. 2009-2013. |
| Lin, Bor-shen, et al., “A Distributed Architecture for Cooperative Spoken Dialogue Agents with Coherent Dialogue State and History”, ASRU'99, 1999, 4 pages. |
| Lind, R., et al., “The Network Vehicle—A Glimpse into the Future of Mobile Multi-Media”, IEEE Aerosp. Electron. Systems Magazine, vol. 14, No. 9, Sep. 1999, pp. 27-32. |
| Miao, Mark Z., “Automatic Training Set Segmentation for Multi-Pass Speech Recognition”, Department of Electrical Engineering, Stanford University, CA, copyright 2005, IEEE, pp. I-685 to I-688. |
| O'Shaughnessy, Douglas, “Interacting with Computers by Voice: Automatic Speech Recognition and Synthesis”, Proceedings of the IEEE, vol. 91, No. 9, Sep. 1, 2003, XP011100665, pp. 1272-1305. |
| Reuters, “IBM to Enable Honda Drivers to Talk to Cars”, Charles Schwab & Co., Inc., Jul. 28, 2002, 1 page. |
| Turunen, “Adaptive Interaction Methods in Speech User Interfaces”, Conference on Human Factors in Computing Systems, Seattle, Washington, 2001, pp. 91-92. |
| Vanhoucke, Vincent, “Confidence Scoring and Rejection Using Multi-Pass Speech Recognition”, Nuance Communications, Menlo Park, CA, 2005, 4 pages. |
| Weng, Fuliang, et al., “Efficient Lattice Representation and Generation”, Speech Technology and Research Laboratory, SRI International, Menlo Park, CA, 1998, 4 pages. |
| Wu, Su-Lin, et al., “Incorporating Information from Syllable-Length Time Scales into Automatic Speech Recognition”, Proceedings of the 1998 IEEE International Conference on Acoustics, Speech and Signal Processing, 1998, vol. 2, IEEE, 1998, 4 pages. |
| Wu, Su-Lin, et al., “Integrating Syllable Boundary Information into Speech Recognition”, IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP-97, 1997, vol. 2, IEEE, 1997, 4 pages. |
| Zhao, Yilin, “Telematics: Safe and Fun Driving”, IEEE Intelligent Systems, vol. 17, Issue 1, 2002, pp. 10-14. |
| Office Action issued in Chinese Patent Application No. 201580060519.1 dated Mar. 13, 2020, with its English translation, 40 pages. |
| Amazon.com, Inc., Amazon.com LLC, Amazon Web Services, Inc., A2Z Development Center, Inc. d/b/a/ Lab126, Rawles LLC, Amzn Mobile LLC, Amzn Mobile 2 LLC, Amazon.com Services, Inc. f/k/a Amazon Fulfillment Services, Inc., and Amazon.com Services LLC (formerly Amazon Digital Services LLC) v. VB Assets, LLC, IPR2020-01346, Decision Granting Institution of Inter Partes Review of U.S. Pat. No. 9,015,049 B2, entered Feb. 4, 2021, 24 pages. |
| Amazon.com, Inc., Amazon.com LLC, Amazon Web Services, Inc., A2Z Development Center, Inc. d/b/a/ Lab126, Rawles LLC, Amzn Mobile LLC, Amzn Mobile 2 LLC, Amazon.com Services, Inc. f/k/a Amazon Fulfillment Services, Inc., and Amazon.com Services LLC (formerly Amazon Digital Services LLC) v. VB Assets, LLC, IPR2020-01367, Decision Granting Institution of Inter Partes Review of U.S. Pat. No. 8,073,681 B2, entered Mar. 11, 2021, 22 pages. |
| Amazon.com, Inc., Amazon.com LLC, Amazon Web Services, Inc., A2Z Development Center, Inc. d/b/a/ Lab126, Rawles LLC, Amzn Mobile LLC, Amzn Mobile 2 LLC, Amazon.com Services, Inc. f/k/a Amazon Fulfillment Services, Inc., and Amazon.com Services LLC (formerly Amazon Digital Services LLC) v. VB Assets, LLC, IPR2020-01374, Decision Denying Institution of Inter Partes Review of U.S. Pat. No. 8,886,536 B2, entered Feb. 4, 2021, 37 pages. |
| Amazon.com, Inc., Amazon.com LLC, Amazon Web Services, Inc., A2Z Development Center, Inc. d/b/a/ Lab126, Rawles LLC, Amzn Mobile LLC, Amzn Mobile 2 LLC, Amazon.com Services, Inc. f/k/a Amazon Fulfillment Services, Inc., and Amazon.com Services LLC (formerly Amazon Digital Services LLC) v. VB Assets, LLC, IPR2020-01377, Decision Denying Institution of Inter Partes Review of U.S. Pat. No. 8,886,536 B2, entered Feb. 4, 2021, 29 pages. |
| Amazon.com, Inc., Amazon.com LLC, Amazon Web Services, Inc., A2Z Development Center, Inc. d/b/a/ Lab126, Rawles LLC, Amzn Mobile LLC, Amzn Mobile 2 LLC, Amazon.com Services, Inc. f/k/a Amazon Fulfillment Services, Inc., and Amazon.com Services LLC (formerly Amazon Digital Services LLC) v. VB Assets, LLC, IPR2020-01380, Decision Denying Institution of Inter Partes Review of U.S. Pat. No. 9,626,703 B2, entered Feb. 4, 2021, 28 pages. |
| Amazon.com, Inc., Amazon.com LLC, Amazon Web Services, Inc., A2Z Development Center, Inc. d/b/a/ Lab126, Rawles LLC, Amzn Mobile LLC, Amzn Mobile 2 LLC, Amazon.com Services, Inc. f/k/a Amazon Fulfillment Services, Inc., and Amazon.com Services LLC (formerly Amazon Digital Services LLC) v. VB Assets, LLC, IPR2020-01381, Decision Denying Institution of Inter Partes Review of U.S. Pat. No. 9,626,703 B2, entered Feb. 4, 2021, 29 pages. |
| Amazon.vom, Inc., Amazon.com LLC, Amazon Web Services, Inc., A2Z Development Center, Inc. d/b/a/ Lab126, Rawles LLC, Amzn Mobile LLC, Amzn Mobile 2 LLC, Amazon.com Services, Inc. f/k/a Amazon Fulfillment Services, Inc., and Amazon.com Services LLC (formerly Amazon Digital Services LLC) v. VB Assets, LLC, IPR2020-01388, Decision Denying Institution of Inter Partes Review of U.S. Pat. No. 9,269,097 B2, entered Feb. 24, 2021, 25 pages. |
| Amazon.com, Inc., Amazon.com LLC, Amazon Web Services, Inc., A2Z Development Center, Inc. d/b/a/ Lab126, Rawles LLC, Amzn Mobile LLC, Amzn Mobile 2 LLC, Amazon.com Services, Inc. f/k/a Amazon Fulfillment Services, Inc., and Amazon.com Services LLC (formerly Amazon Digital Services LLC) v. VB Assets, LLC, IPR2020-01390, Decision Denying Institution of Inter Partes Review of U.S. Pat. No. 7,818,176, entered Mar. 11, 2021, 27 pages. |
| Asthana, A., et al., “A Small Domain Communications System for Personalized Shopping Assistance”, Proceedings of 1994 International Conference on Personal Wireless Communications, IEEE Press, Aug. 1994, pp. 199-203. |
| Huang et al., “Spoken Language Processing: A Guide to Theory, Algorithm, and System Development”, Prentice Hall, 2001, 1010 pages. |
| IEEE 100 The Authoritative Dictionary of IEEE Standards Terms, Seventh Edition, Standards Information Network, IEEE Press, Print ISBN 0-7381-2601-2, Published Dec. 2000, 3 pages. |
| Lucente, Mark, “Conversational Interfaces for E-Commerce Applications”, Communications of the ACM, vol. 43, No. 9, Sep. 2000, pp. 59-61. |
| Seneff, Stephanie, et al., “Hypothesis Selection and Resolution in the Mercury Flight Reservation System”, Spoken Language Systems Group, MIT, 2001, 8 pages. |
| Number | Date | Country | |
|---|---|---|---|
| 20190272821 A1 | Sep 2019 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | 14691445 | Apr 2015 | US |
| Child | 16416884 | US | |
| Parent | 13251712 | Oct 2011 | US |
| Child | 13987645 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | 13987645 | Aug 2013 | US |
| Child | 14691445 | US | |
| Parent | 11580926 | Oct 2006 | US |
| Child | 13251712 | US |
