Claims
- 1. A method of interacting with a user on a computer device, the computer device being capable of receiving a plurality of types of user input and being capable of presenting information in a plurality of types of device output, the method comprising:
(1) receiving a user query in one of the plurality of types of user input; (2) presenting data to the user related to the user query; (3) receiving a second user query associated with the presented data in one of the plurality of types of user input; and (4) presenting a response to the user query or the second user query.
- 2. The method of claim 1, wherein the plurality of types of user input comprises user input via speech, pen, and multi-modally.
- 3. The method of claim 1, wherein the plurality of types of user input comprises speech, text-based pen graphics, and a combination of speech and gestures.
- 4. The method of claim 2, wherein the plurality of types of device output comprises synthesized speech, graphics and a combination of speech and graphics.
- 5. The method of claim 2, wherein multi-modally comprises a combination of speech and gestures.
- 6. The method of claim 1, wherein one of the plurality of types of user input comprises speech and gestures.
- 7. The method of claim 1, wherein the user query relates to a request for a set of businesses within an area.
- 8. The method of claim 7, wherein presenting data to the user related to the user query further comprises presenting a graphical presentation of the set of businesses within the area.
- 9. The method of claim 8, wherein the set of businesses are restaurants.
- 10. The method of claim 8, wherein the set of businesses are retail stores.
- 11. The method of claim 8, wherein the set of businesses are tourist sites.
- 12. The method of claim 8, wherein the set of businesses are theatres.
- 13. The method of claim 12, where in the set of businesses are movie theatres.
- 14. A method of providing information associated with a map to a user via interaction with a computer device, the computer device being capable of receiving a plurality of types of user input comprising speech, pen or multi-modally, the method comprising:
(1) receiving a user query in speech, pen or multi-modally; (2) presenting data to the user related to the user query; (3) receiving a second user query associated with the presented data in one of the plurality of types of user input; and (4) presenting a response to the user query or the second user query.
- 15. The method of claim 14, where multi-modally comprises a combination of speech and gestures.
- 16. The method of claim 14, wherein the response to the user query or the second user query comprises a combination of speech and graphics.
- 17. The method of claim 14, wherein multi-modally includes a combination of speech and handwriting.
- 18. The method of claim 14, wherein the user query relates to a request for a set of businesses within an area.
- 19. The method of claim 14, wherein presenting data to the user related to the user query further comprises presenting a graphical presentation of a set of businesses within the area.
- 20. The method of claim 19, wherein the set of businesses are restaurants.
- 21. The method of claim 19, wherein the set of businesses are retail stores.
- 22. The method of claim 19, wherein the set of businesses are tourist sites.
- 23. The method of claim 19, wherein the set of business are theaters.
- 24. The method of claim 23, wherein the set of businesses are movie theaters.
- 25. A method of providing information to a user via interaction with a computer device, the computer device being capable of receiving user input via speech, pen or multi-modally, the method comprising:
(1) receiving a user business entity query in speech, pen or multi-modally, the user business entity query including a query related to a business location; and (2) presenting a response to the user business entity query.
- 26. The method of claim 25, further comprising, after presenting a response to the user business entity query:
(3) receiving a second user query related to the presented response; and (4) presenting a second response addressing the second user query.
- 27. The method of claim 25, wherein multi-modally comprises a combination of speech and gestures.
- 28. The method of claim 25, wherein multi-modally comprises a combination of speech and handwriting.
- 29. The method of claim 25, wherein presenting a response to the user business entity query further comprises:
graphically illustrating information associated with the user business query; and presenting synthetic speech providing information regarding the graphical information.
- 30. The method of claim 26, wherein presenting a second response addressing the second user query further comprises:
graphically illustrating second information associated with the second user query; and presenting synthetic speech providing information regarding the graphical second information.
- 31. The method of claim 25, wherein the business entity is a restaurant.
- 32. The method of claim 25, wherein the business entity is a retail shop.
- 33. The method of claim 25, wherein the business entity is a tourist site.
- 34. A method of providing business-related information to a user on a computer device, the computer device being capable of receiving input either via speech, pen, or multi-modally, the method comprising:
(1) receiving a user query regarding a business either via speech, pen or multi-modally, the user query including a location component; and (2) in response to the user query, presenting on a map display information associated with the user query.
- 35. The method of claim 34, further comprising, after presenting on a map display information associated with the user query:
(3) receiving a second user query associated with the displayed information; (4) in response to the second user query, presenting on the map display information associated with the second user query.
- 36. The method of claim 34, further comprising:
providing synthetic speech associated with the information presented on the map display in response to the user query.
- 37. The method of claim 35, further comprising:
providing synthetic speech associated with the information presented on the map display in response to the second user query.
- 38. An apparatus for interacting with a user, the apparatus storing a multi-modal recognition module using a finite-state machine to build a single meaning representation from a plurality of types of user input, the apparatus comprising:
(1) means for receiving a user query in one of the plurality of types of user input; (2) means for presenting information on a map display related to the user query; (3) means for receiving further user input in one of the plurality of types of user input; and (4) means for presenting a response to the user query.
- 39. An apparatus for receiving multi-modal input from a user, the apparatus comprising:
a user interface module; a speech recognition module; a gesture recognition module; an integrator module; a facilitator module that communicates with the user interface module, the speech recognition module, the gesture recognition module and the integrator module, wherein the apparatus receives user input as speech through the speech recognition module, gestures through the gesture recognition module, or a combination of speech and gestures through the integrator module, processes the user input, and generates a response to the user input through the facilitator module and the user interface module.
PRIORITY APPLICATION
[0001] The present invention claims priority to provisional Patent Application No. 60/370,044, filed Apr. 3, 2002, the contents of which are incorporated herein by reference. The present invention claims priority to provisional Patent Application No. 60/313,121, filed Aug. 17, 2001, the contents of which are incorporated herein by reference.
Provisional Applications (2)
|
Number |
Date |
Country |
|
60370044 |
Apr 2002 |
US |
|
60313121 |
Aug 2001 |
US |