MULTI-MODAL CONTENT PRESENTATION

Abstract

A method is provided that includes receiving a user input, the user input having been input in a user interface in one of multiple modalities. The method also includes accessing, in response to receiving the user input, a multi-modality content document including content information and presentation information, the presentation information supporting presentation of the content information in each of the multiple modalities. In addition, the method includes accessing, in response to receiving the user input, metadata for the user interface, the metadata indicating that the user interface provides a first modality and a second modality for interfacing with a user. First-modality instructions are generated based on the accessed multi-modality content document and the accessed metadata, the first-modality instructions providing instructions for presenting the content information on the user interface using the first modality. Second-modality instructions are generated based on the accessed multi-modality content document and the accessed metadata, the second-modality instructions providing instructions for presenting the content information on the user interface using the second modality.

Description

DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram of an example of a system for presenting multi-modality content within a user interface.

FIG. 2 is a flow chart of an example of a method of operations that may be performed to present multi-modality content to a user.

FIG. 3A is an example of a first portion of a source multi-modality content document, the first portion including presentation information.

FIG. 3B is an example of a second portion of the source multi-modality content document referred to in FIG. 3A, the second portion including content information.

FIG. 4 is an example of a set of instructions for presenting the multi-modality content information of FIG. 3B using a visual modality of a user interface, in accordance with the presentation information of FIG. 3A.

FIG. 5 is an example of a presentation of the multi-modality content using the visual modality of the user interface based on the instructions of FIG. 4.

FIG. 6 is an example of a set of instructions for presenting the multi-modality content information of FIG. 3B using a voice input/output modality of a user interface, in accordance with the presentation information of FIG. 3A.

FIG. 7 is an example of a set of instructions for presenting the multi-modality content information of FIG. 3B using a voice output modality of a user interface, in accordance with the presentation information of FIG. 3A.

FIG. 8 is an example of a schematic diagram of a first pagination scenario for presenting multi-modality content using visual and voice modalities of a user interface.

FIG. 9 is an example of a schematic diagram of a second pagination scenario for presenting multi-modality content using visual and voice modalities of a user interface.

FIG. 10 is an example of a schematic diagram of a third pagination scenario for presenting multi-modality content using visual and voice modalities of a user interface.

FIG. 11 is an example of a schematic diagram of a fourth pagination scenario for presenting multi-modality content using visual and voice modalities of a user interface.

FIG. 12 is an example of a set of instructions for presenting a first paginated portion of the multi-modality content of FIG. 8 using a visual modality of a user interface.

FIG. 13 is an example of a presentation of the first paginated portion of the multi-modality content of FIG. 8 based on the instructions of FIG. 12 using the visual modality of the user interface.

FIG. 14 is an example of a set of instructions for presenting a first paginated portion of the multi-modality content of FIG. 8 using a voice input/output modality of a user interface.

FIG. 15 is an example of a set of instructions for presenting a first paginated portion of the multi-modality content of FIG. 8 using a voice output modality of a user interface.

FIG. 16 is an example of a set of instructions for presenting a second paginated portion of the multi-modality content of FIG. 8 using a visual modality of a user interface.

FIG. 17 is an example of a presentation of the second paginated portion of the multi-modality content of FIG. 8 based on the instructions of FIG. 16 using the visual modality of the user interface.

FIG. 18 is an example of a set of instructions for presenting a second paginated portion of the multi-modality content of FIG. 8 using a voice input/output modality of a user interface.

FIG. 19 is an example of a set of instructions for presenting a second paginated portion of the multi-modality content of FIG. 8 using a voice output modality of a user interface.

Claims

1. A method comprising: receiving a user input, the user input having been input in a user interface in one of multiple modalities;accessing, in response to receiving the user input, a multi-modality content document including content information and presentation information, the presentation information supporting presentation of the content information in each of the multiple modalities;accessing, in response to receiving the user input, metadata for the user interface, the metadata indicating that the user interface provides a first modality and a second modality for interfacing with a user;generating first-modality instructions based on the accessed multi-modality content document and the accessed metadata, the first-modality instructions providing instructions for presenting the content information on the user interface using the first modality; andgenerating second-modality instructions based on the accessed multi-modality content document and the accessed metadata, the second-modality instructions providing instructions for presenting the content information on the user interface using the second modality.

2. The method of claim 1 wherein: the content information includes a first content portion and a second content portion,the metadata includes a user interface attribute related to pagination, andgenerating the first-modality instructions and the second-modality instructions comprises: determining a particular first-modality pagination describing pagination between the first content portion and the second content portion during presentation on the user interface using the first modality;determining a particular second-modality pagination describing pagination between the first content portion and the second content portion during presentation on the user interface using the second modality, wherein the particular first-modality pagination and the particular second-modality pagination are determined based on (i) desired order of presentation of the first content portion and the second content portion in the first modality on the user interface, (ii) desired order of presentation of the first content portion and the second content portion in the second modality on the user interface, and (iii) the user interface attribute related to pagination;generating, based on the particular first-modality pagination, the first-modality instructions including instructions for (i) presenting the first content portion on the user interface during a first time interval, and (ii) presenting the second content portion on the user interface during a second time interval; andgenerating, based on the particular second-modality pagination, the second-modality instructions including instructions for (i) presenting the first content portion on the user interface during the first time interval, and (ii) presenting the second content portion on the user interface during the second time interval.

3. The method of claim 2 wherein pagination is further based on other metadata.

4. The method of claim 2 wherein the user interface attribute includes a screen size indicator.

5. The method of claim 2 wherein generating the first-modality instructions and the second-modality instructions further comprises: determining multiple first-modality paginations, including the particular first-modality pagination, describing pagination between the first content portion and the second content portion during presentation on the user interface using the first modality; anddetermining multiple second-modality paginations, including the particular second-modality pagination, describing pagination between the first content portion and the second content portion during presentation on the user interface using the second modality, wherein each of the multiple first-modality paginations and second-modality paginations are determined based on (i) desired order of presentation of the first content portion and the second content portion in the first modality on the user interface, (ii) desired order of presentation of the first content portion and the second content portion in the second modality on the user interface, and (iii) the user interface attribute related to pagination;forming a first pagination scenario, the forming including associating with a first of the multiple first-modality paginations a first of the multiple second-modality paginations;determining for the first pagination scenario a first distance characterizing whether (i) the first content portion is configured to be presented on the user interface during a first common time interval using both the first modality and the second modality and (ii) the second content portion is configured to be presented on the user interface during a second common time interval using both the first modality and the second modality;forming a second pagination scenario, the forming including associating with a second of the multiple first-modality paginations a second of the multiple second-modality paginations;determining for the second pagination scenario a second distance characterizing whether (i) the first content portion is configured to be presented on the user interface during a first common time interval using both the first modality and the second modality and (ii) the second content portion is configured to be presented on the user interface during a second common time interval using both the first modality and the second modality;comparing the first distance with the second distance; andselecting a pagination scenario from the first pagination scenario and the second pagination scenario based on a result of the comparing.

6. The method of claim 5 wherein: the content information is divided into multiple elements,the first content portion includes at least a first of the multiple elements,the second content portion includes at least a second of the multiple elements,the first pagination scenario dictates when elements of the first content portion and elements of the second content portion are presented,determining the first distance comprises determining, for the first pagination scenario, a number of element-inserts and a number of element-removals that are required in order that (i) the first content portion is configured to be presented on the user interface during a first common time interval using both the first modality and the second modality and (ii) the second content portion is configured to be presented on the user interface during a second common time interval using both the first modality and the second modality, andthe first distance is based on the number of element-inserts and the number of element-removals.

7. The method of claim 5 wherein selecting a pagination scenario from the first pagination scenario and the second pagination scenario based on a result of the comparing further includes selecting the pagination scenario based on the minimal pagination distance.

8. The method of claim 1 further comprising providing both the first-modality instructions and the second-modality instructions to the user interface.

9. The method of claim 1 wherein the presentation information of the multi-modality content document includes (i) a first layout container including presentation information for presenting content using the first modality and (ii) a second layout container including presentation information for presenting content using the second modality.

10. The method of claim 1 wherein the presentation information of the multi-modality content document includes a generic layout container including presentation information for presenting content using multiple modalities.

11. The method of claim 1 wherein generating the first-modality instructions comprises a browser generating the first-modality instructions for execution by a processor that controls the user interface.

12. The method of claim 1 wherein generating the first-modality instructions comprises generating a first-modality content document that includes the first-modality instructions.

13. The method of claim 12 wherein generating the first-modality content document comprises generating one or more of a Speech Recognition Grammar Specification (SRGS) document, a Hypertext Markup Language (HTML) document, a Voice Extensible Markup Language (VXML) document, or a Speech Synthesis Markup Language (SSML) document.

14. The method of claim 1 wherein the metadata further indicates one or more of (i) content types supported by the user interface or (ii) scalability of graphics on the user interface.

15. An apparatus comprising: means for receiving a user input, the user input having been input in a user interface in one of multiple modalities;means for accessing, in response to receiving the user input, a multi-modality content document including content information and presentation information, the presentation information supporting presentation of the content information in each of the multiple modalities;means for accessing, in response to receiving the user input, metadata for the user interface, the metadata indicating that the user interface provides a first modality and a second modality for interfacing with a user;means for generating first-modality instructions based on the accessed multi-modality content document and the accessed metadata, the first-modality instructions providing instructions for presenting the content information on the user interface using the first modality; andmeans for generating second-modality instructions based on the accessed multi-modality content document and the accessed metadata, the second-modality instructions providing instructions for presenting the content information on the user interface using the second modality.

16. The apparatus of claim 15 wherein the means for generating the first-modality instructions is disposed local to the user interface.

17. An apparatus comprising a computer-readable medium that includes instructions for performing at least the following: receiving a user input, the user input having been input in a user interface in one of multiple modalities;accessing, in response to receiving the user input, a multi-modality content document including content information and presentation information, the presentation information supporting presentation of the content information in each of the multiple modalities;accessing, in response to receiving the user input, metadata for the user interface, the metadata indicating that the user interface provides a first modality and a second modality for interfacing with a user;generating first-modality instructions based on the accessed multi-modality content document and the accessed metadata, the first-modality instructions providing instructions for presenting the content information on the user interface using the first modality; andgenerating second-modality instructions based on the accessed multi-modality content document and the accessed metadata, the second-modality instructions providing instructions for presenting the content information on the user interface using the second modality.

18. The apparatus of claim 17 wherein the apparatus comprises one or more of a portable storage device or a processing device.