Patent Grant
8499232
1. Field of the Invention
The field of the invention is data processing, or, more specifically, methods, systems, and products for differential dynamic content delivery.
2. Description of Related Art
Multimedia presentations through conferencing systems are becoming more common, but they are inflexible because all conference participants must be presented with exactly the same content. For any particular presentation, however, there is typically a wide variety of participant interest, company, group, or department membership, technical knowledge, security authorization, and so on, across almost any dimension in which participants may vary. Targeting content for such a heterogeneous set of users is typically a manual process today in which presenters create wholly separate presentations for each audience, and the content of each such presentation is reduced to the lowest common denominator of any particular audience. There is a substantial need for improved multimedia presentation systems.
Methods, systems, and products are disclosed that operate generally to support improved multimedia presentations by creating a presentation document that includes a content-specific presentation grammar and a structured document. The structured document typically has structural elements such as pages, paragraphs, cells, titles, and the like marked with structural identifiers. A content-specific presentation grammar ties presentation actions to the document structure through these structural element identifiers. A presentation actions directs the presentation of a document such as by moving the presentation to the next page of the document, the previous paragraph of the document and so on. A presentation grammar empowers a presenter to invoke the presentation actions using speech.
In typical embodiments, users are assigned classifications describing any attributes of a user, company name, department name, age, gender, technical knowledge, educational level, subject matters of personal interest, security authorization, and so on. Contents of structural elements from structured documents are then filtered for presentation to individual users in a multi-media, multi-user presentation according to the individual attributes of the participants.
In a presentation regarding marketing of a deep space vehicle for a Mars mission, for example, graphic images and paragraphs of text may be developed in many versions, inserted into the same presentation document with each version classified according to technical level, security level, and so on, so that a member of the marketing department viewing the same paragraph at the same time in the same presentation as a member of the research department will in fact be shown a different version of the paragraph. A graphic diagram of a subsystem presented to the marketer will be a simpler version than the one shown at the same time to the researcher.
More particularly, methods, systems, and computer program products are disclosed for differential dynamic content delivery with a participant alterable session copy of a user profile. Typical embodiments include providing a session document for a presentation, where the session document includes a session grammar and a session structured document; providing a session copy of a participant's user profile including a user classification; receiving, from the participant, a user classification instruction to change a user classification in the session copy of the participant's user profile; changing the user classification in the session copy of the participant's user profile in dependence upon the participant's user classification instruction; selecting from the session structured document a classified structural element in dependence upon a user classification in the session copy of the participant's user profile in the presentation; and presenting the selected structural element to the participant.
In typical embodiments, only the participant is authorized to change a user classification in the session copy of the participant's user profile. In typical embodiments, providing a session copy of a participant's user profile including a user classification is carried out by copying at least a portion of the participant's user profile. Typical embodiments include creating a user classification instruction in dependence upon rules governing changes in a user classification in a session copy of the participant's user profile. Typical embodiments include receiving a presentation control instruction where the presentation control instruction includes a presentation action identifier and optional parameters and selecting a classified structural element includes selecting a classified structural element in dependence upon the presentation action identifier and the parameters.
Typical embodiments include creating the presentation control instruction, including receiving from a user participating in the presentation a key phrase and optional parameters for invoking a presentation action and parsing the key phrase and parameters against a voice response grammar into a presentation control instruction. In typical embodiments, selecting a classified structural element is carried out by selecting a classified structural element having an associated classification identifier that corresponds to the user classification in the session copy of the participant's user profile.
Typical embodiments include updating a session document, including identifying a presentation document for the presentation, where the presentation document includes a presentation grammar and a structured document having structural elements classified with classification identifiers. Updating a session document typically also includes identifying users for the presentation, where the users for the presentation include the participant, and the participant has a session copy of the participant's user profile comprising user classifications. Updating a session document typically also include filtering the structured document in dependence upon the user classifications of the session copy of the participant's user profile and the classification identifiers. In such embodiments, filtering the structured document typically also includes extracting, from the structured document, structural elements having classification identifiers corresponding to the user classifications of the session copy of the participant's user profile and writing the extracted structural elements into the session structured document in the session document. Such embodiments typically also include filtering the presentation grammar, in dependence upon the extracted structural elements, into a session grammar in the session document.
The foregoing and other objects, features and advantages of the invention will be apparent from the following more particular descriptions of exemplary embodiments of the invention as illustrated in the accompanying drawings wherein like reference numbers generally represent like parts of exemplary embodiments of the invention.
The present invention is described to a large extent in this specification in terms of methods for differential dynamic content delivery. Persons skilled in the art, however, will recognize that any computer system that includes suitable programming means for operating in accordance with the disclosed methods also falls well within the scope of the present invention. Suitable programming means include any means for directing a computer system to execute the steps of the method of the invention, including for example, systems comprised of processing units and arithmetic-logic circuits coupled to computer memory, which systems have the capability of storing in computer memory, which computer memory includes electronic circuits configured to store data and program instructions, programmed steps of the method of the invention for execution by a processing unit.
The invention also may be embodied in a computer program product, such as a diskette or other recording medium, for use with any suitable data processing system. Embodiments of a computer program product may be implemented by use of any recording medium for machine-readable information, including magnetic media, optical media, or other suitable media. Persons skilled in the art will immediately recognize that any computer system having suitable programming means will be capable of executing the steps of the method of the invention as embodied in a program product. Persons skilled in the art will recognize immediately that, although most of the exemplary embodiments described in this specification are oriented to software installed and executing on computer hardware, nevertheless, alternative embodiments implemented as firmware or as hardware are well within the scope of the present invention.
Methods, systems, and products are now described for creating a presentation document with reference to the accompanying drawings, beginning with
Content server (106) includes storage for presentation documents (314) each of which is composed of a presentation grammar (120) and a structured document (122). A presentation grammar is a data structure that includes a set of key phrases used to identify presentation action identifiers and optional parameters for use in formulating presentation control instructions relevant to structural elements of a content type. In typical embodiments, presentation control instructions are represented by and formulated from presentation action identifiers (reference 518 on
The profile class (126) also includes a string array storing user classifications (210). Examples of user classifications (210) include any supported data codes describing users, including, for example “company=IBM,” “department=marketing,” “technical level=3,” “security level=2,” and others as will occur to those of skill in the art.
Agent (110) includes software modules for structuring a presentation document according to content type (114) and for classifying presentation document content according to presentation attributes (116).
Agent class (110) also includes a member method named createStructuedDocument( ) (232) which is programmed to convert an original document into a structured document by inserting structural element identifiers. Examples of structural element identifiers include <page>, <paragraph>, <row>, <column>, <cell>, <slide>, <jpeg>, <title>, <heading>, <subheading>, and so on, as will occur to those of skill in the art. These examples of structural elements identifiers are expressed as markup tags such as would be used, for example, in a markup language such as HTML (“HyperText Markup Language”) or XML (“eXtensible Markup Language”), although this is not a limitation of the invention. In fact, it is well within the scope of the present invention to implement structural element identifiers with binary codes, Unicode identifiers, or by use of other structure identifiers as will occur to those of skill in the art.
The method of
The method of
Methods of creating presentation documents are further explained with an exemplary use case. Consider the following example of a structured document:
And assume that this exemplary structured document is associated in a presentation document with the following presentation grammar:
This example is discussed with reference to the exemplary system architecture of
Methods of creating presentation documents are further explained with an exemplary use case. Consider the following example of a structured document:
This example is discussed with reference to the exemplary system architecture of
In this example, a first version of paragraph 2 bears a structural identifier <p></p> identifying it as a paragraph, but this first version of paragraph 2 bears no classification identifier. In this example, presentation session (128) is programmed to display this unclassified version of paragraph 2 to users having either the lowest technical classifications, the lowest security classifications, or no particular technical or security classifications at all. Moreover, in an example, where there were only one version of paragraph 2, all users would be presented with that one version.
In this example, a second version of paragraph 2 is classified with a classification identifier <tech level=“2”>. In this example, presentation session (128) is programmed to display this second version of paragraph 2 to users having user classification indicating technical level 2. That is, when a user having technical level 2 in the user's profile classifications (210 on
Similarly, a user having a user profile classification representing a heightened security authorization, security level 2, is shown the version of paragraph 2 classified by the classification identifier <security level=“2”>. A user having a user profile classification identifying the user as a member of the marketing department is shown the version of paragraph 2 classified by the classification identifier <dept id=“marketing”>. A user having a user profile classification identifying the user as an employee of IBM is shown the version of paragraph 2 classified by the classification identifier <company id=“IBM”>.
For purposes of clarity of explanation, the structural elements in this example are shown with only one classification per element. Persons of skill in the art will recognize, however, that it is well within the scope of the present invention for a structural element of a structured document to be classified with any number of classification identifiers.
In the method of
In one exemplary embodiment of the method of
In this example, the content server returns the presentation document as URI encoded data in an HTTP RESPONSE message. In an example where the returned presentation document has this form:
extracting the grammar from the presentation document may be carried out by extracting the portion of the presentation document identified by the tags:
In another exemplary embodiment of the method of
An example of storing a pathname and a network location is storing a URI for the document in a URI array such as that illustrated at reference (220) on
In such an exemplary embodiment, retrieving (368) a presentation grammar is carried out by requesting the presentation grammar (120) from the content server (106), including communicating the presentation document name as a request parameter; and receiving the presentation grammar (120) in response from the content server (106). In an example where the presentation document is located according to a URI as described above and the content server is implemented with a web server, requesting the presentation grammar (120) from the content server (106), including communicating the presentation document name as a request parameter, may be carried out by parsing the URI into an HTTP GET message:
In another exemplary embodiment of the method of
An example of storing a presentation document identifier and a network location is storing a URI for the document in a URI array such as that illustrated at reference (220) on
In such an exemplary embodiment, retrieving (368) a presentation grammar is carried out by requesting the presentation grammar (120) from the content server (106), including communicating the presentation document name as a request parameter; and receiving the presentation grammar (120) in response from the content server (106). In an example where the presentation document is located according to a URI as described above and the content server is implemented with a web server, requesting the presentation grammar (120) from the content server (106), including communicating the presentation document name as a request parameter, may be carried out by parsing the URI into an HTTP GET message:
In the method of
In the example of
The method of
The user grammar (208) in this example includes a multiplicity of user grammar elements (378), and the method includes selecting (390) user grammar elements (378) in dependence upon the structural element identifiers (322). In this example, selecting (390) user grammar elements (378) in dependence upon the structural element identifiers (322) is carried out by comparing the elements of the user grammar with each structured document of each presentation document in the presentation session and extracting each user grammar element having a structural element identifier for a structural element that occurs in a structured document of a presentation document in the presentation session. In the method of
In the example of
The method of
The method of
The method of
For further explanation, consider an example of creating a session document that begins with a presentation document having the following contents:
In this example, an audience of users identified for a presentation include users having in their user profiles user classifications indicating technical level ‘2’ and membership in IBM. None of the registered users have security authorizations and none of them are from the marketing department. Filtering this exemplary presentation document, extracting structural elements with classification identifiers corresponding to the user classifications, writing those structural elements to a session document, and filtering the presentation grammar in dependence upon the extracted structural elements, results in the following exemplary session document:
In the resulting session document, the structural element identified as page 2 now excludes versions for security level 2 and for marketing, because none of the users listed for the presentation are in the marketing department or have security authorizations of level 2. In addition, the session grammar excludes a grammar element for bullets because, in the session document above, the only structural element having bullets was the version of paragraph 2 for the marketing department. Excluding the bullets as structural elements in the session structured document means that there is no need to have grammar elements for them in the session grammar. Reducing the number of grammar elements in the session grammar reduces the number of grammar elements in the voice response grammar, thereby increasing the efficiency and accuracy of the voice response server and the overall presentation system.
The method of
The method of
A user profile event (272) may be generated by adding a user to the presentation, where the added user has a new user classification for the presentation. That is, one example of a user profile event (272) is adding to a presentation a user whose user classifications include at least one user classification having no corresponding classification identifier in any structural element in the session structured document. In such an example, at least one of the added user's user classifications is currently not part of any user profile of any of the other users identified for the presentation.
A user profile event (272) also may be generated, for a further example, by changing a user classification (210) in a user profile (126) of a user who is participating in the presentation, where the changed user classification includes a new user classification for the presentation. That is, one example of a user profile event (272) is editing a user's profile during a presentation so that the user's user profile now includes a user classification having no corresponding classification identifier in any structural element in the session structured document. In such an example, the new user classification is currently not part of any user profile of any of the other users identified for the presentation.
The method of
For further explanation, consider the following example of amending a session document (266) during a presentation. In this example, a session document is used for a presentation having users whose user profiles include user classifications of technical level ‘2’ and membership in IBM:
This session document in this example was created from the following presentation document:
The session document in this example contains no structural elements classified for users from the marketing department. After beginning the presentation a user from the marketing department joins the presentation. The user's joining the presentation is represented by adding the user's user identification to a list of users identified for the presentation. Adding the user ID to the list identifies (274) a user profile event (272) which is represented by a data structure that includes the user's user identification (205). Amending the session document proceeds by adding (276) to a session structured document (256) one or more structural elements (402) from a structured document in the presentation document from which the session structured document was created. Adding (276) to the session structured document (256) at least one structural element (402) from the presentation document (314) is carried out by adding a structural element (402) having a classification identifier (708) that corresponds to a user classification (210) of the user. User classifications of the user are read from the user profiles (126) using the user identification (205) provided to the adding process (276) by the user profile event (272). In this example, adding a structural element to the session structured documents is carried out by adding the following paragraph from the structured document of the presentation document set forth above:
thereby creating the following amended session document:
Amending the session document also includes adding to the session grammar of the session document a new grammar element from the presentation grammar. There were no bullets in the session structured document before the exemplary user profile event and therefore no grammar elements supporting presentation control instructions for bullets. Adding the marketing paragraph also added bullets, so the method advantageously includes adding grammar elements supporting presentation control instructions for bullets:
thereby creating the following amended session document:
The method of
In the method of
The method of
For further explanation, consider an example using the following exemplary session document:
In this example, assume that a first user participant has in a user profile user classifications indicating that the user is an IBM employee and a second user has user classifications indicating that the user has technical ability level ‘2’. In this example, a presentation server having the above session document installed upon it receives (458) a presentation control instruction (460) to move to the display to the second page of the session structured document. The presentation server then selects (452) from the session structured document (256) for the first user the structural element identified as a version of page two and classified as:
and for the second user the structural element identified as a version of page two and classified as:
The method of
and for the second user the structural element identified as a version of page two and classified as:
respectively, the first sent to the client device of the first user and the second sent to the client device of the second user. Note that in both transmission, the classification identifiers are omitted, <company id=“IBM”>and <tech level=“2”> respectively.
This example of presenting (454) a selected structural element (456) to a user (124) is expressed in terms of HTML and HTTP, a stateless, asynchronous protocol. Many embodiments will statefully hold open a data communications connection, such as a TCP/IP connection, between a presentation server and a user client device. A Stateful Java Enterprise Session Bean™ may be used, for example, to hold open a TCP/IP connection implemented with a Java socket object. Readers of skill in the art will recognize therefore that HTML and HTTP are used for explanation, not for limitation. In fact, any presentation application using any appropriate data communications protocol useful for multi-media presentations may be used to present structural elements to users according to embodiments of the present invention. Such applications may be obtained off-the-shelf commercially or they may be specially developed for particular presentations or kinds of presentation. An example of such an application available commercially is Microsoft NetMeeting™. Examples of other data communications protocols useful with various embodiments of the present invention include the Session Initiation Protocol specified in the IETF's RFC 2543, the Real Time Streaming Protocol as specified in the IETF's RFC 2326, the Real Time Transport Protocol of RFC 1889, and the World Wide Web Consortium's VoiceXML protocol specified in the 2003 document entitled “Voice Extensible Markup Language (VoiceXML) Version 2.0”.
The method of
The method of
Many embodiments include creating a user classification instruction in dependence upon rules governing changes in a user classification in a session copy of the participant's user profile. In the method of
In Table 2, a participant identified as “mike—23” is authorized under rules governing changes in a user classification in a session copy of the participant's user profile to make any change in user classifications regarding technical level, no changes in a user classifications regarding a user's company affiliation, and to only decrease user classification regarding security level. Similarly, the rules governing changes in a user classification in a session copy of the participant's user profile of Table 2 authorize a participant identified as “abe—121” to only increase user classifications regarding technical level and to make any changes in user classifications regarding company affiliations. Such rules governing changes in a user classification in a session copy of the participant's user profile provide restraints to a participant's ability to tailor the presentation.
The method of
In the method of
The method of
For further explanation, consider an example using the following exemplary session document:
In this example, assume that a first participant has in a session copy of a first participant's user profile user classifications indicating that the user is an IBM employee and a second participant has user classifications in a session copy of a second participant's user profile indicating that the user has technical ability level ‘2’. In this example, a presentation server having the above session document installed upon it receives (458) a presentation control instruction (460) to move to the display to the second page of the session structured document. The presentation server then selects (452) from the session structured document (256) for the first user the structural element identified as a version of page two and classified as:
and for the second user the structural element identified as a version of page two and classified as:
The method of
In embodiments described above, a session document was created for a presentation in dependence upon user classifications in user profiles of users participating in the presentation. In some embodiments, however, as described above, a participant is authorized to change a classification in a session copy of the participant's user profile during a presentation. When a participant makes such a change, the session document may advantageously be updated to give effect to the changed user classification.
The method of
The method of
The method of
For further explanation, consider an example of updating a session document that begins with a presentation document having the following contents:
In this example, an audience of users identified for a presentation include users having in their session copies of their user profiles user classifications indicating technical level ‘2’ and membership in IBM. None of the users in the presentation have security authorizations and none of them are from the marketing department. Filtering this exemplary presentation document, extracting structural elements with classification identifiers corresponding to the user classifications of the session copies of the user profiles, writing those structural elements to a session document, and filtering the presentation grammar in dependence upon the extracted structural elements, results in the following exemplary session document:
In the resulting updated session document, the structural element identified as page 2 now excludes versions for security level 2 and for marketing, because none of the users listed for the presentation are in the marketing department or have security authorizations of level 2. In addition, the session grammar excludes a grammar element for bullets because, in the session document above, the only structural element having bullets was the version of paragraph 2 for the marketing department. Excluding the bullets as structural elements in the session structured document means that there is no need to have grammar elements for them in the session grammar. Reducing the number of grammar elements in the session grammar reduces the number of grammar elements in the voice response grammar, thereby increasing the efficiency and accuracy of the voice response server and the overall presentation system.
It will be understood from the foregoing description that modifications and changes may be made in various embodiments of the present invention without departing from its true spirit. The descriptions in this specification are for purposes of illustration only and are not to be construed in a limiting sense. The scope of the present invention is limited only by the language of the following claims.
| Number | Name | Date | Kind |
|---|---|---|---|
| 3644674 | Mitchell et al. | Feb 1972 | A |
| 4941187 | Slater | Jul 1990 | A |
| 5119465 | Jack et al. | Jun 1992 | A |
| 5181162 | Smith et al. | Jan 1993 | A |
| 5327176 | Forler et al. | Jul 1994 | A |
| 5388185 | Terry et al. | Feb 1995 | A |
| 5428400 | Landis et al. | Jun 1995 | A |
| 5649223 | Freeman | Jul 1997 | A |
| 5724416 | Foladare et al. | Mar 1998 | A |
| 5745682 | Keenan | Apr 1998 | A |
| 5748186 | Raman | May 1998 | A |
| 5748725 | Kubo | May 1998 | A |
| 5774879 | Custy et al. | Jun 1998 | A |
| 5787175 | Carter | Jul 1998 | A |
| 5881373 | Elofsson et al. | Mar 1999 | A |
| 5901230 | Walker | May 1999 | A |
| 5915001 | Uppaluru | Jun 1999 | A |
| 5917480 | Tafoya et al. | Jun 1999 | A |
| 5983227 | Nazem et al. | Nov 1999 | A |
| 6026069 | Holtslag et al. | Feb 2000 | A |
| 6029043 | Ho et al. | Feb 2000 | A |
| 6029069 | Takaki | Feb 2000 | A |
| 6029195 | Herz | Feb 2000 | A |
| 6049332 | Boetje et al. | Apr 2000 | A |
| 6052730 | Felciano et al. | Apr 2000 | A |
| 6058416 | Mukherjee et al. | May 2000 | A |
| 6092155 | Olnowich | Jul 2000 | A |
| 6119147 | Toomey et al. | Sep 2000 | A |
| 6138040 | Nicholls et al. | Oct 2000 | A |
| 6161112 | Cragun et al. | Dec 2000 | A |
| 6163768 | Sherwood et al. | Dec 2000 | A |
| 6172640 | Durst et al. | Jan 2001 | B1 |
| 6212275 | Akhteruzzaman | Apr 2001 | B1 |
| 6243454 | Eslambolchi | Jun 2001 | B1 |
| 6249281 | Chen et al. | Jun 2001 | B1 |
| 6266691 | Watanabe et al. | Jul 2001 | B1 |
| 6269275 | Slade | Jul 2001 | B1 |
| 6269336 | Ladd et al. | Jul 2001 | B1 |
| 6269398 | Leong et al. | Jul 2001 | B1 |
| 6304283 | Kitagawa | Oct 2001 | B1 |
| 6314402 | Monaco et al. | Nov 2001 | B1 |
| 6324499 | Lewis et al. | Nov 2001 | B1 |
| 6341402 | Lindquist et al. | Jan 2002 | B2 |
| 6343290 | Cossins et al. | Jan 2002 | B1 |
| 6349136 | Light et al. | Feb 2002 | B1 |
| 6366578 | Johnson | Apr 2002 | B1 |
| 6393460 | Gruen et al. | May 2002 | B1 |
| 6446096 | Holland et al. | Sep 2002 | B1 |
| 6448980 | Kumar et al. | Sep 2002 | B1 |
| 6452915 | Jorgensen | Sep 2002 | B1 |
| 6470384 | O'Brien et al. | Oct 2002 | B1 |
| 6493671 | Ladd et al. | Dec 2002 | B1 |
| 6505195 | Ikeda et al. | Jan 2003 | B1 |
| 6529905 | Bray et al. | Mar 2003 | B1 |
| 6636238 | Amir et al. | Oct 2003 | B1 |
| 6644674 | Simard | Nov 2003 | B2 |
| 6647531 | Isaac et al. | Nov 2003 | B2 |
| 6658414 | Bryan et al. | Dec 2003 | B2 |
| 6665715 | Houri | Dec 2003 | B1 |
| 6677858 | Faris et al. | Jan 2004 | B1 |
| 6681247 | Payton | Jan 2004 | B1 |
| 6717593 | Jennings | Apr 2004 | B1 |
| 6725199 | Brittan et al. | Apr 2004 | B2 |
| 6725256 | Albal et al. | Apr 2004 | B1 |
| 6745163 | Brocious et al. | Jun 2004 | B1 |
| 6763226 | McZeal, Jr. | Jul 2004 | B1 |
| 6766368 | Jakobson et al. | Jul 2004 | B1 |
| 6778989 | Bates et al. | Aug 2004 | B2 |
| 6799208 | Sankaranarayan et al. | Sep 2004 | B1 |
| 6819338 | Heasman et al. | Nov 2004 | B2 |
| 6829745 | Yassin et al. | Dec 2004 | B2 |
| 6879958 | Kojima et al. | Apr 2005 | B1 |
| 6892348 | Truelove et al. | May 2005 | B1 |
| 6931254 | Egner et al. | Aug 2005 | B1 |
| 6981263 | Zhang et al. | Dec 2005 | B1 |
| 6996768 | Elo et al. | Feb 2006 | B1 |
| 7000183 | Crawford, Jr. | Feb 2006 | B1 |
| 7003550 | Cleasby et al. | Feb 2006 | B1 |
| 7058890 | George et al. | Jun 2006 | B2 |
| 7079712 | Silverbrook et al. | Jul 2006 | B1 |
| 7099920 | Kojima et al. | Aug 2006 | B1 |
| 7149694 | Harb et al. | Dec 2006 | B1 |
| 7162692 | Bodin et al. | Jan 2007 | B2 |
| 7174557 | Sanghvi et al. | Feb 2007 | B2 |
| 7246316 | Furlong et al. | Jul 2007 | B2 |
| 7249157 | Stewart et al. | Jul 2007 | B2 |
| 7283621 | Quinton | Oct 2007 | B2 |
| 7287093 | Lynch et al. | Oct 2007 | B2 |
| 7287221 | Bodin et al. | Oct 2007 | B2 |
| 7299405 | Lee et al. | Nov 2007 | B1 |
| 7313595 | Rust | Dec 2007 | B2 |
| 7334050 | Zondervan et al. | Feb 2008 | B2 |
| 7395505 | Khosla et al. | Jul 2008 | B1 |
| 7426538 | Bodin et al. | Sep 2008 | B2 |
| 7428698 | Bodin et al. | Sep 2008 | B2 |
| 7430707 | Bodin et al. | Sep 2008 | B2 |
| 7469380 | Wessling et al. | Dec 2008 | B2 |
| 7475340 | Bodin et al. | Jan 2009 | B2 |
| 7487208 | Bodin et al. | Feb 2009 | B2 |
| 7487209 | Bodin et al. | Feb 2009 | B2 |
| 7487451 | Bodin et al. | Feb 2009 | B2 |
| 7519659 | Bodin et al. | Apr 2009 | B2 |
| 7519683 | Bodin et al. | Apr 2009 | B2 |
| 7519904 | Bodin et al. | Apr 2009 | B2 |
| 7533146 | Kumar | May 2009 | B1 |
| 7567908 | Bodin et al. | Jul 2009 | B2 |
| 7571380 | Bodin et al. | Aug 2009 | B2 |
| 7603112 | Huomo et al. | Oct 2009 | B2 |
| 7634412 | Bodin et al. | Dec 2009 | B2 |
| 7802182 | Bales | Sep 2010 | B2 |
| 7890848 | Bodin et al. | Feb 2011 | B2 |
| 20010032218 | Huang | Oct 2001 | A1 |
| 20010032241 | Braga Illa et al. | Oct 2001 | A1 |
| 20010041973 | Abkowitz et al. | Nov 2001 | A1 |
| 20010054048 | Kelsey | Dec 2001 | A1 |
| 20020032564 | Ehsani et al. | Mar 2002 | A1 |
| 20020035976 | Nakamura et al. | Mar 2002 | A1 |
| 20020046212 | Meifu et al. | Apr 2002 | A1 |
| 20020046221 | Wallace et al. | Apr 2002 | A1 |
| 20020075303 | Thompson et al. | Jun 2002 | A1 |
| 20020087416 | Knutson | Jul 2002 | A1 |
| 20020087974 | Sprague | Jul 2002 | A1 |
| 20020095454 | Reed et al. | Jul 2002 | A1 |
| 20020112240 | Bacso et al. | Aug 2002 | A1 |
| 20020119434 | Beams et al. | Aug 2002 | A1 |
| 20020129354 | Bryan et al. | Sep 2002 | A1 |
| 20020130902 | Shaouy et al. | Sep 2002 | A1 |
| 20020133500 | Arlein et al. | Sep 2002 | A1 |
| 20020138331 | Hosea et al. | Sep 2002 | A1 |
| 20020147749 | Ortiz et al. | Oct 2002 | A1 |
| 20020147782 | Dimitrova et al. | Oct 2002 | A1 |
| 20020152301 | Garrett et al. | Oct 2002 | A1 |
| 20020188451 | Guerra et al. | Dec 2002 | A1 |
| 20030023435 | Josephson | Jan 2003 | A1 |
| 20030028871 | Wang et al. | Feb 2003 | A1 |
| 20030037301 | Rising, III | Feb 2003 | A1 |
| 20030041238 | French et al. | Feb 2003 | A1 |
| 20030056172 | Vukelic et al. | Mar 2003 | A1 |
| 20030060212 | Thomas | Mar 2003 | A1 |
| 20030060214 | Hendrey et al. | Mar 2003 | A1 |
| 20030065712 | Cheung et al. | Apr 2003 | A1 |
| 20030071833 | Dantzig et al. | Apr 2003 | A1 |
| 20030074454 | Peck | Apr 2003 | A1 |
| 20030074606 | Boker | Apr 2003 | A1 |
| 20030085912 | Soo | May 2003 | A1 |
| 20030099343 | Dezonno | May 2003 | A1 |
| 20030112267 | Belrose | Jun 2003 | A1 |
| 20030158969 | Gimson et al. | Aug 2003 | A1 |
| 20030177205 | Liang et al. | Sep 2003 | A1 |
| 20030208473 | Lennon | Nov 2003 | A1 |
| 20030229900 | Reisman | Dec 2003 | A1 |
| 20040003042 | Horvitz et al. | Jan 2004 | A1 |
| 20040006595 | Yeh et al. | Jan 2004 | A1 |
| 20040088654 | Uotani | May 2004 | A1 |
| 20040105127 | Cudd et al. | Jun 2004 | A1 |
| 20040107404 | Burns et al. | Jun 2004 | A1 |
| 20040141016 | Fukatsu et al. | Jul 2004 | A1 |
| 20040193425 | Tomes | Sep 2004 | A1 |
| 20040201610 | Rosen et al. | Oct 2004 | A1 |
| 20040204979 | Eisenberg et al. | Oct 2004 | A1 |
| 20040205081 | Chao | Oct 2004 | A1 |
| 20040205114 | Kinoshita | Oct 2004 | A1 |
| 20040205132 | Czerwonka | Oct 2004 | A1 |
| 20040205452 | Fitzsimons et al. | Oct 2004 | A1 |
| 20040205572 | Fields et al. | Oct 2004 | A1 |
| 20040205629 | Rosenholtz et al. | Oct 2004 | A1 |
| 20050025136 | Anschutz et al. | Feb 2005 | A1 |
| 20050131944 | Patrick et al. | Jun 2005 | A1 |
| 20050132271 | Bodin et al. | Jun 2005 | A1 |
| 20050132273 | Bodin et al. | Jun 2005 | A1 |
| 20050132274 | Bodin et al. | Jun 2005 | A1 |
| 20050132275 | Bodin et al. | Jun 2005 | A1 |
| 20050154970 | Bodin et al. | Jul 2005 | A1 |
| 20050154975 | Bodin et al. | Jul 2005 | A1 |
| 20050165900 | Bodin et al. | Jul 2005 | A1 |
| 20050171780 | Schmid et al. | Aug 2005 | A1 |
| 20050240603 | Bodin et al. | Oct 2005 | A1 |
| 20050240608 | Jones et al. | Oct 2005 | A1 |
| 20050240859 | Bodin et al. | Oct 2005 | A1 |
| 20050240909 | Tersigni | Oct 2005 | A1 |
| 20060010365 | Bodin et al. | Jan 2006 | A1 |
| 20060010370 | Bodin et al. | Jan 2006 | A1 |
| 20060014546 | Bodin et al. | Jan 2006 | A1 |
| 20060015557 | Bodin et al. | Jan 2006 | A1 |
| 20060080614 | Lentz | Apr 2006 | A1 |
| 20060136809 | Fernstrom | Jun 2006 | A1 |
| 20060218245 | Horn | Sep 2006 | A1 |
| 20060277465 | Pandit et al. | Dec 2006 | A1 |
| 20070250602 | Bodin et al. | Oct 2007 | A1 |
| 20070266097 | Harik et al. | Nov 2007 | A1 |
| 20080141117 | King et al. | Jun 2008 | A1 |
| 20080172227 | Bodin et al. | Jul 2008 | A1 |
| 20080177837 | Bodin et al. | Jul 2008 | A1 |
| 20080177838 | Bodin et al. | Jul 2008 | A1 |
| 20080177866 | Bodin et al. | Jul 2008 | A1 |
| 20080178078 | Bodin et al. | Jul 2008 | A1 |
| 20080235028 | Bodin et al. | Sep 2008 | A1 |
| 20080259910 | Bodin et al. | Oct 2008 | A1 |
| 20080263435 | Bodin et al. | Oct 2008 | A1 |
| 20080282168 | Bodin et al. | Nov 2008 | A1 |
| 20090037820 | Bodin et al. | Feb 2009 | A1 |
| 20090048829 | Bodin et al. | Feb 2009 | A1 |
| 20090099842 | Bodin et al. | Apr 2009 | A1 |
| Number | Date | Country |
|---|---|---|
| WO 0120481 | Mar 2001 | WO |
| WO 0124462 | Apr 2001 | WO |
| Entry |
|---|
| U.S. Appl. No. 10/285,615, filed Oct. 31, 2002, Bates. |
| Websgebg et al; Real-time Content-Based Processing of Multicast Video; Cat. No. 98CH36284; 1998; pp. 882-886, vol. 1; IEEE, Piscataway, NJ, USA; USA. |
| Kim et al; An Extended Object Composition Model for Distributed Multimedia Services; Cat. No. 98CH36284; Jan. 2002; pp. 279-286, vol. 1; IEEE, Piscataway, NJ, USA; USA. |
| Agawa et al.; Image Analysis for Face Modeling and Facial Image Reconstruction; SPIE; 1990; pp. 1184-1197; vol. 1360, pt 2; US. |
| Kim et al.; An Extended Object Composition Model for Distributed Multimedia Supports in World-Wide Web; INSPEC AN B9810-6210R-071, C9810-6130M-080; 1997; pp. 159-174; Singapore. |
| “U.S. Appl. No. 11/766,863 Final Office Action”, Jan. 6, 2011 , 26 pages. |
| “U.S. Appl. No. 12/056,333 Final Office Action”, Feb. 4, 2011 , 13 pages. |
| “U.S. Appl. No. 12/058,672 Final Office Action”, Mar. 10, 2011, 15 pages. |
| Christian, Kevin et al., “A Comparison of Voice Controlled and Mouse Controlled Web Browsing”, ASSETS http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.31.6541 (Obtained from the Internet on Oct. 7, 2011) 2000, pp. 72-29. |
| U.S. Appl. No. 10/285,615, filed Oct. 2002, Bates. |
| Notice of Allowance Dated Sep. 29, 2008 0in U.S. Appl. No. 10/733,954. |
| Office Action Dated Jan. 12, 2006 in U.S. Appl. No. 10/734,764. |
| Office Action Dated Jun. 27, 2006 in U.S. Appl. No. 10/734,764. |
| Final Office Action Dated Apr. 18, 2007 in U.S. Appl. No. 10/734,764. |
| Office Action Dated Jan. 12, 2006 in U.S. Appl. No. 10/733,947. |
| Office Action Dated Jun. 27, 2006 in U.S. Appl. No. 10/733,947. |
| Final Office Action Dated Dec. 14, 2006 in U.S. Appl. No. 10/733,947. |
| Office Action Dated May 4, 2007 in U.S. Appl. No. 10/733,941. |
| U.S. Appl. No. 12/331,524, filed Dec. 2008, Bodin et al. |
| Final Office Action Dated Oct. 10, 2007 in U.S. Appl. No. 10/733,941. |
| Office Action Dated Jun. 12, 2008 in U.S. Appl. No. 10/733,941. |
| Final Office Action Dated Dec. 5, 2008 in U.S. Appl. No. 10/733,941. |
| Notice of Allowance Dated Aug. 6, 2009 in U.S. Appl. No. 10/733,941. |
| Office Action Dated Jan. 10, 2006 in U.S. Appl. No. 10/733,937. |
| Final Office Action Dated Jun. 1, 2006 in U.S. Appl. No. 10/733,937. |
| Office Action Dated Jan. 30, 2006 in U.S. Appl. No. 10/733,943. |
| Final Office Action Dated Jun. 28, 2006 in U.S. Appl. No. 10/733,943. |
| Office Action Dated Feb. 27, 2006 in U.S. Appl. No. 10/733,942. |
| Notice of Allowance Dated Jul. 19, 2006 in U.S. Appl. No. 10/733,942. |
| Office Action Dated Mar. 8, 2006 in U.S. Appl. No. 10/756,159. |
| Final Office Action Dated Aug. 31, 2006 in U.S. Appl. No. 10/756,159. |
| Notice of Allowance Dated Jun. 19, 2007 in U.S. Appl. No. 10/756,159. |
| Office Action Dated Sep. 30, 2008 in U.S. Appl. No. 10/756,126. |
| Notice of Allowance Dated Mar. 23, 2009 in U.S. Appl. No. 10/756,126. |
| Office Action Dated Aug. 3, 2006 in U.S. Appl. No. 10/756,107. |
| Office Action Dated Feb. 1, 2007 in U.S. Appl. No. 10/756,107. |
| Final Office Action Dated Sep. 6, 2007 in U.S. Appl. No. 10/756,107. |
| Office Action Dated Feb. 20, 2008 in U.S. Appl. No. 10/756,107. |
| Notice of Allowance Dated Jun. 27, 2008 in U.S. Appl. No. 10/756,107. |
| Final Office Action Dated Sep. 21, 2006 in U.S. Appl. No. 10/756,110. |
| Office Action Dated Mar. 10, 2006 in U.S. Appl. No. 10/756,110. |
| Office Action Dated Mar. 8, 2006 in U.S. Appl. No. 10/756,145. |
| Final Office Action Dated Jan. 18, 2007 in U.S. Appl. No. 10/756,145. |
| Office Action Dated Sep. 6, 2007 in U.S. Appl. No. 10/756,145. |
| Notice of Allowance Dated May 19, 2008 in U.S. Appl. No. 10/756,145. |
| Office Action Dated Mar. 8, 2006 in U.S. Appl. No. 10/756,146. |
| Final Office Action Dated Jan. 18, 2007 in U.S. Appl. No. 10/756,146. |
| Office Action Dated Mar. 22, 2006 in U.S. Appl. No. 10/756,158. |
| Final Office Action Dated Aug. 29, 2006 in U.S. Appl. No. 10/756,158. |
| Office Action Dated Mar. 31, 2008 in U.S. Appl. No. 10/887,516. |
| Notice of Allowance Dated Sep. 24, 2008 in U.S. Appl. No. 10/887,516. |
| Office Action Dated Dec. 19, 2006 in U.S. Appl. No. 10/887,517. |
| Final Office Action Apr. 23, 2007 in U.S. Appl. No. 10/887,517. |
| Notice of Allowance Dated May 19, 2008 in U.S. Appl. No. 10/887,517. |
| Office Action Dated Jul. 25, 2006 in U.S. Appl. No. 10/887,518. |
| Final Office Action Dated Apr. 11, 2007 in U.S. Appl. No. 10/887,518. |
| Office Action Dated Jun. 28, 2006 in U.S. Appl. No. 10/887,430. |
| Final Office Action Dated Dec. 5, 2006 in U.S. Appl. No. 10/887,430. |
| Office Action Dated Jul. 10, 2008 in U.S. Appl. No. 10/887,429. |
| Notice of Allowance Dated Dec. 11, 2008 in U.S. Appl. No. 10/887,429. |
| Office Action Dated Apr. 4, 2008 in U.S. Appl. No. 10/832,006. |
| Notice of Allowance Dated Dec. 2, 2008 in U.S. Appl. No. 10/832,006. |
| Office Action Dated Aug. 6, 2008 in U.S. Appl. No. 10/832,035. |
| Notice of Allowance Dated Feb. 8, 2010 in U.S. Appl. No. 10/832,035. |
| Office Action Dated Jan. 23, 2008 in U.S. Appl. No. 10/832,004. |
| Notice of Allowance Dated Dec. 2, 2008 in U.S. Appl. No. 10/832,004. |
| Office Action Dated Apr. 9, 2007 in U.S. Appl. No. 10/889,786. |
| Office Action Dated Sep. 10, 2007 in U.S. Appl. No. 10/889,786. |
| Office Action Dated Mar. 24, 2008 in U.S. Appl. No. 10/889,786. |
| Final Office Action Dated Dec. 23, 2008 in U.S. Appl. No. 10/889,786. |
| Office Action Dated Feb. 28, 2008 in U.S. Appl. No. 10/889,817. |
| Office Action Dated Aug. 26, 2008 in U.S. Appl. No. 10/889,817. |
| Final Office Action Dated Feb. 27, 2009 in U.S. Appl. No. 10/889,817. |
| Office Action Dated Jun. 1, 2007 in U.S. Appl. No. 10/889,787. |
| Office Action Dated Oct. 17, 2007 in U.S. Appl. No. 10/889,787. |
| Notice of Allowance Dated May 5, 2008 in U.S. Appl. No. 10/889,787. |
| Office Action Dated Aug. 17, 2006 in U.S. Appl. No. 10/832,005. |
| Final Office Action Dated Jan. 24, 2007 in U.S. Appl. No. 10/832,005. |
| Office Action Dated Jan. 25, 2008 in U.S. Appl. No. 10/889,818. |
| Notice of Allowance Dated Jul. 16, 2008 in U.S. Appl. No. 10/889,818. |
| Notice of Allowance Dated Apr. 16, 2009 in U.S. Appl. No. 12/171,779. |
| W3C, “XHTML+Voice Profile 1.0,” Dec. 21, 2001, pp. 1-20. |
| W3C, “Speech Synthesis Markup Language Version 1,” Dec. 2, 2002, downloaded pp. 1-36. |
| IBM, “IBM WebShere Voice Server 2.0 Implementation Guide,” IBM Redbook, May 2002, title page, copyright page, and pp. 251-276 and 291-292. |
| Christian, et al. “A Comparison of Voice controlled and Mouse Controlled Web Browsing,” Assets 2000, 72-79, Arlington, VA, 2000, downloaded from: http://citeseer.its.psu.edu/373105.html, downloaded pages including CiteSeer bibliography, 1-10. |
| Karl, et al., “Speech-Activated versus Mouse-Activated Commands for Word Processing Applications: An Empirical Evaluation,” Int'l J. Man-Machine Studies, vol. 39, pp. 667-687, 1993, downloaded from http://hcil.cs.umd.edu/trs/92-09/92-09.html, downloaded pp. 1-17. |
| Damiani, et al. “A Fine-Grained Access Control System for XML Documents,” ACM Transactions on Information and System Security, vol. 5, No. 2, May 2002, pp. 169-202. |
| Kudo, et al., “XML Document Security based on Provisional Authorization,” CSS'00 Athens, Greece, copyright 2000, pp. 87-96. |
| Begel, et al., “SpeedNav: Document Navigation by Voice”, University of California, Berkeley, Oct. 9, 2002. |
| Goose, et al., “Streaming Speech: A Framework for Generating and Streaming 3-D Text-To-Speech and Audio Presentations to Wireless PDAs as Specified Using Extensions to SMIL”, WWW2002, May 2002, p. 1-16, Honolulu, Hawaii; USA. |
| Freire, et al., “Webviews: Accessing Personalized Web Content and Services”, Proceedings of the 10th International Conference on World Wide Web, May 2001, ACM Press, p. 576-586, Hong Kong. |
| Engelbart; Authorship Provision in Augment; IEEE Cat. No. 84CH2017-2; Spring 1984; p. 465-472; IEEE Comput. Soc. Press, Silver Spring, MD; USA. |
| Wensbeng, et al; Real-time Content Based Processing of Multicast Video; Cat. No. 98CH36284; 1998; pp. 882-886, vol. 1; IEEE, Piscataway, NJ; USA. |
| Kim et al; An Extended Object Composition Model for Distributed Multimedia Services; Cat. No. 98CH36284; Jan. 2002; pp. 279-286, vol. 1; IEEE, Piscataway, NJ; USA. |
| Agawa et al. “Image Analysis for Face Modeling and Facial Image Reconstruction” SPIE; 1990; pp. 1184-1197; vol. 1360, pt 2; USA. |
| Kim et al.; An Extended Object Composition Model for Distributed Multimedia Supports in World Wide Web; INSPEC AN B9810-6210R-071, C9810-6130M-080; 1997; pp. 159-174; Singapore. |
| Capra, et al., “WebContext: Remote Access to Shared Context”, ACM International Conference Proceeding Series, vol. 15, Proceedings of the 2001 Workshop on Perceptive User Interfaces, 2001, p. 1-9. |
| Corcoran, et al., “Home Network Infrastructure for Handheld/Wearable Appliances”, IEEE Transactions on Consumer Electronics, Aug. 2002, vol. 48, Issue 3, p. 490-495. |
| Komlodi et al., “Key Frame Preview Techniques for Video Browsing”, ACM 2002, pp. 118-125. |
| Cruz, et al. “A Visual Approach to Multimedia Que in. And Presentation” ACM 1997, pp. 109-119. |
| Shumate, “Hyperizons”, Google 1997, pp. 1-3. |
| Chigona et al., “Contextualized Preview of Image Map Links”, ACM 2002, pp. 150-158. |
| Rao et al., Evolution of Mobile Location-Based Services. Commun. ACM 46, 12 (Dec. 2003), pp. 61-65. |
| Sripanidkulchai, et al., “Efficient content location using interest-based locality in peer-to-peer systems.” Infocom 2003. Twenty-Second Annual Joint Conference of the IEEE Computer and Communications Societies. IEEE, vol. 3, pp. 2166-2176, Mar. 30-Apr. 3, 2003. |
| Kimbrough et al., “Deontic Logic 2: On Original Generation of Structure in Legal Documents”, Proceedings of the 9th International Conference Artificial Intelligence and Law, Jun. 2003, pp. 152-161. |
| Adelberg, Brad, “NoDoSE—A Tool for Semi-Automatically Extracting Structured and Semistructure Data from Text Documents”, ACM SIGMOD Record, vol. 27, Issue 2, Jun. 1998, pp. 283-294. |
| Final Office Action, U.S. Appl. No. 10/734,764, mailed Dec. 9, 2010. |
| Notice of Allowance, U.S. Appl. No. 10/887,518, mailed Nov. 22, 2010. |
| Office Action, U.S. Appl. No. 11/766,863, mailed Jul. 23, 2010. |
| Office Action, U.S. Appl. No. 12/056,333, mailed Aug. 23, 2010. |
| Office Action, U.S. Appl. No. 12/058,672, mailed Oct. 12, 2010. |
| Final Office Action, U.S. Appl. No. 12/058,673, mailed Dec. 29, 2010. |
| Office Action, U.S. Appl. No. 12/169,823, mailed Oct. 6, 2010. |
| Office Action, U.S. Appl. No. 12/240,395, mailed Aug. 24, 2010. |
| “Final Office Action Dated Dec. 9, 2010 in U.S. Appl. No. 10/734,764”, Dec. 9, 2010 , 14 pages. |
| “Final Office Action Dated Dec. 29, 2010 in U.S. Appl. No. 12/058,673”, Dec. 29, 2010, 10 pages. |
| “Notice of Allowance Dated Nov. 22, 2010 in U.S. Appl. No. 10/887,518”, Nov. 22, 2012 , 4 pages. |
| “Office Action Dated Jul. 23, 2010 in U.S. Appl. No. 11/766,863”, Jul. 23, 2010, 23 pages. |
| “Office Action Dated Aug. 23, 2010 in U.S. Appl. No. 12/056,333”, Aug. 23, 2010, 8 pages. |
| “Office Action Dated Aug. 24, 2010 in U.S. Appl. No. 12/240,395”, Aug. 24, 2010, 20 pages. |
| “Office Action Dated Oct. 6, 2010 U.S. Appl. No. 12/169,823”, Oct. 6, 2010, 5 pages. |
| “Office Action Dated Oct. 12, 2010 in U.S. Appl. No. 12/058,672”, Oct. 12, 2010, 10 pages. |
| “U.S. Appl. No. 11/766,863 Final Office Action”, Apr. 5, 2012, 15 pages. |
| “U.S. Appl. No. 11/766,863 Office Action”, Nov. 9, 2011, 23 pages. |
| Number | Date | Country | |
|---|---|---|---|
| 20050165900 A1 | Jul 2005 | US |
