Patent Application
20080043943
In order to describe the manner in which the above-recited and other advantages and features can be obtained, a more particular description of the subject matter briefly described above will be rendered by reference to specific embodiments which are illustrated in the appended drawings. Understanding that these drawings depict only typical embodiments and are not therefore to be considered limiting in scope, embodiments will be described and explained with additional specificity and detail through the use of the accompanying drawings in which:
Systems and methods are discussed herein for sending and receiving messages that have an audio element, such as a voice message, associated therewith. Embodiments of the invention relate to a systems and methods for managing voice messages and in particular to a graphical user interface that enables users to manage (i.e., access, create, delete, forward, move, reply, listen, etc.) voice messages on a computer.
According to one example, the graphical user interface organizes and displays the messages as entries in an inbox. Further, the graphical user interface provides a view of a voice message thread or provides a view of related voice messages. This enables a user to listen to a specific voice message in a thread. In one example, a hierarchical view of individual messages in a threaded voice message string can be displayed to a user.
According to one example, the voice message is included with the message as an attachment or may be accessed using a provided in the message. The various voice messages in the thread can also be includes as an attachment or as a link. The graphical user interface allows a user to play a voice message directly from an entry in the inbox. A user may also be able to advance to a selected point in the voice message in various ways, such as manual advancement, searching for specific words or phonemes, or time increments.
In addition, the interface may provide an editable notes/subject field associated with voice messages in the inbox, in which a notes/subject field can be generated automatically, edited, and/or forwarded to third parties. The interface further allows a user to listen to selected portions or sub-portions of a received audio or multi-media message and to forward individual portions or sub-portions of the message.
The processor (110) processes incoming data, such as voice messages (120). According to one example, incoming messages (120) are first stored in the database (130) after which the processor (110) processes the stored message (120) to form a message suitable for delivery in the same or in an alternative format. For example, the processor (110) may be configured to process a voice message (120) to be delivered using email protocol. For ease of reference, email protocol will be discussed in reference to an alternative format. In addition to storing messages, the database may also store a user's contact information (150), user preferences (160), and the like.
The combination of an email with an audio recording will be referred to generally as a message. Those of skill in the art will appreciate that other types of message delivery formats may be utilized for delivery of the voice message. Returning to the example of an email, the voice message may be included in the email as an attachment or the processor may provide a link in the email for accessing the stored voice message on the database (130).
The message (120) may include additional information about the original voice message. For example, the message (120) may include information about the sender, information related to the topic of the voice message, and/or the time the message was received by the system (100).
The system (100) may both send and receive voice messages (120) via the selected alternative format, for example, email, to and from other recipients that have access to the system or to and from recipients outside the system. The user interface (140) facilitates the managing (e.g., sending, receiving, listening, deleting, forwarding, replying, etc.) of messages (120), either within the system (100) or residing on other systems. The user interface (140) may reside on the system (100) as shown in
The user interface (140) organizes and displays the messages as entries in an inbox and a user can play a voice message directly from an entry in the inbox, including the ability to advance to a selected point in the voice message. The user interface (140) enables a user to play the voice message without being required to open an alternative audio or multimedia player. The user interface (140) may also provide a hierarchical view of individual messages in a threaded voice message string. In addition, the user interface (140) may be configured to provide an editable notes/subject field associated with a voice message in the inbox, in which a notes/subject field can be generated automatically or manually, edited, and forwarded to third parties. The interface (140) further allows a user to listen to selected portions or sub-portions of a received audio or multi-media message. A user can also forward individual portions or sub-portions of a message. These and further features will be discussed in more detail below.
In this example,
A message received by the system (100) may first be processed prior to being displayed in the inbox display (202). According to one example, processing the message may include determining the source of the message, the time the message was received, whether any topical information is associated with or may be gleaned from the message, whether the message is part of a thread of messages or otherwise includes message history information associated therewith, and the like. Processing the message may also include converting the message to a format appropriate for display in the inbox display (202).
The received message(s) are displayed in a display portion (224) of the inbox display (202). According to one example, a default setting for the display portion (224) sorts incoming messages into rows. In particular, as seen in
The display portion (224) is further organized into columns in this example. Each column includes specific information, if available, about each message. These columns include a source column (228), a notes column (232), a time received column (240), and a length column (244). The notes column (232) includes information related to the voice message. Entries in the notes column may include information that has been selectively extracted by the system (100) as will be discussed in more detail below. The entry may also or alternatively include information entered by the sender or source of the message. An incoming message is processed and relevant information is sorted and listed in the corresponding columns within the message display portion (224) of the graphical user interface (200).
The illustrated graphical user interface 200 further organizes the source column (228) in a hierarchal manner to provide readily identifiable information as to a given message, such as the previous sources, recipients, responses, and the associated with a given message. The source column (228) displays a main or primary message source on a first line. The main or primary source identifies the person who sent the message to the user (210). If the message is not part of a thread, but is a standalone message composed and sent by the sender to the user (210), the identity of the sender is the only item listed in the source column (228). Such an example is seen in a message (250) sent by Jim Woods.
If, however, the message is part of a forwarded message thread, the history information related to the other portions of the thread may also be included in the source column (228) and sorted hierarchically. For example, a message (252) from “Freddy Forward” is part of a thread. In the present scenario, an original message (256) from Cindy Source was sent to Freddy Forward. Freddy Forward in turn recorded an introduction to the original message (256), and then forwarded the resultant message (252) to the user (210).
The graphical user interface (200) may indicate that Cindy's message was forwarded by Freddy by listing Freddy Forward's message entry (252) in the source column (228) on a row above Cindy Source's original message entry (256). Cindy Source's name or other identifying information and the original message (256) is listed on a row below Freddy Forward's message (252) as a second entry. According to the present example, Cindy Source's name may be indented to identify the original message (256) as being a sub-portion or sub-message of the message sent by Freddy Forward.
If, for example, the Cindy Source message (256) had included additional message recordings that were sent to her and which she had subsequently forwarded to Freddy Forward, those messages would also be identified as part of the message thread by including the source of the messages, if known, and indenting the source information relative to at least the first row for the message entry (252). Accordingly, the graphical user interface (200) provides a hierarchical organization for messages that include an audio component.
The graphical user interface (200) allows the user (210) to listen to the message content directly from the inbox display (202). Rather than being required to save the audio recording and manually open the audio recording with another audio or media player, the user (210) can simply play the message directly from the user interface 200. For example, each message (212) may be associated with a play icon (258). By clicking one of the play icons (258), the corresponding audio recording is played directly within the user interface (200) rather than causing a separate audio or media player to be opened in order to play the audio recording.
The graphical user interface (200) allows the user (210) to readily select the portions of the message that the user wishes to hear. For example, using the forwarding example of Cindy Source (256) and Freddy Forward (252), the graphical user interface (200) includes a primary play icon (260) with the message entry (252) and a secondary play icon (264) with each of the subsequent lines, such as with the original message (256). According to one example, selecting the primary play icon (260) plays the recording sent by the primary source. In the scenario discussed above, selecting the primary play icon (260) would play the message recorded by Freddy Forward. Thereafter, the user (210) would be able to listen to any secondary message by selecting the corresponding secondary play icon (264). Continuing with the Freddy Forward/Cindy Source example, selecting the secondary play icon (264) would play the original message (256). Thus, the graphical user interface (200) allows the user (210) to listen to any portion of the message thread while allowing the user to choose how messages are sorted and how the audio portion of the message is played.
Furthermore, where multiple messages are part of a single voice message thread, in one embodiment, by selecting to play one voice message, the user interface (200) may first play the selected message and may then automatically play one or more of the other voice messages within the thread. For example, continuing with the Cindy Source (256) and Freddy Forward (252) example, by selecting to play the Freddy Forward message, the graphical user interface (200) may play a message left by Freddy Forward, and then may automatically play the message left by Cindy Source.
Further, a user (210) can listen to specific portions of a message. When selecting the portion sent by Cindy Source, a user (210) may choose to start at any point in the message. For example, when a user (210) initiates the playback of a message, for example, by clicking a play icon (258), a graphic may be displayed to the user indicating a timeline of the audio recording. The timeline may be controlled by the user (210) so that the user can quickly position the playback feature at any point within the timeline of the audio recording.
In one embodiment, the messages (212) displayed by the graphical user interface (200) can also be accessed via a telephone. In other words, rather than accessing the messages (212) by opening the graphical user interface (200), the user can access the messages (212) by calling into the message system (100) and selecting individual messages via a touchtone or voice recognition interface. When a user accesses voice messages 212 via a telephone interface, the ability to generate voice message threads as disclosed above provides a listener with the context of each message that would not otherwise be available.
For example, when the user (210) listens via a telephone interface to a message (266) that is in reply to the message (268) originally sent by the user, the system may also play the user's original message (268) either immediately after or before playing the reply message (266). Therefore, if the user (210) sends Vicky Mayo a first message regarding a first task that Vicky is responsible for and a second message regarding a second task that Vicky is responsible for, by using threading, Vicky can reply by simply stating “Done.” By listening to the entire threads, the user (210) can easily determine task Vicky Mayo is referring to.
Referring now to
To review the message, the user may select a play icon (340). By default, the play icon (340) may initially begin the playback of the message from the beginning. Alternatively, the user may access any point of time within the message with the use of a slider (350). In particular, the location of the slider (350) is correlated to the time line (334) and the recording viewer (330). By dragging the slider (350) to a desired location along the time line (334), the user is thereby able to access the recording at that point of the message. The graphical representation of the voice sound wave pattern (332) may provide visual clues to assist the user when selecting a starting point within the message.
In addition to allowing the user to review the recorded message, in one embodiment, the message recording display (300) allows the user to edit the message. For example, in one embodiment, the user can rerecord part or the entire message from a selected point. The user may find the desired point by dragging the slider (350) to estimate the location and then listen to message to determine if the desired point has been reached. Once the desired point in the message has been reached, whether at the beginning of the message or some other point, the user may then select the record icon (330) and rerecord the message from that point.
The example message recording display (300) also allows the user to enter notes using a note entry box (360). To enter notes, the user selects the note entry box (360) and inputs notes or any other desired information. When the message is sent, the notes may also be sent to be read by the recipient. In one embodiment, the notes may be treated as a subject line for allowing the recipient to quickly determine the subject matter of the recorded message.
Once the user completes the message, the user may send the message by selecting a send icon (370). The send icon (370) causes the system to send the message to the recipients indicated in the recipient entry box (310). Alternatively, if the user decides not to send the message, the user may select a cancel icon (380). According to one example, if the message is sent or the action is cancelled the user is returned to the previous screen, such as the inbox display (202) shown in
The graphical user interface (200) also allows the user (210) to manage how the messages (212) are treated within the system as well as managing outgoing messages. The management of outgoing messages, including forwarding and replying to messages will now be discussed, followed by a discussion of drafting new messages. Thereafter, the handling of messages within the system will be discussed.
Referring again to
Selecting the forward icon (272) may access the message recording display (300), as described in
Referring again to
The graphical user interface (200) also allows the user to create an original message by selecting a new message icon (288). Selecting the new message icon (288) causes the system to display the message recording display (300). The user is then able to record an original message and enter notes as discussed above.
Thus, the graphical user interface (200) allows a user to manage outgoing messages. As previously introduced, the graphical user interface (200) also allows a user to manage the handling of messages within the system. For example, the graphical user interface (200) may include a delete message icon (290). Choosing the delete message icon (290) deletes a selected message. Additional icons may include, without limitation, a mark as new icon (292), a save message icon (294), a download message icon (296), and a check messages icon (298).
The text of each entry may be selectively formatted or otherwise distinguished to identify the entry as either new or read. According to one example, new messages are listed in bold text while read messages are listed in non-bold text. The mark as new icon (292) formats the text associated with the selected entry to appear as unread or new. The save message icon (294), the download message icon (296), and the check messages icon (298) help manage the interaction between the user interface (200) and the database (130;
While the inbox display (202) is displayed on the example graphical user interface (200), an inbox view selection menu (299) may also be displayed. The inbox view selection menu (299) allows a user to select which aspect of the mailbox she wishes to view. According to one example, an inbox is displayed by default. However, the mail box view selection menu (299) also allows the user to view other aspects of the inbox display (202), including drafts of messages, messages that the user has sent, messages the user has saved, and/or messages that the user has selected for deletion.
In addition to allowing a user to view and manage messages, including threaded messages having audio recordings, the illustrated graphical user interface (200) may be configured to display notes (232) that may be extracted from the audio recordings by using speech analysis.
For example, and according to an embodiment of the invention, the processor (110) may further be configured to utilize a speech analysis engine to perform data analysis of the recipient's voice messages in order to generate topic data containing key words and phrases describing the content of the voice messages. Details of the methods for performing data analysis on the recipient's voice messages in this manner are set forth in U.S. Provisional Patent Application Ser. No. 60/199,978, filed May 12, 2006, which is incorporated herein by reference.
The method 400 further includes, at 404, displaying a graphical user interface screen including a list of at least a portion of the messages and play icons adjacent to each of the messages on the display device. In one embodiment, displaying the graphical user interface screen further includes displaying the messages in a hierarchical manner for providing threaded voice message strings. In another embodiment, displaying the graphical user interface screen also includes displaying notes that are associated with at least a portion of the messages. As described previously, the notes may include text data, for example, describing the subject matter of the voice messages with which the notes are associated.
The method also receives a message selection signal from a user pointing at one of the play icons, as denoted at 400. In response to the message selection signal, the method plays the voice message associated with the selected play icon directly within the graphical user interface screen. The method 400 may further allow a user to select a starting point for playing the message. For example, the method may include receiving a message positioning signal from the user, wherein the method positions the voice message at a selected point to allow the user to listen to the voice message from the selected point. In one embodiment, where the message selected by the user is included within a threaded voice message string, the method may play the voice message associated with the selected play icon and also play at least one of the other messages within the threaded voice message string. By playing multiple messages within the threaded voice message string, the user is provided with additional context for the selected message.
In one embodiment, the method 400 may include receiving a message creation signal from the user. Upon receiving the message creation signal, the method 400 may include displaying a message recording display within the graphical user interface, such as the message recording display illustrated in
Embodiments herein may comprise a special purpose or general-purpose computer including various computer hardware, as discussed in greater detail below. Embodiments may also include computer-readable media for carrying or having computer-executable instructions or data structures stored thereon. Such computer-readable media can be any available media that can be accessed by a general purpose or special purpose computer. By way of example, and not limitation, such computer-readable media can comprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to carry or store desired program code means in the form of computer-executable instructions or data structures and which can be accessed by a general purpose or special purpose computer. When information is transferred or provided over a network or another communications connection (either hardwired, wireless, or a combination of hardwired or wireless) to a computer, the computer properly views the connection as a computer-readable medium. Thus, any such connection is properly termed a computer-readable medium. Combinations of the above should also be included within the scope of computer-readable media.
Computer-executable instructions comprise, for example, instructions and data which cause a general purpose computer, special purpose computer, or special purpose processing device to perform a certain function or group of functions. Although the subject matter has been described in language specific to structural features and/or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.
The present invention may be embodied in other specific forms without departing from its spirit or essential characteristics. The described embodiments are to be considered in all respects only as illustrative and not restrictive. The following claims are presented to illustrate various novel features of the present invention. The claims are not exhaustive of the embodiments of the invention, nor are they to be construed as limiting the scope of the invention. Moreover, the invention disclosed in detail herein can be defined with other claims, including those that will be included in any related non-provisional applications that will be filed during the pendency of this provisional application.
This application claims the benefit of U.S. Provisional Application No. 60/823,065, filed Aug. 21, 2006, which is incorporated herein by reference in its entirety.
| Number | Date | Country | |
|---|---|---|---|
| 60823065 | Aug 2006 | US |
