Patent Application
20130024521
A growing amount of communication involves the exchange of electronic messages. With advances in technology, the number of electronic messaging modalities has also grown. With a growing number and use of different messaging modalities, communication may become more convenient to a user. However, the increasing number of messaging modalities may result in difficulties associated with managing, tracking, and organizing message data that is exchanged electronically, especially when multiple messaging modalities or multiple devices are employed. These difficulties are further compounded as the amount of message data to manage, track, and organize increases. As such, the overall communication effectiveness of users may be diminished.
For instance, the incorporation of messaging modalities into mobile communication devices may enable users to use a mobile device to exchange electronic messages. Many mobile devices in use today have the ability to communicate using a number of messaging modalities including, but not limited to, voice communication, voice over Internet Protocol (VoIP), voicemail, email, short message service (SMS or “text message”), multimedia messaging service (MMS), instant messaging or other chat protocols, video calling, social media posts (e.g., Twitter ®, Facebook®, etc), or any combination thereof. Depending on, for example, the nature of the communication, the recipient(s) of the communication, the location of the user, and/or the user's preference, a user may choose any one of or a plurality of different messaging modalities to exchange messages. As such, related message data may be spread among a number of messaging modalities or devices such that it may be difficult for a user to quickly and easily access related messages (e.g., messages belonging to a common message thread).
This may present difficulties when a user attempts to search messages. The message data a user desires to search may be spread among a plurality of messaging modalities and/or devices. Furthermore, traditional keyword searching may prove ineffective to find related messages due to potential synonymy and polysemy of the vocabulary used in messages. As such, keyword searching may return unrelated messages or overlook related messages due to variations in the terms used. As such, the users communication effectiveness may be further reduced.
A first aspect includes a method for use of a communication system. The method includes storing a plurality of portions of message data in machine readable format on at least one storage device. The plurality of portions of message data correspond to messages exchanged via one or more electronic messaging modalities. The method further includes performing a semantic analysis on said plurality of portions of message data. The method also includes identifying, based on said semantic analysis, at least two related portions of said plurality of portions of message data. The method further includes notifying a user of said at least two related portions by way of a notice provided on a user interface.
A number of feature refinements and additional features are applicable to the first aspect. These feature refinements and additional features may be used individually or in any combination. As such, each of the following features that will be discussed may be, but are not required to be, used with any other feature or combination of features of the first aspect.
For example, in one embodiment, the method further includes indexing said plurality of portions of message data. The method may also further include representing corresponding content of each said plurality of portions of message data as a vector. The vector may be stored in the indexing operation. The vector may be defined in multi dimensional Euclidian space.
Additionally, the semantic analysis may include evaluating a relationship between at least a first vector and a second vector. The relationship may describes the relatedness of said first vector and said second vector. The semantic analysis may include performing a distance analysis with respect to said first vector and said second vector. The identifying may be based on the relationship such that if said relationship exceeds a threshold value, a first portion of message data corresponding to said first vector and a second portion of message data corresponding to said second vector comprise said at least two related portions.
In one embodiment, the first portion of message data may be associated with a first messaging modality and said second portion of message data may be associated with a second messaging modality different than said first messaging modality. The first messaging modality and the second messaging modality may be individually chosen from the group comprising voice communication, VoIP, voicemail, email, text messaging, multimedia messaging, instant messaging, video calling, social media posts, or any combination thereof.
The first portion of message data may be stored on a first device and said second portion of message data is stored on a second device. The method may also include accessing at least one of said first portion of message data or said second portion of message data from said notice provided by said user interface. At least one of said first portion of message data or said second portion of message data may be transcribed from a first form to a second form prior to performing said semantic analysis. The first form may include audio data and said second form may be textual data.
A second aspect includes a communication system. The system includes a semantic analysis module that has access to at least first portion of message data and second portion of message data. The semantic analysis module is operable to calculate a first vector corresponding to content of said first portion of message data and a second vector corresponding to content of said second portion of message data. The semantic analysis module is further operable to determine a relationship between said first vector and said second vector. If said relationship exceeds a predetermined threshold said first portion of message data and said second portion of message data are identified as being related. The system also includes an alert module in operative communication with said semantic analysis module such that if said first vector and said second vector are identified as being related, the alert module is operative to notify a user that said first portion of message data and said second portion of message data are related.
A number of feature refinements and additional features are applicable to the second aspect. These feature refinements and additional features may be used individually or in any combination. As such, each of the following features that will be discussed may be, but are not required to be, used with any other feature or combination of features of the any of the second aspects.
For example, at least one of said first portion of message data or said second portion of message data may be stored locally with respect to the semantic analysis module. Additionally or alternatively, at least one of said first portion of message data and said second portion of message data may be stored remotely with respect to the semantic analysis module. In one embodiment, the first portion of message data may be associated with a different messaging modality than said second portion of message data.
In one embodiment, at least one of said first portion of message data and said second portion of message data comprise body data of a corresponding message. Additionally or alternatively, at least one of said first portion of message data and said second portion of message data may comprise metadata of a corresponding message.
A third aspect includes a method for a mobile computing device operative to access at least a first portion of message data associated with a first messaging modality and a second portion of message data associated with a second messaging modality. The method includes calculating a first vector describing said first portion of message data and a second vector describing said second portion of message data and comparing said first and second vector to quantify a distance relationship between said first vector and said second vector. The method also includes presenting a cross-modal alert to a user based on said distance relationship. The cross-modal alert provides access to information regarding said first portion of message data and said second portion of message data.
The present disclosure is generally related to systems and methods for use of multiple communication messaging systems. Specifically, message data may be analyzed to determine the relatedness of different portions of message data. For instance, a semantic analysis technique may be used to analyze a plurality of portions of message data. Related message data may be identified as a result of the semantic analysis and a user may be notified of any portions of message data related to, for example, a message of interest. The related portions of message data may also be accessed (e.g., read) by the user. The message data analyzed may be associated with a plurality of different messaging modalities such as, for example, cellular voice communication, voicemails, emails, text messages, MMS messages, instant messages, chats, VoIP communication, video calling, social media posts (e.g., to Twitter®, Facebook®, etc.), or any combination thereof. Furthermore, the message data may be generated using (or stored on) a plurality of different devices (e.g., personal computers, mobile phones, personal digital assistants, tablets, etc.). The identification of related portions of message data may provide for a more robust evaluation of potentially large amounts of message data spread across a number of different communication modalities and/or a number of different devices to retrieve related messages and help prevent overlooked, unread, or lost messages. Furthermore, in contrast to keyword searches, the present invention facilitates a more robust semantic analysis to identify related message data that accounts for potential variations (e.g., synonymy and/or polysomy) in the language in related portions of message data.
Identifying related portions of message data that have been sent or received may include storing machine readable portions of message data. Portions of message data may correspond to at least a portion of a message. That is, a portion of message data may not, but may encompass the entire message. As such, a message may have a plurality of portions of message data which is stored. These stored portions of message data may be indexed such that a representation of the content of the message data is stored (e.g., locally or remotely) in a message data index.
As an example, a message thread (i.e., a series of messages exchanged related to or regarding a similar topic or conversation) may be exchanged using more than one messaging modality or more than one device. Due to, for instance, use of multiple different messaging modalities or multiple different devices, messages in the message thread may be overlooked, unread, or lost. For sensitive communication (e.g., such as that associated with business transactions, legal affairs, diplomatic communication, etc.), it may be important for a user to be notified of or be able to easily access related messages. The fact that much electronic communication may occur rapidly (e.g., near instantaneously in some instances) serves to further complicate the ability to effectively manage messages. For instance, in the time it takes to generate and send a reply to an original message, a subsequent related message may be received, of which the user is unaware.
In another example, a user may send a text message from a mobile device in a first instance. The receiver of the text message may respond to the sender via an email. The sender may receive the email reply on a laptop computer. A response to the reply may in turn be sent using a chat program on a tablet device. As such, three different messaging modalities using at least three different devices were used to carry out the message thread in this example provided. Especially when considering the possibility that a number of different parallel message threads may exist, the ability to manage, track, or organize related messages may become burdensome.
In this regard, the indexing of messages may allow for a representation (e.g., a vector existing in Euclidian space having multiple dimensions) to be generated or calculated that describes the content of a portion of message data. In turn, a comparison (e.g., a distance analysis) may be carried out on a plurality of the representations. Based on one or more thresholds regarding the relationship between the representations of the various portions of message data, vectors exhibiting a relationship that exceeds the one or more thresholds may be determined to be related. As such, a user may be presented with an alert notifying him or her of the related messages corresponding to the representations that exhibit a relationship exceeding a threshold. The alert may be presented to a user upon a number of different events or upon request by the user.
Reference will now be made to the accompanying drawings, which at least will assist in illustrating the various pertinent features of the present invention. In this regard, the following description is presented for purposes of illustration and description and is not intended to limit the invention to the form disclosed herein. Consequently, variations and modifications commensurate with the following teachings, skill, and knowledge of the relevant art, are within the scope of the present invention. The embodiments described herein are further intended to enable others skilled in the art to utilize the invention as described or in other embodiments and with various modifications required by the particular applications or uses of the present invention.
With reference to
In one embodiment, a representation of the content of each of the plurality of portions of message data 102 may be stored in the messaging data index 110. The representations of the content of the plurality of portions of message data 102 may take a number of forms. For example, a mathematical construct such as a matrix, vector, or the like may be used to represent the content of the plurality of portions of message data.
A semantic analysis module 120 may be operative to access the message data index 110. As stated above, the message data index 110 may be stored locally or remotely with respect to the semantic analysis module 120. As such, the message data index 110 may reside on the same device on which the semantic analysis module 120 executes or may reside on a device separate from the device on which the semantic analysis module 120 executes. The semantic analysis module 120 may utilize the representations of the content of the plurality of portions of message data 102 stored in the message data index 110 in order to compare the different plurality of portions of message data 102 to identify related messages.
In one particular embodiment, the semantic analysis module 120 may access the message data index 110 and retrieve a representation of at least two of the plurality of portions of message data 102. In one embodiment, the semantic analysis module 120 may be operative to access a local message data index 110, for example, residing in a memory of the device executing the semantic analysis module 120. Alternatively or additionally, the semantic analysis module may be operative to access a remote memory to retrieve information. The remote memory may be associated with another device used to exchange messages or may comprise a remote data repository. In this regard, the semantic analysis module 120 may be in operative communication with the remote memory (e.g., via a wireless or wired network or the like). In the case of a remote data repository, any device a user utilizes to exchange messages may be operative to store message data or representations of the content thereof at the remote data repository. Accordingly, regardless of the device used to communicate, a copy of message data or a representation of the content thereof may be stored at the remote data repository.
The representations retrieved by the semantic analysis module 120 may be a vector describing a corresponding portion of message data 102 in Euclidian space. For example, the vector may be generated by way of mathematical operations performed on a matrix describing the usage of terms which comprise the body or metadata of the message data. The Euclidian space may have a plurality of dimensions in which a vector may have a component. Such Euclidian space is not limited to any number of dimensions such that the number of dimensions may be selected based on a number of factors (e.g., the amount of message data indexed, etc.). As each portion of message data 102 may have a corresponding vector which describes content of that portion of the message data, the relatedness of the plurality of portions of message data 102 may be analyzed by comparing the relationship between the corresponding vectors.
The relatedness of one or more portions of message data 102 may be described by a relationship between vectors corresponding to the one or more portions of message data 102. In one embodiment, a distance analysis may be performed on at least two vectors to determine the relationship between the at least two vectors. A threshold relationship between vectors may be established such that vectors having a relationship that exceeds the threshold (e.g., within a certain distance of one another) may be identified as having a relationship above the threshold. Vectors exceeding the threshold relationship indicate the message portions to which the vectors correspond are related. As such, the semantic analysis module 120 may be operative to compare the relatedness of message data by measuring a relationship between vectors corresponding to the message data portions to determine the relatedness of the messages. Other means of quantifying or describing the relationship between vectors may be used (e.g., relative angular positions, relative lengths, etc.). Threshold relationships may likewise be determined for these other means of describing the relationship between vectors to identify related message portions.
In one embodiment, the semantic analysis module 120 may perform a semantic analysis technique similar to that described in U.S. Pat. No. 4,839,453, the entirety of which is incorporated by reference. Alternative methods of analysis including, but not limited to, latent semantic indexing, latent semantic analysis, probabilistic latent semantic analysis, probabilistic latent semantic indexing, n-gram analysis, topic-based vector space models, latent dirichlet allocation or other appropriate type of semantic analysis or information retrieval technique may be employed by the semantic analysis module 120 as well, without limitation.
In any regard, an alert module 130 may be in operative communication with the semantic analysis module 120. The alert module 130 may be operative to provide an alert to a user when related messages are identified by the semantic analysis module 120. The alert module 130 may be operative to generate an alert which is presented to the user (e.g., by way of a user interface). The alert presented to the user by the alert module 130 may notify the user that portions of message data exist that are related to a message of interest.
The alert module 130 may be responsive one or more of a plurality of events. A message of interest may be identified for such an event. The message of interest may comprise message data to which the plurality of portions of message data 102 is compared. The message of interest may be compared to the plurality of portions of message data 102 to identify portions of message data related to the message of interest. In one embodiment, the alert module 130 may be responsive to the sending of a message by a user. The message of interest in this embodiment may be the outgoing message. As such, prior to sending the message of interest, the semantic analysis module 120 may perform an analysis on the message of interest against the message data index 110 to identify portions of message data 102 related to the message of interest. Thus, the user may be alerted to the presence of related portions of messages data prior to the sending of the message of interest.
In another example, a user may select a message to read. In this regard, the selected message may be identified as a message of interest. Again, the semantic analysis module 120 may perform an analysis on the message of interest against the message data index 110 to indentify portions of message data related to the message of interest. As such, upon selecting a message to read, an alert may be generated by the alert module 130 informing the user of related portions of message data. Other events may also result in an alert from the alert module. Examples may include alerting a user upon identifying related message portions to a message when the message is received. In this case, the received message may be the message of interest. Additionally or alternatively, the user may be alerted to related messages upon other actions taken with respect to a message (e.g., deletion of a message, forwarding of a message, replying to a message, indicating interest or approval of a message, etc.). Additionally, the user may be able to manually select a message in order to find a related portions of message data.
The alert module 130 may also be in operative communication with integration module 140. The integration module 140 may be operative to integrate related portions of message data 102 and present the related portions of message data 102 to the user. For example, the user may access the related portions of message data 102 presented in an alert generated by the alert module 130. The integration module 140 may be operative to access the related portions of message data. As such, in examples presented above, when a user receives an alert regarding related portions of message data, the user may access the related portions of message data or the messages from which the related portion is taken.
The modules described above, including the semantic analysis module 120, alert module 130, and integration module 140, may be characterized by software, firmware, or hardware of any kind which is capable of functioning in the manner described above. Furthermore, functionality of the modules described above may be integrated into less modules than described or may be separated into additional modules. In one particular embodiment, the modules may comprise one or more portions of machine readable code stored in at least one memory that is accessible by one or more microprocessors to perform the functionality described above.
Turning to
The mobile device 250 may be operable to send or receive messages via a plurality of messaging modalities 258. For example, the processor 254 may execute clients (e.g., communication programs or other applications executable by the processor 254) operative to communicate according to messaging modality protocols. The messaging modalities 258 of the mobile device 250 may include, but are not limited to, voice communications, VoIP, voicemail, email, text messages, MMS messages, instant messages, chat transcripts, video calling, social media posts, other messaging modalities now available or later developed, or any combination thereof. When receiving or sending messages by way of messaging modalities 258, a machine readable record corresponding to the message may be stored (e.g., in memory 262) on the mobile device 250. Additionally or alternatively, a machine readable record corresponding to messages may be stored remotely (e.g., at remote device 210) at a data repository or the like.
The processor 254 as shown in
The processor 254 may also be in operative communication with remote message data index 220 located at a remote device 210. For example, the remote device 210 may be another device owned or controlled by a user of the mobile device 250. Additionally or alternatively, the remote device 210 may be a server or the like for storing the remote message data index 220 (e.g., the remote device 210 may serve as a data repository). In various embodiments, the remote device 210 may be a computer at the user's work, a computer at the user's home, a tablet, or other device the user may use to send or receive messages. In this regard, the remote device 210 may also have a plurality of messaging modalities associated therewith for sending and receiving messages. Alternatively, the remote device 210 may serve only as a data repository for storing the remote message data index 220. In any regard, the remote message data index 220 and/or the local message data index 256 may be accessible by the processor 254 of the mobile device 250.
The processor 254 of the mobile device 250 may be operative to perform a semantic analysis (such as the semantic analysis operation described above in
Also depicted in
The process 300 may further include indexing the message data 330. The indexing may include generation of a matrix, vector, or other mathematical construct which represents the content of the message data. As such, the process 300 may further include calculating 340 a representation of the content of a plurality of portions of message data to be stored in the index. The process may further include comparing 350 first and second message data representations. For example, the comparing step 350 may include performing a distance analysis on a plurality of vectors representing portions of message data.
The process 300 may further include alerting 360 a user to relatedness of messages as determined in the comparing operation 350. Additionally, the process 300 may include accessing 370 the subject messages of the alert. In this regard, the alert may include the ability to access the related messages identified in the comparing step 350 such that the user may view the related messages as presented by the alert. For example, the alert may include a portion of the related message data, a link to the related message, or other means of accessing the related message.
While the invention has been illustrated and described in detail in the drawings and foregoing description, such illustration and description is to be considered as exemplary and not restrictive in character. For example, certain embodiments described hereinabove may be combinable with other described embodiments and/or arranged in other ways (e.g., process elements may be performed in other sequences). Accordingly, it should be understood that only the preferred embodiment and variants thereof have been shown and described and that all changes and modifications that come within the spirit of the invention are desired to be protected.
