The disclosure relates generally to processing electronic messages, and more specifically to providing multiple display states for objects representing messages.
Electronic messaging applications, such as email, convey various types of information to a recipient such as messages to and from others. In conventional messaging applications, it is not always possible to ascertain the relative importance of each message in an incoming box without first reviewing their full content. Moreover, even for relatively important messages, it is often necessary to open the message, that is review their full content, in order to derive necessary or useful information. Thus, a recipient of messages falls into a recurring pattern of not only checking for new messages, but also manually opening each message to obtain needed or useful information. What are needed in the art are systems and methods for assisting a recipient with obtaining useful information from messages more quickly and with fewer manual operations.
Disclosed implementations address the above deficiencies and other problems associated with reviewing messages. In accordance with some implementations, a method of processing messages executes at a computing device having one or more processors and memory. The memory stores one or more programs configured for execution by the one or more processors. A message, addressed to a user, is analyzed for the presence of a first structured content element. Nonlimiting examples of structured content elements include those that identify the message as regarding an airline flight, those that identify the message as regarding a hotel reservation, those that identify the message as regarding a restaurant reservation, those that identify the message as regarding a purchase, those that identify the message as regarding a ticketed event, those that identify the message as including an attachment, those that identify the message as including an article or video, those that identify the message as regarding an invitation in which an RSVP is requested.
When the first structured content element is found in the message, a method is performed that comprises executing an action associated with the first structured content element thereby obtaining content from a first data source that is distinct from the message. In some embodiments, the first structured content element is an identification of an airline flight within the message and the data source is current flight information for the airline flight.
Typically, the message is analyzed by comparing the message, or portions of the message to multiple different structured content elements to determine if the message has any such structured content element. When the message is deemed to have or match one of the different structured content elements, action based on this particular different structured content element is taken in order to obtain content from a first data source that is distinct from the message. In some instances, these multiple different structured content elements are predetermined, stored content elements that are system defined, as opposed to user defined.
In some such embodiments, the first structured content element is in a plurality of structured content elements and the analyzing of the message comprises analyzing the message for the presence of each structured content element in the plurality of elements. When a match between the message and one of the different structured content elements is found, an action associated with that one structured content element is performed thereby obtaining content from a data source that is distinct from the message.
In some embodiments, a first display state of the object representing the message is formed when the first structured content element is found in the message. This first display state comprises a combination of a notification of the message and the content found based on the presence of the first structured content element.
In some embodiments, each structured content element in the plurality of content elements is associated with a different image stored in a second data source that is distinct from both the first data source and the message. In such embodiments, forming a first display state comprises including the image associated with the first structured content element (the structured element in the plurality of content elements that was deemed to have been found in the message) in the first display state of the object representing the message along with the combination of the notification of the message and the content. In an example of such an embodiment is one in which the message pertains to a flight reservation, the first data source is real time flight information for a flight in the flight reservation, and the second data source includes an image of the destination of a flight in the flight reservation.
Responsive to identifying the presence of the first structured content element in the message, a user interface for a messaging application is updated. The user interface comprises an electronic message list in an electronic message list panel. The electronic message list comprises a plurality of objects. Each respective object in the plurality of objects is visually distinct in the electronic message list. Each respective object in at least a subset of the plurality of objects comprises one or more electronic messages in a plurality of messages addressed to the user.
Responsive to user selection of a first object of the plurality of objects in the electronic message list panel, the first object representing at least the message, the system facilitates user initiated toggling of a display state of the object representing the message between the first display state and a second display state. The second display state differs from the first display state by providing direct access to the message. That is, whereas the first display state provides a notification of the message in addition to information that is not in the message, it is necessary to go to the second display state in order to see the message, or a direct link to the message.
In some embodiments, a task is associated with the first structured content element and the forming the first display state comprises including information for performing the task in the first display state of the object representing the message along with the combination of the notification of the message and the content. In some embodiments, the information for performing the task is in the form of a uniform resource locator. In some embodiments, the first structured content element identifies an airline flight and the task is an option to cancel the flight. In some embodiments, the first structured content element identifies a hotel or restaurant reservation and the task is an option to cancel the hotel or restaurant reservation, an option to confirm the hotel or restaurant reservation, an option to modify the hotel or restaurant reservation, or an option to view a map of a location of an address associated with the hotel or restaurant reservation. In some such embodiments, the forming the first display state comprises including a date associated with the hotel or restaurant reservation, where the date is retrieved from the message.
In some embodiments, the first structured content element is an airline flight with a flight origin and a flight destination, and the method further comprises obtaining a first image associated with the flight origin and a second image associated with the flight destination from a second data source that is distinct from (i) the message and (ii) the first data source and the forming the first display state comprises including the first image and the second image in the first display state of the object representing the message along with the combination of the notification of the message and the content. An example of content from the first data source in such embodiments is real time flight information.
In some embodiments, the first structured content element identifies a purchase of a product and the content is a picture of the product. In some embodiments, the first structured content element identifies a purchase of a product and the content is an estimated delivery date of the product. In some such embodiments, a task is associated with the first structured content element and the forming the first display state comprises including information for performing the task in the first display state of the object representing the message along with the combination of the notification of the message and the content. For example, in some embodiments, the task is viewing an original order for the product or viewing tracking details for a delivery of the product.
In some embodiments, the first structured content element identifies a ticketed event and the content is a picture associated with the ticketed event. In some such embodiments, a task is associated with the first structured content element and the forming the first display state comprises including information for performing the task in the first display state of the object representing the message along with the combination of the notification of the message and the content. In some embodiments, the task is viewing tickets for the ticketed events or viewing a map associated with a location of the ticketed event.
In some embodiments, the first structured content element is a link to an image and the content is a rendering of the image. In some embodiments, the first structured content element is a link to a video and the content is a frame from the video.
In some embodiments, the first object exclusively represents the message. That is, in such embodiments, the first object does not represent two or more messages. In other embodiments, the first object represents a plurality of messages and the object includes the first display state of the object representing the message.
Another aspect of the present disclosure provides a computing device comprising one or more processors, memory, and one or more programs stored in the memory. The one or more programs are configured for execution by the one or more processors. The one or more programs comprise instructions for analyzing a message, addressed to a user, for the presence of a first structured content element. When the first structured content element is found in the message, a method is performed that comprises (i) executing an action associated with the first structured content element thereby obtaining content from a first data source that is distinct from the message, and (ii) forming a first display state of the object representing the message. The first display state of the object representing the message comprises a combination of a notification of the message and the content. Responsive to identifying the presence of the first structured content element in the message, a user interface for a messaging application is updated. The user interface comprises an electronic message list in an electronic message list panel. The electronic message list comprises a plurality of objects. Each respective object in the plurality of objects is visually distinct in the electronic message list. Each respective object in at least a subset of the plurality of objects comprises one or more electronic messages in a plurality of messages addressed to the user. Responsive to user selection of a first object of the plurality of objects in the electronic message list panel, the first object representing at least the message, user initiated toggling of a display state of the object representing the message between first and second display states is facilitated. The second display state differs from the first display state by providing direct access to the message.
Another aspect of the present disclosure provides a non-transitory computer readable storage medium storing one or more programs configured for execution by a computing device having one or more processors and memory. The one or more programs comprise instructions for analyzing a message, addressed to a user, for the presence of a first structured content element. When the first structured content element is found in the message, a method is performed that comprises (i) executing an action associated with the first structured content element thereby obtaining content from a first data source that is distinct from the message, and (ii) forming a first display state of the object representing the message. The first display state of the object representing the message comprises a combination of a notification of the message and the content. Responsive to identifying the presence of the first structured content element in the message, a user interface for a messaging application is updated. The user interface comprises an electronic message list in an electronic message list panel. The electronic message list comprises a plurality of objects. Each respective object in the plurality of objects is visually distinct in the electronic message list. Each respective object in at least a subset of the plurality of objects comprises one or more electronic messages in a plurality of messages addressed to the user. Responsive to user selection of a first object of the plurality of objects in the electronic message list panel, the first object representing at least the message, user initiated toggling of a display state of the object representing the message between first and second display states is facilitated. The second display state differs from the first display state by providing direct access to the message.
For a better understanding of the aforementioned implementations of the invention as well as additional implementations thereof, reference should be made to the Description of Implementations below, in conjunction with the following drawings in which like reference numerals refer to corresponding parts throughout the figures.
Reference will now be made in detail to implementations, examples of which are illustrated in the accompanying drawings. In the following detailed description, numerous specific details are set forth in order to provide a thorough understanding of the present invention. However, it will be apparent to one of ordinary skill in the art that the present invention may be practiced without these specific details.
In some implementations, the server system 110 consists of a single server 300. More commonly, the server system 110 includes a plurality of servers 300. In some implementations, the servers 300 are connected by an internal communication network of bus 124. The server system 110 includes one or more web servers 112, which receive requests from users (e.g., from client devices 102) and return appropriate information, resources, links, and so on. In some implementations, the server system 110 includes one or more application servers 114, which provide various applications, such as a messaging application 106. The server system 110 typically includes one or more databases 116, which store information such as web pages, a user list 118, various user information 120 (e.g., user names and encrypted passwords, user preferences, and so on), a plurality of predefined candidate task reminders 122, first data source 126 and second data source 128.
In some implementations, the memory 214 includes high-speed random access memory, such as DRAM, SRAM, DDR RAM or other random access solid state memory devices. In some implementations, memory 214 includes non-volatile memory, such as one or more magnetic disk storage devices, optical disk storage devices, flash memory devices, or other non-volatile solid state storage devices. In some implementations, the memory 214 includes one or more storage devices remotely located from the CPU(s) 202. The memory 214, or alternately the non-volatile memory device(s) within memory 214, comprises a non-transitory computer readable storage medium. In some implementations, the memory 214, or the computer readable storage medium of memory 214, stores the following programs, modules, and data structures, or a subset thereof:
Each of the above identified executable modules, applications, or sets of procedures may be stored in one or more of the previously mentioned memory devices and corresponds to a set of instructions for performing a function described above. The above identified modules or programs (e.g., sets of instructions) need not be implemented as separate software programs, procedures, or modules, and thus various subsets of these modules may be combined or otherwise re-arranged in various implementations. In some implementations, the memory 214 stores a subset of the modules and data structures identified above. Furthermore, in some embodiments, the memory 214 stores additional modules or data structures not described above.
Although
In some implementations, the memory 314 includes high-speed random access memory, such as DRAM, SRAM, DDR RAM or other random access solid state memory devices. In some implementations, the memory 314 includes non-volatile memory, such as one or more magnetic disk storage devices, optical disk storage devices, flash memory devices, or other non-volatile solid state storage devices. In some implementations, the memory 314 includes one or more storage devices remotely located from the CPU(s) 302. The memory 314, or alternately the non-volatile memory device(s) within memory 314, comprises a non-transitory computer readable storage medium. In some implementations, the memory 314, or the computer readable storage medium of memory 314, stores the following programs, modules, and data structures, or a subset thereof:
Candidate task reminders 122 and/or 228 are in any form suitable for reminding a user of a task. Moreover, it will be appreciated that
Each of the above identified elements in
Although
As illustrated in
In
Continuing to refer to
In some embodiments, when the first structured content element is found in the message, an action associated with the first structured content element is performed thereby obtaining content from multiple data sources, such as the message that includes the first structured content element, one or more databases, and/or one or more other messages received by the user. For instance, in some embodiments, information from across multiple messages is obtained. As one example, information is obtained from one message that pertains to an order confirmation and additional information is obtained from another message that pertains to order shipment. As another example, information is obtained from one message that pertains to a hotel reservation confirmation and additional information is obtained from another message that pertains to cancellation of the hotel reservation.
When a structured content element is found in the message and content received from the data source distinct from the message, a first display state of a first object representing at least the message is formed. The first display state of the first object comprises a combination of a notification of the message and the content that was retrieved from a data source that is distinct from the message (e.g., the current departure time for the flight). It will be appreciated that the notification of the message is not the message itself, nor does it provide direct access to the message. The user interface 402 is updated to reflect the addition of the object representing the message. In this example, the message (not shown) included two flights, and thus information on departure time for both flights is provided in an object 412 that represents the message. As
It will be appreciated that data can be obtained from any number of data sources distinct from the message to create the first display state of the object representing the message in addition to, or instead of, information from the message itself. Accordingly, in some embodiments, the identification of the first structured content element is used as a basis for obtaining content from more than one data source that is distinct from the analyzed (first) message. For instance, in the message that forms the basis of creating object 412 of
In some instances, a task is also inferred from the message and added to the object representing the message. Ways to infer a task from an electronic message are disclosed in U.S. patent application Ser. No. 14/161,368, entitled “Identifying Tasks in Messages,” filed Jan. 22, 2014, which is hereby incorporated by reference herein in its entirety. In some embodiments a message is processed in order to determine whether the message includes a task by determining, for each respective predefined candidate task in a plurality of predefined tasks, whether there is a match between the predefined candidate task and all or a portion of the respective message. In some embodiments, this match between the predefined candidate task and all or a portion of the respective message includes natural language processing of the all or a portion of the respective message as disclosed in U.S. patent application Ser. No. 14/161,368, entitled “Identifying Tasks in Messages,” filed Jan. 22, 2014, which is hereby incorporated by reference herein in its entirety. In some embodiments, each predefined candidate task is a rule, heuristic, phrase, or other expression that is matched or applied against respective messages addressed to a user (e.g., the incoming message) to see if there is a match between the predefined candidate task and all or a portion of the respective message. In some embodiments, a predefined candidate task in the plurality of candidate tasks is from a source other than (external to, independent of) the user. For instance, in some embodiments there is a library of standard or common predefined candidate task formats that are provided for matching. When a match is deemed to have been found, the object corresponding to the message that had a match is updated to include the task derived from the corresponding message. In some alternative embodiments, the task is determined from the identity of the structured content element found in the message. Thus, in the case of
Responsive to user selection of object 412, user initiated toggling of the display state of the object 412 between the first display state (412-1) and a second display state (412-2) is facilitated.
As
In some embodiments, the expanded display state 412-2 of
Turning once again to
In some embodiments, the collapsed display state 412-1 of
Prior examples provided object display states for messages that were not incorporated into clusters. As indicated before, it is possible for a message to be clustered in to a cluster using, for example the techniques disclosed in U.S. patent application Ser. No. 14/145,005, entitled “Systems and Methods for Throttling Display of Electronic Messages,” filed Dec. 31, 2013, and U.S. patent application Ser. No. 14/139,205, entitled “Systems and Methods for Clustering Electronic Messages,” filed Dec. 23, 2013, each of which is hereby incorporated by reference herein in its entirety.
Referring to
A message, addressed to a user, is analyzed for the presence of a first structured content element. When the first structured content element is found in the message, a method is performed that comprises (i) executing an action associated with the first structured content element thereby obtaining content from a first data source that is distinct from the message and (ii) forming a first display state of a first object representing at least the message, where the first display state of the first object comprises a combination of a notification of the message and the content (506). In some embodiments, the first structured content element is an identification of an airline flight and the first data source is current flight information for the airline flight 508.
In some embodiments, the first structured content element is in a plurality of candidate structured content elements and the analyzing of the message comprises analyzing the message for the presence of each candidate structured content element in the plurality of candidate structured content elements thereby identifying the first structured content element as a match between all or a portion of the message and a candidate structured content element of the plurality of candidate structured content elements (510). In some such embodiments, each candidate structured content element in the plurality of candidate structured content elements is associated with a different image stored in a second data source that is distinct from the message, and the forming the first display state comprises including the image associated with the first structured content element in the first display state of the first object along with the combination of the notification of the message and the content (512).
In some embodiments, there is a task is associated with the first structured content element and the forming the first display state comprises including information for performing the task in the first display state of the first object along with the combination of the notification of the message and the content (514). In some embodiments, the information for performing the task is in the form of a uniform resource locator (516). In some embodiments, the first structured content element identifies an airline flight and the task is an option to cancel the flight (518). In some embodiments, the first structured content element identifies a hotel or restaurant reservation and the task is an option to cancel the hotel or restaurant reservation, an option to confirm the hotel or restaurant reservation, an option to modify the hotel or restaurant reservation, or an option to view a map of a location of an address associated with the hotel or restaurant reservation (520). In some such embodiments, the forming the first display state comprises including a date associated with the hotel or restaurant reservation, wherein the date is retrieved from the message (522).
In some embodiments, the first structured content element is an airline flight with a flight origin and a flight destination and the method further comprises obtaining a first image associated with the flight origin and a second image associated with the flight destination from a second data source that is distinct from (i) the message and (ii) the first data source; and the first display state includes the first image and the second image along with the combination of the notification of the message and the content (524).
In some embodiments, the first structured content element identifies a purchase of a product and the content is a picture of the product (526).
In some embodiments, the first structured content element identifies a purchase of a product and the content is an estimated delivery date of the product (528). In some such embodiments, there is a task is associated with the first structured content element and the forming the first display state comprises including information for performing the task (e.g., viewing an original order for the product or viewing tracking details for a delivery of the product) in the first display state of the first object along with the combination of the notification of the message and the content (530).
In some embodiments, the first structured content element identifies a ticketed event and the content is a picture associated with the ticketed event (532). In some such embodiments, a task is associated with the first structured content element and the forming the first display state comprises includes information for performing the task (e.g., viewing tickets for the ticketed events or viewing a map associated with a location of the ticketed event) in the first display state of the message along with the combination of the notification of the message and the content (534).
In some embodiments, the first structured content element is a link to an image and the content is a rendering of the image (536). In some embodiments, the first structured content element is a link to a video and the content is a frame from the video (538). In some embodiments, the first object exclusively represents the message (540). In some embodiments, the first object represents a plurality of messages (542).
Responsive to identifying the presence of the first structured content element in the message, a user interface for a messaging application is updated. The user interface comprises an electronic message list in an electronic message list panel. The electronic message list comprises a plurality of objects including the first object. Each respective object in the plurality of objects is visually distinct in the electronic message list. Further, each respective object in at least a subset of the plurality of objects comprises one or more electronic messages in a plurality of messages addressed to the user (544). Responsive to user selection of the first object, user initiated toggling of a display state of the first object is facilitated between the first display state and a second display state, where the second display state differs from the first display state by providing direct access to the message and where the first display state is a collapsed display state and the second display state is an expanded display state (546).
The terminology used in the description of the invention herein is for the purpose of describing particular implementations only and is not intended to be limiting of the invention. As used in the description of the invention and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that the term “and/or” as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items. It will be further understood that the terms “comprises” and/or “comprising,” when used in this specification, specify the presence of stated features, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, steps, operations, elements, components, and/or groups thereof.
The foregoing description, for purpose of explanation, has been described with reference to specific implementations. However, the illustrative discussions above are not intended to be exhaustive or to limit the invention to the precise forms disclosed. Many modifications and variations are possible in view of the above teachings. The implementations described herein were chosen and described in order to best explain the principles of the invention and its practical applications, to thereby enable others skilled in the art to best utilize the invention and various implementations with various modifications as are suited to the particular use contemplated.
This application is a continuation of U.S. application Ser. No. 16/160,234, filed Oct. 15, 2018, which is a continuation of U.S. application Ser. No. 14/473,571, filed Aug. 29, 2014, now U.S. Pat. No. 10,104,028, which are incorporated herein by reference in their entirety.
Number | Name | Date | Kind |
---|---|---|---|
6463420 | Guidice | Oct 2002 | B1 |
8949283 | Cornwell et al. | Feb 2015 | B1 |
9124546 | Sharp et al. | Sep 2015 | B2 |
9191486 | Tseng et al. | Nov 2015 | B2 |
9503509 | Hunt et al. | Nov 2016 | B1 |
10104028 | Agarwal et al. | Oct 2018 | B2 |
20050268237 | Crane | Dec 2005 | A1 |
20060178951 | Rund, III | Aug 2006 | A1 |
20070067196 | Usui | Mar 2007 | A1 |
20070198308 | Crean et al. | Aug 2007 | A1 |
20080086703 | Flynt et al. | Apr 2008 | A1 |
20090012824 | Brockway et al. | Jan 2009 | A1 |
20110154338 | Ramanathaiah et al. | Jun 2011 | A1 |
20110225257 | Tilden et al. | Sep 2011 | A1 |
20110264685 | Cheng | Oct 2011 | A1 |
20120042024 | Kirshenboim | Feb 2012 | A1 |
20120117507 | Tseng | May 2012 | A1 |
20120221474 | Eicher et al. | Aug 2012 | A1 |
20130124238 | Burrows et al. | May 2013 | A1 |
20140070945 | Dave et al. | Mar 2014 | A1 |
20140149308 | Ming | May 2014 | A1 |
20140164478 | Kleppmann | Jun 2014 | A1 |
20150052035 | Caiman et al. | Feb 2015 | A1 |
20150205782 | Subramanya et al. | Jul 2015 | A1 |
20190052589 | Agarwal et al. | Feb 2019 | A1 |
Number | Date | Country |
---|---|---|
WO-2005054990 | Jun 2005 | WO |
Entry |
---|
Bernard Kerr and Eric Wilcox. “Designing remail: reinventing the email client through innovation and integration.” Apr. 2004. ACM, CHI '04 Extended Abstracts on Human Factors in Computing Systems, pp. 837-852. (Year: 2004). |
Prosecution History from U.S. Appl. No. 14/473,571, dated May 31, 2017 through Jun. 11, 2018, 92 pp. |
Prosecution History from U.S. Appl. No. 16/160,234, dated Jan. 10, 2020 through Nov. 5, 2020, 52 pgs. |
Number | Date | Country | |
---|---|---|---|
20210194838 A1 | Jun 2021 | US |
Number | Date | Country | |
---|---|---|---|
Parent | 16160234 | Oct 2018 | US |
Child | 17192466 | US | |
Parent | 14473571 | Aug 2014 | US |
Child | 16160234 | US |