The invention relates to the field of smartphone messaging systems, and in particular messaging systems requiring multi-threaded topics.
As smartphone messaging systems (“Messaging”) continues to rapidly replace email, SMS and phone calls across consumer and business applications, it faces increasing pressure to act more like a platform with app-like functionality. Messaging platforms like Facebook Messenger and Slack are becoming mainstay communication media for consumers and businesses respectively. In all of this, chat or text-based messaging has evolved very little in its ability to convey complex conversations covering multiple topics, integrate external app functionality and display visual media. This leads to a fractured experience where users need to leave the primary conversation to perform in-context tasks in external apps, discuss related topics or view simple media like video and interactive web.
There is a need therefore for a messaging UI/UX (user interface/user experience) design that fixes the fractured messaging experience, allowing users to remain in the primary conversation and perform more complex topic, task and media execution.
The foregoing examples of the related art and limitations related thereto are intended to be illustrative and not exclusive. Other limitations of the related art will become apparent to those of skill in the art upon a reading of the specification and a study of the drawings.
The following embodiments and aspects thereof are described and illustrated in conjunction with systems, tools and methods which are meant to be exemplary and illustrative, not limiting in scope. In various embodiments, one or more of the above-described problems have been reduced or eliminated, while other embodiments are directed to other improvements.
The present embodiment therefore provides a method of visually organizing multiple parallel chat topics on the display screens of mobile devices used in messaging conversations. Visual media and external apps are integrated into such displays, by forming parallel orthogonally spaced chat threads, where preferably each chat thread has a visual element associated with it displayed in a visual display area. The different chat participants can each manipulate or interact with the visual element of any selected chat thread.
More particularly there is disclosed a method of providing a graphical user interface on the display screens of a plurality of computing devices which are accessible to a computer network to enable a plurality of participants in a messaging conversation to participate in a plurality of parallel chat topics over the course of said conversation, the method comprising: i) providing on the graphical user interface a topic name display area and a vertical chat thread display area; ii) enabling a first participant to start a main chat thread by sending a first message which is displayed in each participant's chat thread display area thereby starting a first main chat topic thread; iii) enabling one or more of said participants to start one or more additional chat topic threads by sending a topic-beginning message which includes a topic name and a topic-creating code and which is displayed in each said participant's chat thread display area; iv) displaying each the topic name in the topic name display area; v) enabling each participant to add messages to each the participant's chat thread display area in relation to one of the topic names by carrying out a topic-adding step; thereby forming a plurality of secondary chat threads in addition to the main chat topic thread in relation to each topic name in parallel with the main chat topic thread. According to a preferred embodiment a visual topic display area is displayed on the graphical user interface, and a participant can select visual content to be displayed in the visual topic display area in association with a topic name. More preferably a participant can navigate between parallel topic threads by selecting a control that replaces a first visual content associated with a first topic name to a second visual content associated with a second or subsequent topic name. Preferably all participants are enabled to view the interaction of one participant with the visual content. In this way all chat participants can interact with the visual content.
The embodiments also describe a non-transitory computer readable storage medium having program code stored thereon for carrying out the method and a system for carrying out the afore-mentioned method.
In addition to the exemplary aspects and embodiments described above, further aspects and embodiments will become apparent by reference to the drawings and by study of the following detailed descriptions.
Exemplary embodiments are illustrated in referenced figures of the drawings. It is intended that the embodiments and figures disclosed herein are to be considered illustrative rather than restrictive.
Throughout the following description specific details are set forth in order to provide a more thorough understanding to persons skilled in the art. However, well known elements may not have been shown or described in detail to avoid unnecessarily obscuring the disclosure. Accordingly, the description and drawings are to be regarded in an illustrative, rather than a restrictive, sense.
When used herein the term “visual” means visual elements in addition to text. For example a visual display area is an area which can display images in addition to or instead of simple text, such as graphic images and video. “Computing devices” may include smartphones, tablets, laptops, desktops, mobile devices and any other computing device that is accessible to a wired or wireless network and has a display screen. References to actions on a touch screen of a computing device include equivalent actions using a keyboard, mouse or other input device. A “messaging conversation” or “chat conversation” includes a series of messages where a number of participants exchange messages including or consisting of text messages wherein there are at least two common participants over the course of the conversation. The series of such messages is referred to as a “chatstream”. A “chat thread” refers to a series of messages in a chatstream which are interrelated, generally by subject or topic, or intended to be responsive to another message in the chat thread.
The ‘fractured’ messaging experience is a problem facing messaging users in part due to current technology where users must leave the chat environment to perform tasks related to the chat, the results of which are difficult to share back into the original chat. This fractured experience creates the problems with the following areas:
To address these issues, the embodiments described below provide a messaging UI/UX which accommodates a richer conversation, keeping everything persistent and in-line in one messaging conversation. This also requires looking at the messaging ‘chatstream’ in any conversation as a continuously growing and evolving graph of nodes and edges rather than just a linear order of related statements. By structuring the chatstream as a growing ‘chat graph’ of comments and media, an improved UI/UX system gives users a superior messaging environment. By providing a more functional and visually oriented chat interface, Google apps, video chat and Dropbox-like capabilities are more naturally integrated into the messaging environment solving the fractured experience problem.
Multiple Parallel Topics & The ‘Kitchen Party’ Problem
The Kitchen Party problem in messaging is a well-known phenomenon wherein in chat groups of three people or more, multiple parallel topics of interest arise within the same group—much like where the kitchen at a party becomes the center of the party where many topics are discussed all within close proximity, where participants freely bounce between conversations. This is a desirable natural mode of conversation. The described Messaging UI/UX delivers a more comfortable and effective messaging experience. The problem with current messaging UI/UX is that topics are forced to be delivered serially in the chatstream and unless a user is prepared to scroll back in the chat history, they are ignorant of any other topics discussed other than what they see in real time. The Kitchen Party problem dictates that in every kitchen party, everyone in the kitchen can hear the topics being discussed and freely contribute to each one as they see fit.
Having more than one topic thread per chat often will result in users leaving the primary chat and having a parallel chat with others from the same group for a separate chat around a particular parallel topic. This increases the complexity of the conversation and the risk that topic discovery via scrolling or search will only show topic keywords out of context to the conversation. To preserve topic context, the user must scroll to see the complete context around the topic of interest to fully understand the conversation around that topic. By dedicating the upper part of the chat UI to displaying all current active topics, the user can determine “What are we talking about? What is going on?”. To preserve context, a Visual Topic Area is provided to display the current topic and mark subsequent chat entries with that topic focus. This allows the entire conversation to be clearly multi-topic, multi-threaded but with the ability to see all active topics and sequentially related content in the chatstream at a glance, as occurs in a Kitchen Party.
External App Integration & the Visual Area Topic Tabs
In most messaging conversations, topics often lead to the need to perform a task such as searching for a place to eat or seeing a movie together. This then leads to the user leaving the primary chat to open an external application (Google search, Yelp or other), perform the task and then try to communicate the result of that task back into the primary chat. The moment the user leaves the primary chat, that user is disconnected from that ongoing conversation and often misses vital information regarding the task they are trying to fulfil. If the user manages to communicate or share the task information into the chatstream, it is sequential like everything else and will disappear in the chat history, making it difficult to resurface in the conversation or discover.
To try to solve this, some messaging platforms use the concept of in-line ‘bots’ to perform tasks which can be invoked and respond like a member of the primary conversation. While this method works for current platform supported features and functions, bots cannot perform tasks beyond what the messaging platform supports and are thus extremely limited. There are a small handful of messaging platforms that have begun to allow some third party app integration to provide greater feature access and customisation for their Users (WeChat, Facebook Messenger, Kik, Slack). While this is preferable, users relying on external apps or webview mediated HTML web-apps to perform tasks and then share the results continue to have the chat history problem which involves scrolling out of view and having to re-scroll to access it in the chat history and miss new chat updates.
The present embodiment forms a Visual Topic Area orthogonal to the chatstream which presents a list of topics and task results, a visual summary of the activity in the conversation. This has the added benefit of allowing other users to invoke the same apps directly from the Visual Topic Area to build on the topic tasks from the previous user. For example using Yelp to search for a restaurant, one could share the result in the Visual Topic Area as a picture of the meal selected, the map of the restaurant's location etc. as a persistent entity at the top of the chatstream. The rest of the chat participants can discuss eating there or not, and could click/tap the Visual Area Tab of the restaurant, which would invoke the Yelp app where they could conduct a different search and push those results in to replace the current Visual Tab contents with a different restaurant's details.
Visual Media Integration & the Visual Area Tabs
Like the problems faced by External apps, Visual media is especially challenged as there is currently no way to display persistent in-line Visual media in messaging, without leaving the primary conversation for an external application like Youtube. This may be addressed by allowing visual media to be pushed into a Visual Topic Area D described below, forming a Visual Tab for each vertical chat thread, and enabling direct collaboration by users on the visual media in the Visual Tab. A user can find a video on Youtube for example and share that in a new Visual Tab with the rest of the users in the primary chat. This enables the video controls to be shared so that one of the users can pause or advance the video for the entire group. This also applies for images, allowing users to zoom in or pan and those actions shown in real time for others to see on their own devices.
With reference to
The foregoing include examples of some possible control features. Many others will be apparent to those skilled in the art, such as zoom, pan, tap and drag, pinch and zoom and the like. An information app may also be provided to display which users are active in the chat, the number of visual tabs, which user injected which visual tab, and all current visual tabs shown as thumbnails. The visual tab may also have a “close” function.
With reference to
In
In
As shown in
By allowing visual media to be pushed into a Visual Topic Area D as described, forming a Visual Tab for each vertical chat thread, which is shared among chat participants, direct collaboration by users on the visual media in the Visual Tab is enabled. When a user finds a video on Youtube for example and share that in a new Visual Tab with the rest of the users in the primary chat, the video controls are shared so that any one of the users can pause or advance the video for the entire group. This also applies for images, allowing users to zoom in or pan and those actions shown in real time for others to see on their own devices.
The Visual Topic Area D may be used for Apps and interactive content, as well as images and video. Apps may be loaded directly into a Visual Tab for collaboration. For example a search may be conducted where each user puts a keyword into the query field and then the search is conducted and results shown in the Visual Topic Area. Group users may collaborate on filling out a form together. Group users may manipulate a map and each show their own position or destination for example. One user can mark
Bot activity may also be displayed in Visual Topic Area D. The Bot may also display its own information to the group of chat participants in Visual Topic Area D. For example a Bot may be executing a script and each stage of this script may display what the external (non-group) user sees in Messenger, for example, where one user is in Facebook Messenger and is linked to a group using one of the present embodiments. When Bots put content into a chat in Messenger (and others), that information also scrolls up and out of view, whereas in the present embodiments the Bot may have its own visual tab as a topic thread so it always has its tasks and results available without having to scroll back through the history of the chat threads.
When a user interacts with the Visual Topic Area such as by injecting an image or video or moving, expanding or annotating the image, the same modification is displayed on the other user displays for those participating in the chat conversation.
According to one aspect, the activity graph shown in
According to the described embodiment therefore, multiple topics can be organized orthogonally in parallel to the main chatstream. This provides a more coherent, understandable visual-based customer interface for the users to conduct multi-topic chats and allows agents to logically organize topics orthogonal to the main chat stream. The addition of a visual area to each vertical topic chatstream into which visual content can be injected adds further practical utility for the chat group.
Example of Application to Facebook Messenger
Thus Messenger currently applies the WebView modal browser window right on top of the conversation, and one cannot type or do anything in Messenger until the WebView window is closed, thereby effectively locking the user out of the conversation. The present visual messaging approach offers better conversational efficiency and focus in that there is a fixed visual area with a fixed topic and the conversation can still run as it should, thereby enhancing the Facebook Messenger user interface and user experience.
While a number of exemplary aspects and embodiments have been discussed above, those of skill in the art will recognize certain modifications, permutations, additions and sub-combinations thereof.
The present application claims the benefits, under 35 U.S.C. § 119(e), of U.S. Provisional Application Ser. No. 62/263,446 filed Dec. 4, 2015 entitled “Visual Messaging Method and System” which is incorporated herein by this reference.
Filing Document | Filing Date | Country | Kind |
---|---|---|---|
PCT/CA2016/051427 | 12/5/2016 | WO | 00 |
Publishing Document | Publishing Date | Country | Kind |
---|---|---|---|
WO2017/091910 | 6/8/2017 | WO | A |
Number | Name | Date | Kind |
---|---|---|---|
6029141 | Bezos et al. | Feb 2000 | A |
6557027 | Cragun | Apr 2003 | B1 |
7039677 | Fitzpatrick et al. | May 2006 | B2 |
7328242 | McCarthy et al. | Feb 2008 | B1 |
7475110 | Kirkland et al. | Jan 2009 | B2 |
7885641 | Tysowski | Feb 2011 | B2 |
8693643 | Mikan et al. | Apr 2014 | B2 |
9117227 | Agrawal | Aug 2015 | B1 |
9286397 | Suleman et al. | Mar 2016 | B1 |
9489458 | Haugen et al. | Nov 2016 | B1 |
20030212746 | Fitzpatrick et al. | Nov 2003 | A1 |
20050149621 | Kirkland et al. | Jul 2005 | A1 |
20050262199 | Chen | Nov 2005 | A1 |
20060059235 | Peterson et al. | Mar 2006 | A1 |
20060195521 | New et al. | Aug 2006 | A1 |
20080208812 | Quoc et al. | Aug 2008 | A1 |
20090106040 | Jones | Apr 2009 | A1 |
20090125499 | Cross et al. | May 2009 | A1 |
20090164449 | Huang | Jun 2009 | A1 |
20090245500 | Wampler | Oct 2009 | A1 |
20090313334 | Seacat et al. | Dec 2009 | A1 |
20100205541 | Rapaport et al. | Aug 2010 | A1 |
20110137884 | Anantharajan et al. | Jun 2011 | A1 |
20110163971 | Wagner | Jul 2011 | A1 |
20110289011 | Hull et al. | Nov 2011 | A1 |
20120143843 | Smyth et al. | Jun 2012 | A1 |
20120158633 | Eder | Jun 2012 | A1 |
20120203846 | Hull et al. | Aug 2012 | A1 |
20130145303 | Prakash | Jun 2013 | A1 |
20130215116 | Siddique et al. | Aug 2013 | A1 |
20130254199 | Surendran et al. | Sep 2013 | A1 |
20140074951 | Misir | Mar 2014 | A1 |
20140136990 | Gonnen | May 2014 | A1 |
20140236953 | Rapaport et al. | Aug 2014 | A1 |
20140324982 | Agrawal et al. | Oct 2014 | A1 |
20140344718 | Rapaport et al. | Nov 2014 | A1 |
20150066479 | Pasupalak et al. | Mar 2015 | A1 |
20150178388 | Winnemoeller et al. | Jun 2015 | A1 |
20150235289 | Jeremais | Aug 2015 | A1 |
20150295873 | Orr | Oct 2015 | A1 |
20150302103 | Jeremias | Oct 2015 | A1 |
Number | Date | Country |
---|---|---|
1998518 | May 2009 | EP |
Entry |
---|
PCT International Search Report dated Jul. 24, 2015 for PCT Appl. No. PCT/CA2015/050444. |
PCT International Search Report dated Jan. 19, 2017 for PCT Appl. No. PCT/CA2016/051427. |
Published article from Avaya Inc. Enhancing Customer Support Through Avaya Automated Chat, 2013. |
Extended European Search Report dated Oct. 9, 2017 for European Appl. No. 15792073.7-1871 / 3143576 PCT/CA2015050444. |
“Conceptual Graphs as a Universal Knowledge Representation”, John F. Sowa, Computers & Mathematics With Applications, vol. 23, No. 2-5, Jan. 1, 1992, pp. 75-93. |
Supplementary European Search Report issued on EP Appl. No. 16869455, completed May 13, 2019. |
Fechner, Thore et al., “Ethermap—Real-time Collaborative Map Editing”, Interactive & Multi-Surface Maps, CHI 2015, Crossings, Seoul, Korea; Human Factors in Computing Systems, ACM, 2 Penn Plaza, Suite 701 New York, NY, USA (Apr. 18, 2015). |
Number | Date | Country | |
---|---|---|---|
20180356952 A1 | Dec 2018 | US |
Number | Date | Country | |
---|---|---|---|
62263446 | Dec 2015 | US |