The background description provided herein is for the purpose of generally presenting the context of the disclosure. Work of the presently named inventors, to the extent it is described in this background section, as well as aspects of the description that may not otherwise qualify as prior art at the time of filing, are neither expressly nor impliedly admitted as prior art against the present disclosure.
Electronic messaging, such as text messaging and emailing, has become one of the primary methods by which people communicate. There are times, however, when it can be difficult for a user to type an electronic message using a keyboard. Conventional systems can suggest one or more of a few pre-generated messages, such as in response to a shorthand input (“brt” can result in the suggestion “be right there,” “omw” can result in the suggestion “on my way,” etc.). These pre-generated messages, however, are generic and lack any specific context, such as a context of the user's current activity.
According to one aspect of the present disclosure, a computer-implemented method is presented. The method can include detecting, by a computing device having one or more processors, an initiation of composing an electronic message by a user; obtaining, by the computing device, contextual information for the electronic message from a source external to a text of the electronic message; obtaining, by the computing device, a first suggestion for the text of the electronic message based on the contextual information; detecting, by the computing device, an operating condition indicative of a user activity during which the user is likely to experience difficulty in typing; in response to detecting the operating condition, obtaining, by the computing device, a second suggestion for the electronic message based on the contextual information, the second suggestion being more detailed than the first suggestion; and outputting, by the computing device, one of the first and second suggestions depending on one or more other conditions.
In some implementations, the detecting of the operating condition is based on a measured gravitational force. In some implementations, the user activity is walking or running, and the measured gravitational force indicates that the user is walking or running. In some implementations, the user activity is waiting in a line, and the measured gravitational force indicates that the user is waiting in a line.
In some implementations, the one or more other conditions include the user arriving at a known location, the outputting includes outputting the second suggestion, and the second suggestion relates to the arrival at the known location. In some implementations, (i) the first suggestion is a word and the second suggestion is a phrase or a sentence or (ii) the first suggestion is the word or the phrase and the second suggestion is the sentence.
In some implementations, the contextual information includes previous electronic messages associated with the user. In some implementations, the contextual information includes a date/time stamp for a particular previous electronic message comprising a question to the user.
In some implementations, the contextual information includes calendar information for the user. In some implementations, the calendar information includes at least one of (i) a type of an activity the user is currently engaged in and (ii) at least one of a start time and an end time of the activity that the user is currently engaged in.
In some implementations, the computing device is a client computing device associated with the user. In some implementations, the computing device is a server computing device that is distinct from a client computing device associated with the user.
According to another aspect of the present disclosure, a computing device is presented. The computing device can include a non-transitory computer-readable medium having a set of instructions stored thereon and one or more processors configured to execute the set of instructions, which causes the one or more processors to perform operations. The operations can include detecting an initiation of composing an electronic message by a user; obtaining contextual information for the electronic message from a source external to a text of the electronic message; obtaining a first suggestion for the text of the electronic message based on the contextual information; detecting an operating condition indicative of a user activity during which the user is likely to experience difficulty in typing; in response to detecting the operating condition, obtaining a second suggestion for the electronic message based on the contextual information, the second suggestion being more detailed than the first suggestion; and outputting one of the first and second suggestions depending on one or more other conditions.
In some implementations, the detecting of the operating condition is based on a measured gravitational force. In some implementations, the user activity is walking or running, and the measured gravitational force indicates that the user is walking or running. In some implementations, the user activity is waiting in a line, and the measured gravitational force indicates that the user is waiting in a line.
In some implementations, the one or more other conditions include the user arriving at a known location, the outputting includes outputting the second suggestion, and the second suggestion relates to the arrival at the known location. In some implementations, (i) the first suggestion is a word and the second suggestion is a phrase or a sentence or (ii) the first suggestion is the word or the phrase and the second suggestion is the sentence.
In some implementations, the contextual information includes previous electronic messages associated with the user. In some implementations, the contextual information includes a date/time stamp for a particular previous electronic message comprising a question to the user.
In some implementations, the contextual information includes calendar information for the user. In some implementations, the calendar information includes at least one of (i) a type of an activity the user is currently engaged in and (ii) at least one of a start time and an end time of the activity that the user is currently engaged in.
In some implementations, the second suggestion is obtained in response to the user inputting a prefix of the electronic message. In some implementations, the computing device is a client computing device associated with the user. In some implementations, the computing device is a server computing device that is distinct from a client computing device associated with the user.
Further areas of applicability of the present disclosure will become apparent from the detailed description provided hereinafter. It should be understood that the detailed description and specific examples are intended for purposes of illustration only and are not intended to limit the scope of the disclosure.
The present disclosure will become more fully understood from the detailed description and the accompanying drawings, wherein:
As mentioned above, typical pre-generated suggestions for electronic messaging lack any specific context, such as a context of a user's current activity. Some systems may derive a context of the electronic message from one or more previous words, but this still fails to capture any context outside of those one or more words. Such systems also do not work when the user has yet to input any words. Accordingly, improved techniques for suggesting electronic messages based on user activity and other context are presented. These techniques can leverage other sources of information to determine context for the electronic message and thereby provide more detailed suggestions. Such sources of information can be referred to as “external sources” that indicate any context of a particular electronic message that is not derived from its text.
Non-limiting examples of these external sources include user calendar information, other electronic messages, past and/or present user location information, and accelerometer information. In some implementations, an operating condition can be detected that is indicative of a user activity during which the user is likely to experience difficulty typing at their client computing device. This operating condition could be detected, for example, when a gravitational force (“g-force”) being experienced by the client computing device exceeds a threshold (e.g., indicative of the user walking or running). When such an operating condition is detected, a more detailed suggestion could be provided based on the obtained contextual information. Instead of suggesting just a few words, such as “On my way,” a full sentence could be suggested, such as “I'm jogging down Main Street and should be there in 10 minutes.”
The techniques of the present disclosure could be at least partially implemented by an electronic messaging application executing at a client computing device. This electronic messaging application could be, for example only, a standalone electronic messaging application. Non-limiting example user interfaces are discussed herein and illustrated in
The present disclosure is directed to, inter alia, the technical problem of interpreting and classifying user input in a messaging application. More specifically, the present disclosure is directed to techniques for providing appropriate suggested messages in a messaging application based on the external context of the user and her/his associated client computing device. The techniques utilize various hardware components and other forms of previously input information to analyze the external context of the messaging application. Based on the external context, and when the user initiates the composition of a message, the techniques provide for determining and outputting one or more fully formed output suggestions that are appropriate for the external context of the message (e.g., the context of the message independent of its previously input text).
Referring now to
Referring now to
The term “processor” as used herein can refer to both a single processor and a plurality of processors operating in a parallel or distributed architecture. In one implementation, the memory 212 may store a set of instructions that, when executed by the processor 216, causes the computing device 200 to perform at least a portion of the techniques of the present disclosure. The computing device 200 can also include other systems, such as an accelerometer 220 and a global positioning satellite (GPS) receiver 224. The accelerometer 220 can be any suitable accelerometer or system of accelerometers configured to measure a gravitational force (“g-force”) experienced by the computing device 200. The GPS receiver 224 can communicate with a GPS system (not shown) to obtain global position data (e.g., location information) for the computing device 200. The use of the accelerometer 220 and GPS receiver 224 will be discussed in more detail below.
According to one aspect of the present disclosure, the computing system 100 can determine suggestion(s) for a text of an electronic message being composed by the first user 116 of the client computing device 104. These techniques can be initiated, for example, upon the initiation of composing the electronic message (e.g., selecting or clicking a text entry box or an input field of a messaging application). After the composing of the electronic message is initiated, the computing system 100 can determine context information from a source external to the text of the electronic message. This “external context” refers to any information other than the actual text of the electronic message (e.g., previous characters or words). One example of this external context information is previous messages associated with the first user 116. For example, if the first user 116 is addressing the electronic message to the second user 124, previous electronic messages between the two could be analyzed to determine the external context information.
In one implementation, the computing system 100 can obtain a conversation topic from the previous electronic messages. For example, the conversation topic could be meeting for dinner. The computing system 100 can identify dates/times across the whole conversation and resolve the dates and times relative to the time the message was sent. The computing system 100 can also identify open or unanswered questions from the previous electronic messages. For example, one of the more recent electronic messages could be “when are you coming?” or “when will you arrive?” The question can be identified as open if a response has not yet been provided. An estimated time of arrival for the first user 116 could be estimated using external context information, such as the type of activity and/or location information, and then utilized in responding to such a question, e.g., “on the bus, be there in 15 minutes.” The suggestions could also be generated and/or filtered as the first user 116 enters text. For example, if the first user 116 types “I'm” in connection with a previous message “when will you arrive,” then the suggestion could be “I'm on my way.”
As mentioned above, the accelerometer 220 can be configured to measure a g-force being experienced by the client computing device 104. This information can then be used to detect a certain type of motion. Once example type of motion is the first user 116 is walking or running. While walking or running, the first user 116 may experience difficulty in typing at the client computing device 104 and thus the suggestions may be more detailed, e.g., a full sentence instead of a word or phrase. Other types of motion can be initiating or being in the process of a particular type of transportation, such as getting into a car, boarding public transportation (e.g., a bus or train), getting onto a bicycle, or entering an elevator. For example, it may be difficult for a user to type when they need to hold an object, such as a pole while standing on a train or subway car. Location information and movement information captured by the GPS receiver 224 can also be used to detect the type of motion. Other types of motion can include waiting in a line or queue, which can involve periodic slow movements separated by static or non-moving periods, but it still may be difficult for the users to type a full message due to the repeated interruption and/or fear of causing a backup of others behind them in line.
As mentioned above, location information for the client computing device 104 can be utilized to provide more detailed suggestions, such as in response to a change in user location. Known locations (e.g., previously/frequently visited locations), such as a workplace or home of the first user 116, could be leveraged to provide a related message, such as “just got home” or “just got to work.” There could be even more detailed location information, such as indoor location information or indoor positioning, and thus a suggestion could be as detailed as “just arrived at my desk.” Calendar information could be similarly leveraged to determine when the first user 116 has a scheduled event and, when the client computing device 104 arrives at the corresponding location, the suggested message could be “just arrived.” Suggestions could also be provided based on the known current activity, which could again be determined from user history (previously/frequently visited locations), location information, and/or calendar/schedule information. For example, the location could be a grocery store, and the suggested message could be “just getting some groceries from the store.” Their current activity could also be one where they would not want to continue communicating, such as while they are in a work meeting, which could be determined from their calendar information (e.g., activity description, start time, and end time). In this type of scenario, a suggested message could be “I'll contact you later.”
The exact suggested messages can also be specific to the first user 116. More specifically, a user-specific language model can be generated and trained based on the activity by the first user 116 (e.g., by using machine learning techniques). These machine learning techniques can identify or recognize patterns associated with the user, such as often used words and phrases. These patterns can be leveraged to create the user-specific language model, which can then predict a most-likely message to suggest (e.g., a possible suggest message having a highest respective probability score). The term language model as used herein can refer to a probability distribution over a sequence of text (characters, word, phrases, etc.) that is derived from (or “trained” based on) training data. In some implementations, a language model can assign a probability to a piece of unknown text (character, word, phrase, etc.) based on external context information (previous messages, location information, etc.) and a corpus of training data upon which the language model is trained. This training data, for example, can be related to the first user 116.
By utilizing this user-specific language model, the suggestions should mimic the style/tone of the first user 116, which may make the suggested electronic messages more applicable for the first user 116 because the receiving user (e.g., the second user 124) may be more likely to believe that the first user 116 actually typed the message as opposed to just selecting one of a few pre-generated messages. Cross-user machine learning could also be performed, and a global or non-user-specific language model could be generated and trained (e.g., based on user feedback indicative of which suggestions are high quality suggestions). Learning could also be performed to associate certain types of activities with average statistics (e.g., grocery shopping takes an average user approximately 30 minutes), which could then be leveraged to provide even better suggestions (e.g., “I'm at the grocery store. I'll be done in 20 minutes!”). There can also be various sub-categories for certain activities/statistics. For example only, grocery shopping may take an average user approximately 20 minutes during the middle of a work day, such as at 11:00 am, whereas it may take an average user approximately 40 minutes in the evening of the work day, such as at 6:00 pm.
Referring now to
At 316, the computing system 100 can detect an operating condition indicative of a user activity during which the first user 116 is likely to experience difficulty in typing at the client computing device 104. When the operating condition is detected, the method 300 can proceed to 320. Otherwise, the method 300 can proceed to 324. At 320, the computing system 100 can obtain a second suggestion for the electronic message based on the contextual information, the second suggestion being more detailed than the first suggestion. At 324, the computing system 100 can output (e.g., from the server computing device 108 to the client computing device 104) either the first or second suggestion, depending on the decision at 316. The first user 116 could then utilize the provided suggestion in composing the electronic message, which can then be sent to the other computing device 120 via the network 112.
Referring now to
Some examples of the context that can be derived from these electronic messages are the type of activity (eating), the location (“Joe's Diner”) and the confirmation (“Sure!”). These electronic messages are also proximate in time, as opposed to the first user 116 of the client computing device 104 responding “Sure!” at 6:00 pm, which would not be appropriate for still meeting for lunch. As shown, the other user (e.g., second user 124) has also sent another message at 12:19 pm (“Where are you? ETA?”). This provides further context for the first suggestion “On my way.” That is, the other user has inquired where the first user 116 of the client computing device 104 is and when they will arrive. As previously discussed herein, the first suggestion 412 may be inadequate in certain scenarios. As shown in
As shown in
One or more systems and methods discussed herein do not require collection or usage of user personal information. In situations in which certain implementations discussed herein may collect or use personal information about users (e.g., user data, information about a user's social network, user's location and time, user's biometric information, user's activities and demographic information), users are provided with one or more opportunities to control whether the personal information is collected, whether the personal information is stored, whether the personal information is used, and how the information is collected about the user, stored and used. That is, the systems and methods discussed herein collect, store and/or use user personal information only upon receiving explicit authorization from the relevant users to do so. In addition, certain data may be treated in one or more ways before it is stored or used so that personally identifiable information is removed. As one example, a user's identity may be treated so that no personally identifiable information can be determined. As another example, a user's geographic location may be generalized to a larger region so that the user's particular location cannot be determined.
Example embodiments are provided so that this disclosure will be thorough, and will fully convey the scope to those who are skilled in the art. Numerous specific details are set forth such as examples of specific components, devices, and methods, to provide a thorough understanding of embodiments of the present disclosure. It will be apparent to those skilled in the art that specific details need not be employed, that example embodiments may be embodied in many different forms and that neither should be construed to limit the scope of the disclosure. In some example embodiments, well-known procedures, well-known device structures, and well-known technologies are not described in detail.
The terminology used herein is for the purpose of describing particular example embodiments only and is not intended to be limiting. As used herein, the singular forms “a,” “an,” and “the” may be intended to include the plural forms as well, unless the context clearly indicates otherwise. The term “and/or” includes any and all combinations of one or more of the associated listed items. The terms “comprises,” “comprising,” “including,” and “having,” are inclusive and therefore specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof. The method steps, processes, and operations described herein are not to be construed as necessarily requiring their performance in the particular order discussed or illustrated, unless specifically identified as an order of performance. It is also to be understood that additional or alternative steps may be employed.
Although the terms first, second, third, etc. may be used herein to describe various elements, components, regions, layers and/or sections, these elements, components, regions, layers and/or sections should not be limited by these terms. These terms may be only used to distinguish one element, component, region, layer or section from another region, layer or section. Terms such as “first,” “second,” and other numerical terms when used herein do not imply a sequence or order unless clearly indicated by the context. Thus, a first element, component, region, layer or section discussed below could be termed a second element, component, region, layer or section without departing from the teachings of the example embodiments.
As used herein, the term module may refer to, be part of, or include: an Application Specific Integrated Circuit (ASIC); an electronic circuit; a combinational logic circuit; a field programmable gate array (FPGA); a processor or a distributed network of processors (shared, dedicated, or grouped) and storage in networked clusters or datacenters that executes code or a process; other suitable components that provide the described functionality; or a combination of some or all of the above, such as in a system-on-chip. The term module may also include memory (shared, dedicated, or grouped) that stores code executed by the one or more processors.
The term code, as used above, may include software, firmware, byte-code and/or microcode, and may refer to programs, routines, functions, classes, and/or objects. The term shared, as used above, means that some or all code from multiple modules may be executed using a single (shared) processor. In addition, some or all code from multiple modules may be stored by a single (shared) memory. The term group, as used above, means that some or all code from a single module may be executed using a group of processors. In addition, some or all code from a single module may be stored using a group of memories.
The techniques described herein may be implemented by one or more computer programs executed by one or more processors. The computer programs include processor-executable instructions that are stored on a non-transitory tangible computer readable medium. The computer programs may also include stored data. Non-limiting examples of the non-transitory tangible computer readable medium are nonvolatile memory, magnetic storage, and optical storage.
Some portions of the above description present the techniques described herein in terms of algorithms and symbolic representations of operations on information. These algorithmic descriptions and representations are the means used by those skilled in the data processing arts to most effectively convey the substance of their work to others skilled in the art. These operations, while described functionally or logically, are understood to be implemented by computer programs. Furthermore, it has also proven convenient at times to refer to these arrangements of operations as modules or by functional names, without loss of generality.
Unless specifically stated otherwise as apparent from the above discussion, it is appreciated that throughout the description, discussions utilizing terms such as “processing” or “computing” or “calculating” or “determining” or “displaying” or the like, refer to the action and processes of a computer system, or similar electronic computing device, that manipulates and transforms data represented as physical (electronic) quantities within the computer system memories or registers or other such information storage, transmission or display devices.
Certain aspects of the described techniques include process steps and instructions described herein in the form of an algorithm. It should be noted that the described process steps and instructions could be embodied in software, firmware or hardware, and when embodied in software, could be downloaded to reside on and be operated from different platforms used by real time network operating systems.
The present disclosure also relates to an apparatus for performing the operations herein. This apparatus may be specially constructed for the required purposes, or it may comprise a general-purpose computer selectively activated or reconfigured by a computer program stored on a computer readable medium that can be accessed by the computer. Such a computer program may be stored in a tangible computer readable storage medium, such as, but is not limited to, any type of disk including floppy disks, optical disks, CD-ROMs, magnetic-optical disks, read-only memories (ROMs), random access memories (RAMs), EPROMs, EEPROMs, magnetic or optical cards, application specific integrated circuits (ASICs), or any type of media suitable for storing electronic instructions, and each coupled to a computer system bus. Furthermore, the computers referred to in the specification may include a single processor or may be architectures employing multiple processor designs for increased computing capability.
The algorithms and operations presented herein are not inherently related to any particular computer or other apparatus. Various general-purpose systems may also be used with programs in accordance with the teachings herein, or it may prove convenient to construct more specialized apparatuses to perform the required method steps. The required structure for a variety of these systems will be apparent to those of skill in the art, along with equivalent variations. In addition, the present disclosure is not described with reference to any particular programming language. It is appreciated that a variety of programming languages may be used to implement the teachings of the present disclosure as described herein, and any references to specific languages are provided for disclosure of enablement and best mode of the present invention.
The present disclosure is well suited to a wide variety of computer network systems over numerous topologies. Within this field, the configuration and management of large networks comprise storage devices and computers that are communicatively coupled to dissimilar computers and storage devices over a network, such as the Internet.
The foregoing description of the embodiments has been provided for purposes of illustration and description. It is not intended to be exhaustive or to limit the disclosure. Individual elements or features of a particular embodiment are generally not limited to that particular embodiment, but, where applicable, are interchangeable and can be used in a selected embodiment, even if not specifically shown or described. The same may also be varied in many ways. Such variations are not to be regarded as a departure from the disclosure, and all such modifications are intended to be included within the scope of the disclosure.
Number | Name | Date | Kind |
---|---|---|---|
6092102 | Wagner | Jul 2000 | A |
7603413 | Herold et al. | Oct 2009 | B1 |
8391618 | Chuang et al. | Mar 2013 | B1 |
8423577 | Lee et al. | Apr 2013 | B1 |
8515958 | Knight | Aug 2013 | B2 |
8554701 | Dillard et al. | Oct 2013 | B1 |
8589407 | Bhatia | Nov 2013 | B2 |
8650210 | Cheng | Feb 2014 | B1 |
8688698 | Black et al. | Apr 2014 | B1 |
8700480 | Fox et al. | Apr 2014 | B1 |
8825474 | Zhai et al. | Sep 2014 | B1 |
8938669 | Cohen | Jan 2015 | B1 |
8996639 | Faaborg et al. | Mar 2015 | B1 |
9191786 | Davis | Nov 2015 | B2 |
9213941 | Petersen | Dec 2015 | B2 |
9230241 | Singh et al. | Jan 2016 | B1 |
9262517 | Feng et al. | Feb 2016 | B2 |
9560152 | Jamdar et al. | Jan 2017 | B1 |
9674120 | Davis | Jun 2017 | B2 |
9715496 | Sapoznik et al. | Jul 2017 | B1 |
9805371 | Sapoznik et al. | Oct 2017 | B1 |
9807037 | Sapoznik et al. | Oct 2017 | B1 |
9817813 | Faizakof et al. | Nov 2017 | B2 |
10146748 | Barndollar et al. | Dec 2018 | B1 |
20030105589 | Liu et al. | Jun 2003 | A1 |
20060029106 | Ott et al. | Feb 2006 | A1 |
20060156209 | Matsuura et al. | Jul 2006 | A1 |
20070030364 | Obrador et al. | Feb 2007 | A1 |
20070094217 | Ronnewinkel | Apr 2007 | A1 |
20070162942 | Hamynen et al. | Jul 2007 | A1 |
20070244980 | Baker et al. | Oct 2007 | A1 |
20090076795 | Bangalore et al. | Mar 2009 | A1 |
20090119584 | Herbst | May 2009 | A1 |
20090282114 | Feng et al. | Nov 2009 | A1 |
20100228590 | Muller | Sep 2010 | A1 |
20100260426 | Huang et al. | Oct 2010 | A1 |
20110074685 | Causey et al. | Mar 2011 | A1 |
20110107223 | Tilton et al. | May 2011 | A1 |
20110164163 | Bilbrey et al. | Jul 2011 | A1 |
20110252108 | Morris et al. | Oct 2011 | A1 |
20120033876 | Momeyer et al. | Feb 2012 | A1 |
20120041941 | King et al. | Feb 2012 | A1 |
20120041973 | Kim et al. | Feb 2012 | A1 |
20120042036 | Lau et al. | Feb 2012 | A1 |
20120131520 | Tang et al. | May 2012 | A1 |
20120179717 | Kennedy et al. | Jul 2012 | A1 |
20120224743 | Rodriguez et al. | Sep 2012 | A1 |
20120245944 | Gruber et al. | Sep 2012 | A1 |
20130050507 | Syed et al. | Feb 2013 | A1 |
20130061148 | Das et al. | Mar 2013 | A1 |
20130073366 | Heath | Mar 2013 | A1 |
20130260727 | Knudson et al. | Oct 2013 | A1 |
20130262574 | Cohen | Oct 2013 | A1 |
20130346235 | Lam | Dec 2013 | A1 |
20140004889 | Davis | Jan 2014 | A1 |
20140035846 | Lee et al. | Feb 2014 | A1 |
20140047413 | Sheive et al. | Feb 2014 | A1 |
20140067371 | Liensberger | Mar 2014 | A1 |
20140088954 | Shirzadi et al. | Mar 2014 | A1 |
20140108562 | Panzer | Apr 2014 | A1 |
20140163954 | Joshi et al. | Jun 2014 | A1 |
20140164506 | Tesch et al. | Jun 2014 | A1 |
20140171133 | Stuttle | Jun 2014 | A1 |
20140189027 | Zhang et al. | Jul 2014 | A1 |
20140189538 | Martens et al. | Jul 2014 | A1 |
20140201675 | Joo et al. | Jul 2014 | A1 |
20140228009 | Chen | Aug 2014 | A1 |
20140237057 | Khodorenko | Aug 2014 | A1 |
20140317030 | Shen et al. | Oct 2014 | A1 |
20140337438 | Govande et al. | Nov 2014 | A1 |
20140372349 | Driscoll | Dec 2014 | A1 |
20150006143 | Skiba et al. | Jan 2015 | A1 |
20150032724 | Thirugnanasundaram et al. | Jan 2015 | A1 |
20150058720 | Smadja et al. | Feb 2015 | A1 |
20150095855 | Bai et al. | Apr 2015 | A1 |
20150100537 | Grieves et al. | Apr 2015 | A1 |
20150178371 | Seth et al. | Jun 2015 | A1 |
20150178388 | Winnemoeller et al. | Jun 2015 | A1 |
20150207765 | Brantingham et al. | Jul 2015 | A1 |
20150227797 | Ko et al. | Aug 2015 | A1 |
20150248411 | Krinker et al. | Sep 2015 | A1 |
20150250936 | Thomas et al. | Sep 2015 | A1 |
20150286371 | Degani | Oct 2015 | A1 |
20150302301 | Petersen | Oct 2015 | A1 |
20160037311 | Cho | Feb 2016 | A1 |
20160042252 | Sawhney et al. | Feb 2016 | A1 |
20160043974 | Purcell et al. | Feb 2016 | A1 |
20160072737 | Forster | Mar 2016 | A1 |
20160140447 | Cohen et al. | May 2016 | A1 |
20160140477 | Karanam et al. | May 2016 | A1 |
20160162791 | Petersen | Jun 2016 | A1 |
20160179816 | Glover | Jun 2016 | A1 |
20160210279 | Kim et al. | Jul 2016 | A1 |
20160224524 | Kay | Aug 2016 | A1 |
20160226804 | Hampson et al. | Aug 2016 | A1 |
20160234553 | Hampson et al. | Aug 2016 | A1 |
20160283454 | Leydon et al. | Sep 2016 | A1 |
20160342895 | Gao et al. | Nov 2016 | A1 |
20160350304 | Aggarwal et al. | Dec 2016 | A1 |
20160378080 | Uppala et al. | Dec 2016 | A1 |
20170075878 | Jon et al. | Mar 2017 | A1 |
20170093769 | Lind et al. | Mar 2017 | A1 |
20170098122 | el Kaliouby et al. | Apr 2017 | A1 |
20170134316 | Cohen et al. | May 2017 | A1 |
20170142046 | Mahmoud et al. | May 2017 | A1 |
20170149703 | Willett et al. | May 2017 | A1 |
20170153792 | Kapoor et al. | Jun 2017 | A1 |
20170171117 | Carr et al. | Jun 2017 | A1 |
20170180276 | Gershony et al. | Jun 2017 | A1 |
20170180294 | Milligan et al. | Jun 2017 | A1 |
20170187654 | Lee | Jun 2017 | A1 |
20170250930 | Ben-Itzhak | Aug 2017 | A1 |
20170250935 | Rosenberg | Aug 2017 | A1 |
20170250936 | Rosenberg et al. | Aug 2017 | A1 |
20170293834 | Raison et al. | Oct 2017 | A1 |
20170308589 | Liu et al. | Oct 2017 | A1 |
20170324868 | Tamblyn et al. | Nov 2017 | A1 |
20170344224 | Kay et al. | Nov 2017 | A1 |
20170357442 | Peterson et al. | Dec 2017 | A1 |
20180004397 | Mazzocchi | Jan 2018 | A1 |
20180005272 | Todasco et al. | Jan 2018 | A1 |
20180012231 | Sapoznik et al. | Jan 2018 | A1 |
20180013699 | Sapoznik et al. | Jan 2018 | A1 |
20180060705 | Mahmoud et al. | Mar 2018 | A1 |
20180083898 | Pham | Mar 2018 | A1 |
20180083901 | McGregor et al. | Mar 2018 | A1 |
20180090135 | Schlesinger et al. | Mar 2018 | A1 |
20180137097 | Lim et al. | May 2018 | A1 |
20180196854 | Burks | Jul 2018 | A1 |
20180210874 | Fuxman et al. | Jul 2018 | A1 |
20180293601 | Glazier | Oct 2018 | A1 |
20180309706 | Kim et al. | Oct 2018 | A1 |
20180316637 | Desjardins | Nov 2018 | A1 |
20180336226 | Anorga et al. | Nov 2018 | A1 |
20180336415 | Anorga et al. | Nov 2018 | A1 |
20180373683 | Hullette et al. | Dec 2018 | A1 |
Number | Date | Country |
---|---|---|
1475908 | Feb 2004 | CN |
102222079 | Oct 2011 | CN |
102395966 | Mar 2012 | CN |
102467574 | May 2012 | CN |
103548025 | Jan 2014 | CN |
1376392 | Jan 2004 | EP |
1394713 | Mar 2004 | EP |
2523436 | Nov 2012 | EP |
2560104 | Feb 2013 | EP |
2688014 | Jan 2014 | EP |
2703980 | Mar 2014 | EP |
3091445 | Nov 2016 | EP |
20110003462 | Jan 2011 | KR |
20130008036 | Jan 2013 | KR |
20130061387 | Jun 2013 | KR |
2004104758 | Dec 2004 | WO |
2011002989 | Jan 2011 | WO |
2016130788 | Aug 2016 | WO |
2018089109 | May 2018 | WO |
Entry |
---|
PCT International Search Report and Written Opinion dated Oct. 11, 2017 for PCT Application No. PCT/US2017/046858, 10 pages. |
USPTO, Notice of Allowance for U.S. Appl. No. 15/624,638, dated Feb. 28, 2019, 21 Pages. |
EPO, Written Opinion of the International Preliminary Examination Authority for International Patent Application No. PCT/US2017/057044, dated Dec. 20, 2018, 8 pages. |
EPO, Written Opinion for International Patent Application No. PCT/US2017/057044, dated Jan. 18, 2018, 8 pages. |
EPO, Written Opinion of the International Preliminary Examining Authority for International Patent Application No. PCT/US2017/052349, dated Aug. 6, 2018, 9 pages. |
EPO, Written Opinion for International Patent Application No. PCT/US2016/068083, dated Mar. 9, 2017, 9 pages. |
Fuxman, ““Aw, so cute!”: Allo helps you respond to shared photos”, Google Research Blog, https://research.googleblog.com/2016/05/aw-so-cute-allo-helps-you-respond-to.html, May 18, 2016, 6 pages. |
International Bureau of WIPO, International Preliminary Report on Patentability for International Patent Application No. PCT/US2016/068083, dated Jul. 5, 2018, 9 pages. |
Kannan, et al., “Smart Reply: Automated Response Suggestions for Email”, Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD '16, ACM Press, New York, New York, Aug. 13, 2016, pp. 955-965. |
Khandelwal, “Hey Allo! Meet Google's AI-powered Smart Messaging App”, The Hacker News, http://web.archive.org/web/20160522155700/https://thehackernews.com/2016/05/google-allo-messenger.html, May 19, 2016, 3 pages. |
KIPO, Preliminary Rejection for Korean Patent Application No. 10-2018-7013953, dated Oct. 29, 2018, 5 pages. |
Lee, et al., “Supporting multi-user, multi-applet workspaces in CBE”, Proceedings of the 1996 ACM conference on Computer supported cooperative work, ACM, Nov. 16, 1996, 10 pages. |
Microsoft Corporation, “Windows Messenger for Windows XP”, Retrieved from Internet: http://web.archive.org/web/20030606220012/messenger.msn.com/support/features.asp?client=0 on Sep. 22, 2005, Jun. 6, 2003, 3 pages. |
EPO, International Search Report and Written Opinion for International Application No. PCT/US2018/021028, dated Jun. 15, 2018, 11 pages. |
Pinterest, “Pinterest Lens”, www.help.pinterest.com/en/articles/pinterest-lens, 2 pages. |
Russell, “Google Allo is the Hangouts Killer We've Been Waiting For”, Retrieved from the Internet: http://web.archive.org/web/20160519115534/https://www.technobuffalo.com/2016/05/18/google-allo-hangouts-replacement/, May 18, 2016, 3 pages. |
USPTO, Notice of Allowance for U.S. Appl. No. 15/709,418, dated Mar. 1, 2018, 11 pages. |
USPTO, Non-final Office Action for U.S. Appl. No. 15/386,162, dated Nov. 27, 2018, 12 pages. |
USPTO, Notice of Allowance for U.S. Appl. No. 14/618,962, dated Nov. 8, 2016, 14 pages. |
USPTO, Non-final Office Action for U.S. Appl. No. 15/709,418, dated Nov. 21, 2017, 15 pages. |
USPTO, First Action Interview, Office Action Summary for U.S. Appl. No. 16/003,661, dated Dec. 14, 2018, 16 pages. |
USPTO, Notice of Allowance for U.S. Appl. No. 15/428,821, dated Jan. 10, 2018, 20 pages. |
USPTO, Non-final Office Action for U.S. Appl. No. 14/618,962, dated Feb. 26, 2016, 25 pages. |
USPTO, Notice of Allowance for U.S. Appl. No. 15/415,506, dated Jul. 23, 2018, 25 pages. |
USPTO, Non-final Office Action for U.S. Appl. No. 15/428,821, dated May 18, 2017, 30 pages. |
USPTO, First Action Interview, Pre-Interview Communication for U.S. Appl. No. 15/624,637, dated Oct. 19, 2018, 4 pages. |
USPTO, First Action Interview, Office Action Summary for U.S. Appl. No. 15/350,040, dated Oct. 30, 2018, 4 pages. |
USPTO, Non-final Office Action for U.S. Appl. No. 15/946,342, dated Jul. 26, 2018, 40 pages. |
USPTO, First Action Interview, Pre-Interview Communication for U.S. Appl. No. 15/415,506, dated Apr. 5, 2018, 5 pages. |
USPTO, First Action Interview, Office Action Summary for U.S. Appl. No. 15/624,637, dated Jan. 25, 2019, 5 pages. |
USPTO, First Action Interview, Pre-Interview Communication for U.S. Appl. No. 15/350,040, dated Jul. 16, 2018, 5 pages. |
USPTO, First Action Interview, Pre-Interview Communication for U.S. Appl. No. 15/386,760, dated Nov. 6, 2018, 5 pages. |
USPTO, First Action Interview, Pre-Interview Communication for U.S. Appl. No. 16/003,661, dated Aug. 29, 2018, 6 pages. |
USPTO, First Action Interview, Office Action Summary for U.S. Appl. No. 15/386,760, dated Jan. 30, 2019, 8 pages. |
Vinyals, et al., “Show and Tell: A Neural Image Caption Generator”, arXiv:1411.4555v2 [cs.CV], Apr. 20, 2015, pp. 1-9. |
Yeh, et al., “Searching the web with mobile images for location recognition”, Proceedings of the 2004 IEEE Computer Society Conference on Pattern Recognition, vol. 2, Jun.-Jul. 2004, pp. 1-6. |
Australia IP, Notice of Acceptance for Australian Patent Application No. 2015214298, dated Apr. 20, 2018, 3 pages. |
Australia IP, Examination Report No. 1 for Australian Patent Application No. 2015214298, dated Apr. 24, 2017, 3 pages. |
Australia IP, Examination Report No. 2 for Australian Patent Application No. 2015214298, dated Nov. 2, 2017, 3 pages. |
USPTO, International Search Report and Written Opinion for International Patent Application No. PCT/US2015/014414, dated May 11, 2015, 8 pages. |
Blippar, “Computer Vision API”, www.web.blippar.com/computer-vision-api, 2018, 4 pages. |
Chen, et al., “Bezel Copy: An Efficient Cross-0Application Copy-Paste Technique for Touchscreen Smartphones.”, Advanced Visual Interfaces, ACM, New York, New York, May 27, 2014, pp. 185-192. |
CNIPA, First Office Action for Chinese Patent Application No. 201580016692.1, dated Nov. 2, 2018, 7 pages. |
EPO, International Search Report and Written Opinion for International Patent Application No. PCT/US2016/068083, dated Mar. 9, 2017, 13 pages. |
EPO, International Search Report for International Patent Application No. PCT/US2017/052713, dated Dec. 5, 2017, 4 Pages. |
EPO, International Search Report for International Patent Application No. PCT/US2016/068083, dated Mar. 9, 2017, 4 pages. |
EPO, International Search Report for International Patent Application No. PCT/US2018/022501, dated May 14, 2018, 4 pages. |
EPO, International Search Report for International Patent Application No. PCT/US2017/052349, dated Dec. 13, 2017, 5 pages. |
EPO, International Search Report for International Patent Application No. PCT/US2017/057044, dated Jan. 18, 2018, 5 pages. |
EPO, Written Opinion for International Patent Application No. PCT/US2017/052713, dated Dec. 5, 2017, 6 pages. |
EPO, International Search Report for International Patent Application No. PCT/US2018/022503, dated Aug. 16, 2018, 6 pages. |
EPO, Written Opinion for International Patent Application No. PCT/US2017/052349, dated Dec. 13, 2017, 6 pages. |
EPO, Written Opinion for International Patent Application No. PCT/US2018/022501, dated May 14, 2018, 6 pages. |
EPO, Written Opinion of the International Preliminary Examining Authority for International Patent Application No. PCT/US2017/052713, dated Oct. 15, 2018, 6 pages. |
EPO, Extended European Search Report for European Patent Application No. 15746410.8, dated Sep. 5, 2017, 7 pages. |
EPO, Written Opinion for International Patent Application No. PCT/US2018/022503, dated Aug. 16, 2018, 8 pages. |
EPO, Communication Pursuant to Article 94(3) EPC for European Patent Application No. 16825663.4, Apr. 16, 2019, 5 pages. |
EPO, Communication Pursuant to Article 94(3) EPC for European Patent Application No. 1682566.7, Apr. 23, 2019, 6 pages. |
KIPO, Notice of Final Rejection for Korean Patent Application No. 10-2018-7013953, dated Jun. 13, 2019, 4 pages. |
KIPO, Notice of Preliminary Rejection for Korean Patent Application No. 10-2018-7019756, dated May 13, 2019, 9 pages. |
KIPO, Notice of Preliminary Rejection for Korean Patent Application No. 10-2019-7011687, dated May 7, 2019, 3 pages. |
KIPO, Notice of Final Rejection for Korean Patent Application No. 10-2018-7013953, dated May 8, 2019, 4 pages. |
USPTO, Notice of Allowance for U.S. Appl. No. 15/624,637, dated Apr. 19, 2019, 6 pages. |
USPTO, Notice of Allowance for U.S. Appl. No. 15/350,040, dated Apr. 24, 2019, 16 pages. |
USPTO, Final Office Action for U.S. Appl. No. 15/386,162, dated Jun. 5, 2019, 13 pages. |
USPTO, Notice of Allowance for U.S. Appl. No. 16/003,661, dated May 1, 2019, 11 pages. |
USPTO, Non-final Office Action for U.S. Appl. No. 15/709,423, dated May 2, 2019, 21 Pages. |
USPTO, Final Office Action for U.S. Appl. No. 15/386,760, dated May 30, 2019, 13 pages. |
Number | Date | Country | |
---|---|---|---|
20180052909 A1 | Feb 2018 | US |