Patent Application
20200301566
Today, emojis have become common place in electronic communications both in casual and in professional environments. Emojis are small digital images, pictures, or icons often used in emails, instant messages, or text messages to express or represent ideas, emotions, objects, or meanings. For example, a smiling face emoji can be used to express happiness. On the other hand, a frowny face emoji can be used to express sadness. Other emojis can also be used to express or represent additional facial expressions, objects, places, types of weather, animals, etc.
This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter.
To facilitate efficient composition of emails, instant messages, text messages, or other suitable types of electronic messages, computing systems can provide certain applications to facilitate selection of emojis by users via pages. Such applications are generally referred to as emoji pickers. In certain emoji pickers, different emojis are grouped for display into different tabs, sections, panes, windows, or other suitable types of user interface elements (collectively referred to herein as “pages”). For instance, a facial expression page can include emojis of various facial expressions. A food item page can include emojis of ice cream, pizza, soda, pasta, and other food items. When composing an electronic message, a user can browse through multiple pages to locate, select, and insert emojis into electronic messages. Certain emoji pickers can also track emojis previously selected and/or inserted by users into electronic messages and group these emojis in a frequently used page. Certain emoji pickers can also provide a search field that allows a user to query all the pages for emojis using keywords (e.g., “ice cream”) or other suitable search criteria.
Grouping emojis in the foregoing manners, however, may not allow users to efficiently locate emojis when composing electronic messages. For example, during November and December, a user may browse for emojis often associated with the Thanksgiving and Christmas holiday season, such as emojis for turkeys, Santa Claus, reindeer, sleigh, snowman, etc. These emojis, however, may be located in different pages based on corresponding categories. For instance, emojis of turkeys may be located in an animal or food page while emojis of Santa Claus may be located in a person page. As such, a user may spend a long time browsing through multiple emoji pages in order to locate all desired emojis for composition of a holiday electronic message. In another example, when a user first starts using an emoji picker, the frequently used page would be empty. Only after a period of usage, the frequently used page can be populated with previously selected emojis after browsing through multiple emoji pages. Facilitating a user's browsing through multiple emoji pages can place a heavy burden on a computing load and/or network bandwidth consumption of the computing device.
Several embodiments of the disclosed technology can address at least some of the foregoing drawbacks by implementing an emoji picker having one or more pages (referred to herein as “context pages”) configured to provide emojis suggested based on environmental context data. In certain embodiments, the environmental context data can include data representing one or more of a date, a time, a season, or a location, and/or other suitable environment parameters related to an environment in which a user or a corresponding computing device executing the emoji picker is located. For example, the emoji picker can be configured to provide a context page that displays multiple emojis related to Thanksgiving and Christmas holiday season when the emoji picker determines that a current date is within a date range, e.g., from November 1 to December 31. In another example, the emoji picker can also provide a context page that displays multiple emojis related to summer activities (e.g., surfing, sunbathing, etc.) when the emoji picker determines that a current date is from within another date range, such as June 15 to September 10.
In other examples, the emoji picker can also be configured to provide one or more context pages having selected emojis based on a location or a season at the location of the user or computing device in addition to or in lieu of a current date or time. For instance, when the emoji picker determines that a current date is within a date range from November 1 to December 31 but a current location is Australia, the emoji picker can be configured to provide a context page that includes emojis for summer activities (e.g., swimming) instead of winter activities (e.g., skiing) in addition to emojis for Santa Claus and Christmas trees. In another example, when the emoji picker determines that a current date is within a date range from January 10 to February 15 and a current location is China, the emoji picker can be configured to provide a context page that includes emojis for Chinese Spring Holiday (e.g., fire crackers). In further examples, the emoji picker can also be configured to filter emojis in existing and/or context pages based on a current location of the user or computing device in accordance with local traditions. For instance, emojis for beef may be removed from any of the pages when the current location is India.
Association of emojis with certain date, time, location, and/or season can be developed in various ways in accordance with aspects of the disclosed technology. In one embodiment, a software developer of the emoji picker can manually configure the association via rules, decision trees, and/or other suitable techniques. For instance, an example rule can include data that indicates that a Santa Claus emoji is associated with a date range (e.g., November 1 to December 31) and a particular location (e.g., North America) of the user or computing device. Such manually configured rules can also be updated periodically via software updates or other suitable techniques.
In other embodiments, such association can be generated and updated based on monitored emoji usage at a location in which the computing system is located during certain dates, times, seasons, or other time periods. For example, data representing used emojis (referred to below as “emoji usage data”) in text messages in a cellular network may be collected as anonymous data and/or with user consent. The emoji usage data can then be analyzed to determine usage patterns and/or correlations of used emojis and a date, a time, a location, and/or other suitable environmental parameters. As such, a model developer can be configured to develop an emoji model that can be used to predict a probability that an emoji is likely to be used based on a date/time, a date/time range, a location, and/or other suitable environmental parameters.
In some implementations, the emoji model can include data representing a set of weight values corresponding to various environment parameters and a decision tree derived via machine learning. For example, the model developer can be configured to identify the various weight values using a “neural network” or “artificial neural network” configured to “learn” or progressively improve performance of tasks by studying known examples. In certain implementations, a neural network can include multiple layers of objects generally refers to as “neurons” or “artificial neurons.” Each neuron can be configured to perform a function, such as a non-linear activation function, based on one or more inputs via corresponding connections. Artificial neurons and connections typically have a contribution value that adjusts as learning proceeds. The contribution value increases or decreases a strength of an input at a connection. Typically, artificial neurons are organized in layers. Different layers may perform different kinds of transformations on respective inputs. Signals typically travel from an input layer, to an output layer, possibly after traversing one or more intermediate layers. In some implementations, the model developer can also be configured to update the emoji model using additional emoji usage data. In other embodiments, the model developer can be configured to perform such model development and/or update based on administrator provided rules or via other suitable techniques.
In additional embodiments, the environmental context data can also include data representing user's activities collected from user's calendars, social network posting, etc. with suitable user consent. For instance, a user's calendar items can be monitored in a corporate environment to provide a context page of emojis selected based on activities corresponding to the monitored calendar items. In one example, a user can have a calendar item for a scheduled bowling night. In response, the emoji picker can be configured to provide a context page having emojis representing bowling alleys, bowling pins, etc. In another example, a user can have a calendar item for a vacation in Paris France. In response, the emoji picker can be configured to provide another context page having emojis of Eiffel tower, Arc de Triomphe, etc. In other examples, a user's social network postings, sports activities, personal interests provided by the user, and/or other suitable information can also be used to generate and output to the user context pages with corresponding emojis.
Several embodiments of the disclosed technology can thus provide a user interface that allows users to efficiently locate, select, and insert emojis into electronic messages. In certain embodiments, the emoji picker can be configured to select emojis based on environmental context data and provide the selected emojis in a context page separate from other pages. For example, an emoji model can be used to calculate a probability that an emoji is likely to be used based on the various environment parameters. Upon determining that the probability is above a threshold, the emoji can be added to the context page. As such, the emojis in the context page can be more relevant to the user based on the current date, time, location, and/or other environment parameters. Thus, time spent by the user to browse through and locate desired emojis may be reduced to improve user experience. In addition, by providing an efficient context page to allow the user to locate desired emojis, a computing load and/or network bandwidth consumption of the computing device may be reduced to improve performance of the computing device.
Certain embodiments of systems, devices, components, modules, routines, data structures, and processes for environmental context-based emoji selection in computing devices are described below. In the following description, specific details of components are included to provide a thorough understanding of certain embodiments of the disclosed technology. A person skilled in the relevant art will also understand that the technology can have additional embodiments. The technology can also be practiced without several of the details of the embodiments described below with reference to
As used herein, the term “emoji” generally refers to digital images, pictures, or icons usable in emails, instant messages, text messages, or other suitable types of electronic messages to express or represent ideas, emotions, objects, or meanings. Emojis can be encoded using various encoding standards such as Unicode. For example, a grinning face emoji can have a Unicode of U+1F600. In another example, a thumbs up emoji can have a Unicode of U+1F44D. Also used herein, an “emoji interface” generally refers to a space surfaced on a display (e.g., a screen) of a computing device where interactions between humans and the computing device. Emoji interfaces can include one or more tabs, sections, panes, windows, or other suitable types of user interface elements (collectively referred to herein as “pages” or “emoji pages”). Each page can be configured to contain and display one or more emojis according to categories, frequency of use, and/or other suitable criteria.
Also used herein, “environmental context data” or “context data” generally refers to data representing various circumstances of a user and/or a computing device associated with the user. For example, environmental context data can include data representing one or more of a date, a time, a season, or a location, and/or other suitable environment parameters related to an environment in which a user or a corresponding computing device executing the emoji picker is located. In other examples, environmental context data can also include data representing personal interest of a user, activities of a user as reflected from calendar items, social network postings, or other suitable sources.
Further used herein, a “context page” generally refers to a page or an emoji page in an emoji interface that is configured to provide emojis suggested based on environmental context data of a user and/or a computing device associated with the user. For example, a context page can be configured to display emojis related to Thanksgiving and Christmas holiday season when a current date is within a date range, e.g., from November 1 to December 31. In another example, a context page can be configured to display emojis related to summer activities (e.g., surfing, sunbathing, etc.), food items (e.g., ice cream), weather conditions (e.g., sunshine), or other suitable emojis when a current date is from within another date range, such as June 15 to September 10.
Also used herein, an “emoji picker” generally refers to a software application configured to surface an emoji interface on a display of a computing device. The software application can also be configured to provide facilities for browsing through pages of the emoji interface, accepting a user input to select emojis from the pages of the emoji interface, and inserting the selected emojis into an email, instant message, text message, or other suitable types of electronic messages. One example emoji picker is iPhone Emoji Keyboard provided by Apple, Inc. of Mountain View, California.
Certain emoji pickers can provide multiple pages of emojis grouped according to categories. For example, a facial expression page can include emojis of various facial expressions. A food item page can include emojis of ice cream, pizza, soda, pasta, and other food items. Certain emoji pickers can also provide a frequently used page that contains emojis previously used by users. Grouping emojis in the foregoing manners, however, can be inconvenient for users to locate desired emojis when composing electronic messages. For example, during November and December, a user may browse for emojis often associated with the Thanksgiving and Christmas holiday season, such as emojis for turkeys, Santa Claus, reindeer sleigh, snowman, etc. These emojis, however, may be located in different pages based on corresponding categories. For instance, emojis of turkeys may be located in an animal or food page while emojis of Santa Claus may be located in a person page. As such, a user may spend a long time browsing through multiple emoji pages in order to locate all desired emojis for composition of a holiday electronic message. In another example, when a user first starts using an emoji picker, the frequently used page would be empty. Only after a period of usage, the frequently used page can be populated with previously selected emojis after browsing through multiple emoji pages. Facilitating a user's browsing through multiple emoji pages can place a heavy burden on a computing load and/or network bandwidth consumption of the computing device.
Several embodiments of the disclosed technology can address at least some of the foregoing drawbacks by implementing an emoji picker having one or more context pages configured to provide emojis suggested based on environmental context data. For example, a context page can be configured to provide multiple emojis related to the Thanksgiving and Christmas holiday season based on a current date associated with a user and/or a computing device of the user. Thus, in response to determining that a current date is within a date range, e.g., from November 1 to December 31, the emoji picker can surface the context page related to Thanksgiving and Christmas holiday season to allow the user to efficiently and conveniently select desired holiday emojis. Thus, time spent by the user to browse through and locate desired emojis may be reduced to improve user experience. In addition, by providing the context page to allow the user to locate desired emojis, a computing load and/or network bandwidth consumption of the computing device may be reduced to improve performance of the computing device, as described in more detail below with reference to
Components within a system may take different forms within the system. As one example, a system comprising a first component, a second component and a third component can, without limitation, encompass a system that has the first component being a property in source code, the second component being a binary compiled library, and the third component being a thread created at runtime. The computer program, procedure, or process may be compiled into object, intermediate, or machine code and presented for execution by one or more processors of a personal computer, a network server, a laptop computer, a smartphone, and/or other suitable computing devices.
Equally, components may include hardware circuitry. A person of ordinary skill in the art would recognize that hardware may be considered fossilized software, and software may be considered liquefied hardware. As just one example, software instructions in a component may be burned to a Programmable Logic Array circuit or may be designed as a hardware circuit with appropriate integrated circuits. Equally, hardware may be emulated by software. Various implementations of source, intermediate, and/or object code and associated data may be stored in a computer memory that includes read-only memory, random-access memory, magnetic disk storage media, optical storage media, flash memory devices, and/or other suitable computer readable storage media excluding propagated signals.
As shown in
Even though the application 140 and the emoji picker 106 are shown as a software component executed on the computing device 102 in
The computing device 102 can be configured to facilitate the user 101 to perform various tasks. For example, the computing device 102 can facilitate the user 101 to compose emails, instant messages, text messages or other suitable types of electronic messages. In other examples, the computing device 102 can also facilitate the user 101 to perform various computational, communication, or other suitable types of tasks. In the illustrated embodiment, the computing device 102 includes a desktop computer having one or more processors 304 (shown in
The one or more processors 304 of the computing device 102 can be configured to execute suitable instructions to provide the operating system 104, the application 140, and the emoji picker 106. For instance, as shown in
In the illustrated example in
In response to receiving the request 103, the emoji picker 106 can be configured to provide an emoji interface 146 that facilitate the user 101 to locate, select, and insert one or more emojis 112 into the email 144. As shown in
The emoji model 110 can include data representing correlations of the one or more emojis 112 and the one or more environmental parameters of the user 101 and/or the computing device 102. In one example, the environmental parameters can include a current date and/or current location of the user 101 or computing device 102. In other examples, the other environmental parameters can include a current time, a current season, activities of the user 101, and/or other suitable parameters. In certain embodiments, the correlations in the emoji model 110 can be represented by multiple weight values corresponding to the individual environmental parameters for each emoji 112. Higher weight values may indicate a higher likelihood of use of an emoji 112 based on the corresponding environmental parameter. For instance, a weight value of 1.0 may be assigned to correspond to a current date for a Santa Claus emoji while another weight value of 0.5 may be assigned to the same Santa Claus emoji for a current location. The weight values can then be multiplied by values of the current date and current location to produce a contribution to a probability that the Santa Claus emoji is likely to be used. In other embodiments, such correlations can be represented as mathematically formulas, polynomials, and/or in other suitable ways.
The emoji model 110 can be developed in various ways in accordance with aspects of the disclosed technology. In one embodiment, a software developer of the emoji picker 106 can manually configure the correlations via rules, decision trees, and/or other suitable techniques. For instance, an example rule can include data that indicates that a Santa Claus emoji is associated with a particular date range (e.g., November 1 to December 31) and a particular location (e.g., North America) of the user or computing device. Such manually configured rules can also be updated periodically via software updates or other suitable techniques.
In other embodiments, such correlations can be generated and updated based on monitored emoji 112 usage at a location in which the computing device 102 is located during certain dates, times, seasons, or other time periods. For example, data representing used emojis 112 (referred to below as “emoji usage data”) in text messages in a cellular network may be collected as anonymous data and/or with user consent. The emoji usage data can then be analyzed to determine usage patterns and/or correlations of used emojis and a date, a time, a location, and/or other suitable environmental parameters. As such, a model developer 150 (shown in
As shown in
The interface component 132 can be configured to interface with the operating system 104, the data store 108, and the other suitable components of the computing device 102. For example, the interface component 132 can be configured to receive the request 103 from the user 101. In response to receiving the request 103, the interface component 132 can be configured to retrieve the context data 130 from the operating system 104. The interface component 134 can then forward the request 103 and the retrieved context data 130 to the analysis component 134 for further processing.
The analysis component 134 can be configured to select one or more emojis 112 from the data store 108 based on the emoji model 110 and the context data 130. In one embodiment, the analysis component 134 can be configured to select a number of the emojis 112 having highest frequency of use in all of the emojis 112 at current values of date, time, or location of the computing device 102 as reflected in the retrieved context data. In another embodiment, the analysis component 134 can be configured to derive a sum of products of the current values of date, time, or location and corresponding weight values in the emoji model 110. The analysis component 134 can then determine whether the derived sum exceeds a threshold. In response to determining that the derived sum exceeds a threshold, the analysis component 134 can mark one of the emojis 112 as one of the selected one or more emojis 112. Otherwise, the emoji 112 can be omitted from the context page 118. In further embodiments, the analysis component 134 can be configured to sort the multiple emojis 112 according to respectively sums and select one or more of the multiple emojis 112 having highest sums indicating probability values from the sorted multiple emojis 112. In yet further embodiments, the analysis component 134 can be configured to select the one or more emojis 112 for the context page 118 in other suitable manners based on the context data 130.
The analysis component 134 can then provide the selected one or more emojis 112 for the context page 118 to the control component 136 for further processing. The control component 136 can be configured to generate a context page 118 of emojis 112 by grouping the selected one or more emojis 112 into the context page 118. Though only one context page 118 is shown in
In the illustrated example in
The context page 118 can facilitate the user 101 to locate, select, and insert desired emojis 112 into the email 144. For instance, as shown in
Even though the examples shown in
Based on the information in the calendar items 133, the emoji picker 106 can be configured to generate and provide a corresponding context page 118. For example, a calendar item 133 of the user 101 retrieved by the interface component 132 from the calendar folder 105 may include an appointment for a birthday party. In response, the analysis component 134 can select one or more emojis 112 that are likely be used when discussing a birthday party. For example, as shown in
Several embodiments of the emoji picker 106 can thus provide an emoji interface 146 that allows the user 101 to efficiently locate, select, and insert emojis 112 into email 144. The emojis in the context page 118 can be more relevant to the user 101 based on the current date, time, location, activities related to the user 101, and/or other environment parameters. Thus, time spent by the user 101 to browse through and locate desired emojis 112 may be reduced to improve user experience. In addition, by providing an efficient context page 118 to allow the user to locate desired emojis, a computing load and/or network bandwidth consumption of the computing device 102 may be reduced to improve performance of the computing device 102.
As shown in
As shown in
Depending on the desired configuration, the processor 304 can be of any type including but not limited to a microprocessor (μP), a microcontroller (μC), a digital signal processor (DSP), or any combination thereof. The processor 304 can include one more level of caching, such as a level-one cache 310 and a level-two cache 312, a processor core 314, and registers 316. An example processor core 314 can include an arithmetic logic unit (ALU), a floating-point unit (FPU), a digital signal processing core (DSP Core), or any combination thereof. An example memory controller 318 can also be used with processor 304, or in some implementations memory controller 318 can be an internal part of processor 304.
Depending on the desired configuration, the system memory 306 can be of any type including but not limited to volatile memory (such as RAM), non-volatile memory (such as ROM, flash memory, etc.) or any combination thereof. The system memory 306 can include an operating system 320 (e.g., the operating system 104 in
The computing device 300 can have additional features or functionality, and additional interfaces to facilitate communications between basic configuration 302 and any other devices and interfaces. For example, a bus/interface controller 330 can be used to facilitate communications between the basic configuration 302 and one or more data storage devices 332 via a storage interface bus 334. The data storage devices 332 can be removable storage devices 336, non-removable storage devices 338, or a combination thereof. Examples of removable storage and non-removable storage devices include magnetic disk devices such as flexible disk drives and hard-disk drives (HDD), optical disk drives such as compact disk (CD) drives or digital versatile disk (DVD) drives, solid state drives (SSD), and tape drives to name a few. Example computer storage media can include volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information, such as computer readable instructions, data structures, program modules, or other data. The term “computer readable storage media” or “computer readable storage device” excludes propagated signals and communication media.
The system memory 306, removable storage devices 336, and non-removable storage devices 338 are examples of computer readable storage media. Computer readable storage media include, but not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other media which can be used to store the desired information and which can be accessed by computing device 300. Any such computer readable storage media can be a part of computing device 300. The term “computer readable storage medium” excludes propagated signals and communication media.
The computing device 300 can also include an interface bus 340 for facilitating communication from various interface devices (e.g., output devices 342, peripheral interfaces 344, and communication devices 346) to the basic configuration 302 via bus/interface controller 330. Example output devices 342 include a graphics processing unit 348 and an audio processing unit 350, which can be configured to communicate to various external devices such as a display or speakers via one or more A/V ports 352. Example peripheral interfaces 344 include a serial interface controller 354 or a parallel interface controller 356, which can be configured to communicate with external devices such as input devices (e.g., keyboard, mouse, pen, voice input device, touch input device, etc.) or other peripheral devices (e.g., printer, scanner, etc.) via one or more I/O ports 358. An example communication device 346 includes a network controller 360, which can be arranged to facilitate communications with one or more other computing devices 362 over a network communication link via one or more communication ports 364.
The network communication link can be one example of a communication media. Communication media can typically be embodied by computer readable instructions, data structures, program modules, or other data in a modulated data signal, such as a carrier wave or other transport mechanism, and can include any information delivery media. A “modulated data signal” can be a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal. By way of example, and not limitation, communication media can include wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, radio frequency (RF), microwave, infrared (IR) and other wireless media. The term computer readable media as used herein can include both storage media and communication media.
The computing device 300 can be implemented as a portion of a small-form factor portable (or mobile) electronic device such as a cell phone, a personal data assistant (PDA), a personal media player device, a wireless web-watch device, a personal headset device, an application specific device, or a hybrid device that include any of the above functions. The computing device 300 can also be implemented as a personal computer including both laptop computer and non-laptop computer configurations.
From the foregoing, it will be appreciated that specific embodiments of the disclosure have been described herein for purposes of illustration, but that various modifications may be made without deviating from the disclosure. In addition, many of the elements of one embodiment may be combined with other embodiments in addition to or in lieu of the elements of the other embodiments. Accordingly, the technology is not limited except as by the appended claims.
