Patent Application
20150278370
Many users perform tasks using computing devices. In an example, a user may map directions from a current location to an amusement park using a mobile device. In another example, a user may read a book using a tablet device. Various types of input may be used to perform tasks, such as touch gestures, mouse input, keyboard input, voice commands, search query input, etc. For example, while performing a vacation booking task, a user may input a search query “Florida vacations” into a search engine, and the search engine may return a variety of vacation search results that the user may use to complete the vacation booking task.
This summary is provided to introduce a selection of concepts in a simplified form that are further described below in the detailed description. This summary is not intended to identify key factors or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter.
Among other things, one or more systems and/or techniques for facilitating task completion are provided herein. In an example, a natural language input may be received from a user of a client device (e.g., a voice command “what do I wear”). The natural language input may be evaluated using a set of user contextual signals associated with the user to identify a user task intent (e.g., a user may take affirmative action to provide opt-in consent for granting access to various types of user contextual signals and/or the user may opt-out to prevent access to certain types of user contextual signals). In an example, a time user signal (e.g., a current time of 6:00 pm), a geolocation user signal (e.g., a downtown hotel location), email data (e.g., a dinner reservation email at a fancy restaurant), a user social network profile (e.g., indicating that the user is a female), and/or other information may be used to identify a user task intent of viewing formal cocktail dress ideas through a fashion app. In an example of identifying the user task intent, a user intent query may be constructed based upon the natural language input, and a task intent data structure (e.g., hosted by a remote server) may be queried to obtain a global intent candidate (e.g., what tasks users of a search engine performed after submitting search queries similar to the user intent query) that may evaluated using the set of user contextual signals to identify the user task intent.
Task completion functionality may be exposed to the user based upon the user task intent. For example, a fashion app may be executed for the user. In an example, the fashion app may be deep launched into a contextual state that may be relevant to the user. For example, a task execution context (e.g., a female clothing parameter, a formal wear parameter, and/or other contextual information/parameters) may be identified based upon the user task intent. The fashion app may be deep launched into a female clothing wear shopping interface (e.g., populated with clothing corresponding to the female clothing parameter and the formal wear parameter) based upon the task execution context. In this way, task completion functionality may be exposed to the user based upon natural language input.
In an example, a task facilitator component may be implemented on the client device for facilitating task completion (e.g., the task facilitator component may identify and/or locally utilize user contextual signals, which may promote preservation of privacy of user data). In another example, a user intent provider component may be implemented on a server, remote from the client device, for facilitating task completion (e.g., the user intent provider component may receive the natural language input and/or a user intent query derived from the natural language input, and may provide a global intent candidate and/or an instruction to expose task completion functionality to the client device).
To the accomplishment of the foregoing and related ends, the following description and annexed drawings set forth certain illustrative aspects and implementations. These are indicative of but a few of the various ways in which one or more aspects may be employed. Other aspects, advantages, and novel features of the disclosure will become apparent from the following detailed description when considered in conjunction with the annexed drawings.
The claimed subject matter is now described with reference to the drawings, wherein like reference numerals are generally used to refer to like elements throughout. In the following description, for purposes of explanation, numerous specific details are set forth to provide an understanding of the claimed subject matter. It may be evident, however, that the claimed subject matter may be practiced without these specific details. In other instances, structures and devices are illustrated in block diagram form in order to facilitate describing the claimed subject matter.
One or more techniques and/or systems for facilitating task completion are provided herein. Natural language input may be evaluated to semantically and/or contextually understand a user intent to perform a task. The natural language input may be evaluated based upon global user information (e.g., what tasks various users of a search engine performed after submitting a search query) and/or personalized user information (e.g., content currently consumed by the user, a location (e.g., GPS) of the user, an email, a calendar appointment, and/or other user contextual signals that the user has opted-in to provide for facilitating task completion). In this way, task completion functionality may be exposed to the user based upon global and/or personalized evaluations of the natural language input. For example, an application may be deep launched into a contextual state associated with a task execution context identified from the user task intent (e.g., a restaurant app may be launched into a view of a menu for a Mexican restaurant based upon a voice command “I am hungry” and user contextual signals such as a location of the user, a social network profile interest in Mexican food, etc.).
An embodiment of facilitating task completion is illustrated by an exemplary method 100 of
In an example, the natural language input (e.g., and/or the global intent candidate) may be evaluated using a set of user contextual signals associated with the user to identify a user task intent. The set of user contextual signals may comprise a geolocation (e.g., the user may be at a coffee shop), a time, an executing application (e.g., a car design application), an installed application (e.g., an art drawing application), an app store application (e.g., a car review application), calendar data (e.g., a calendar entry to create a car review), email data, social network data (e.g., an indication that the user works for a car magazine company), a device form factor (e.g., desktop computer at work), a user search log (e.g., the user may have recently visited car photography websites), content consumed by the user (e.g., car photos and/or videos), community user intent for the natural language input (e.g., the global intent candidate corresponding to the draw query to art application intent entry). The set of user contextual signals may comprise information that the user may have opted-in to share for the purpose of facilitating user task completion. In an example, a user task intent to execute the art drawing application and draw a car may be identified.
In an example, a user refinement interface may be provided to the user based upon the user task intent (e.g., the user may be asked as to whether the user task intent is correct). A user task refinement input or a user acknowledgement may be received through user refinement interface. For example, the user may indicate that the user has a refined user task intent to open the car review application and create a car review with a drawing of a car. Accordingly, the user task intent may be revised based upon the user task refinement input.
At 108, task completion functionality may be exposed to the user based upon the user task intent. Task completion functionality may comprise providing the user with access to a document, an application (e.g., executing an application, deep launching an application, downloading an application from an app store, etc.), an operating system setting, a music entity, a video, a photo, a social network profile, a map, a search result, and/or a variety of other objects and/or functionality (e.g., functionality to purchase a book, functionality to reserve a table at a restaurant, etc.). In an example, the task completion functionality may comprise executing the car review application based upon the refined user task intent to open the car review application and create a car review with a drawing of a car. A task execution context may be identified based upon the user task intent (e.g., a car review creation interface of the car review application may be identified as the task execution context). The car review application may be deep launched into a contextual state associated with the task execution context (e.g., the car review application may be instructed to display the car review creation interface, as opposed to a car review reading interface). In an example, the task execution context may comprise one or more application parameters (e.g., a display car drawing interface parameter used to specify whether a car drawing interface is to be displayed through the car review creation interface). The car review application may be populated with information corresponding to the one or more application parameters (e.g., the car drawing interface may be displayed). In this way, natural language input may be used to expose task completion functionality, such as a deep launched application in a contextually relevant state, to a user.
In an example, user feedback for the task completion functionality may be identified. For example, the user may indicate that the user would have preferred to receive suggestions of car review creation apps to download from an app store as part of the task completion functionality. The user feedback may be provided to the server (e.g., the remote server hosting the task intent data structure) for training a task intent model used to populate the task intent data structure (e.g., a new query to intent entry may be created to match the natural language input and/or the user intent query to the task of previewing and downloading car review creation apps). In this way, facilitating task completion based upon natural language input may be improved. At 110, the method ends.
The user intent provider component 210 may be configured to receive a user intent query 242 from a client device. The user intent query 242 may be derived from a natural language input received on the client device (e.g., a user intent query to view vacation media may be derived from a natural language input of “show me my vacation”). The user intent provider component 210 may query the task intent data structure 208 using the user intent query 242 to identify a global intent candidate 214 (e.g., a global intent candidate of display photos comprising metadata associated with vacation). The global intent candidate 214 may be provided to the client device for facilitating task completion associated with a user task intent derived from the natural language input (e.g., a photo viewer app may be deep launched into a contextual state where vacation photos are displayed).
The task facilitator component 306 may be configured to expose task completion functionality 312 to the user. For example, the task completion functionality 312 may correspond to deep launching a restaurant app 314. The current location of the user may be used to identify a set of Mexican restaurant entity candidates corresponding to the user task intent 310. A Mexican restaurant entity candidate may be selected from the set of Mexican restaurant entity candidates based upon a proximity of the Mexican restaurant entity candidate to the current location of the user. In this way, the restaurant app 314 may be deep launched where information associated with the Mexican restaurant entity candidate is populated within the restaurant app 314 (e.g., walking directions, a menu, etc.). In this way, the restaurant app 314 is deep launched into a contextually relevant state based upon the natural language input 304 and/or the set of user contextual signals 308.
Still another embodiment involves a computer-readable medium comprising processor-executable instructions configured to implement one or more of the techniques presented herein. An example embodiment of a computer-readable medium or a computer-readable device is illustrated in
Although the subject matter has been described in language specific to structural features and/or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing at least some of the claims.
As used in this application, the terms “component,” “module,” “system”, “interface”, and/or the like are generally intended to refer to a computer-related entity, either hardware, a combination of hardware and software, software, or software in execution. For example, a component may be, but is not limited to being, a process running on a processor, a processor, an object, an executable, a thread of execution, a program, and/or a computer. By way of illustration, both an application running on a controller and the controller can be a component. One or more components may reside within a process and/or thread of execution and a component may be localized on one computer and/or distributed between two or more computers.
Furthermore, the claimed subject matter may be implemented as a method, apparatus, or article of manufacture using standard programming and/or engineering techniques to produce software, firmware, hardware, or any combination thereof to control a computer to implement the disclosed subject matter. The term “article of manufacture” as used herein is intended to encompass a computer program accessible from any computer-readable device, carrier, or media. Of course, many modifications may be made to this configuration without departing from the scope or spirit of the claimed subject matter.
Although not required, embodiments are described in the general context of “computer readable instructions” being executed by one or more computing devices. Computer readable instructions may be distributed via computer readable media (discussed below). Computer readable instructions may be implemented as program modules, such as functions, objects, Application Programming Interfaces (APIs), data structures, and the like, that perform particular tasks or implement particular abstract data types. Typically, the functionality of the computer readable instructions may be combined or distributed as desired in various environments.
In other embodiments, device 812 may include additional features and/or functionality. For example, device 812 may also include additional storage (e.g., removable and/or non-removable) including, but not limited to, magnetic storage, optical storage, and the like. Such additional storage is illustrated in
The term “computer readable media” as used herein includes computer storage media. Computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions or other data. Memory 818 and storage 820 are examples of computer storage media. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, Digital Versatile Disks (DVDs) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by device 812. Computer storage media does not, however, include propagated signals. Rather, computer storage media excludes propagated signals. Any such computer storage media may be part of device 812.
Device 812 may also include communication connection(s) 826 that allows device 812 to communicate with other devices. Communication connection(s) 826 may include, but is not limited to, a modem, a Network Interface Card (NIC), an integrated network interface, a radio frequency transmitter/receiver, an infrared port, a USB connection, or other interfaces for connecting computing device 812 to other computing devices. Communication connection(s) 826 may include a wired connection or a wireless connection. Communication connection(s) 826 may transmit and/or receive communication media.
The term “computer readable media” may include communication media. Communication media typically embodies computer readable instructions or other data in a “modulated data signal” such as a carrier wave or other transport mechanism and includes any information delivery media. The term “modulated data signal” may include a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal.
Device 812 may include input device(s) 824 such as keyboard, mouse, pen, voice input device, touch input device, infrared cameras, video input devices, and/or any other input device. Output device(s) 822 such as one or more displays, speakers, printers, and/or any other output device may also be included in device 812. Input device(s) 824 and output device(s) 822 may be connected to device 812 via a wired connection, wireless connection, or any combination thereof. In one embodiment, an input device or an output device from another computing device may be used as input device(s) 824 or output device(s) 822 for computing device 812.
Components of computing device 812 may be connected by various interconnects, such as a bus. Such interconnects may include a Peripheral Component Interconnect (PCI), such as PCI Express, a Universal Serial Bus (USB), firewire (IEEE 1394), an optical bus structure, and the like. In another embodiment, components of computing device 812 may be interconnected by a network. For example, memory 818 may be comprised of multiple physical memory units located in different physical locations interconnected by a network.
Those skilled in the art will realize that storage devices utilized to store computer readable instructions may be distributed across a network. For example, a computing device 830 accessible via a network 828 may store computer readable instructions to implement one or more embodiments provided herein. Computing device 812 may access computing device 830 and download a part or all of the computer readable instructions for execution. Alternatively, computing device 812 may download pieces of the computer readable instructions, as needed, or some instructions may be executed at computing device 812 and some at computing device 830.
Various operations of embodiments are provided herein. In one embodiment, one or more of the operations described may constitute computer readable instructions stored on one or more computer readable media, which if executed by a computing device, will cause the computing device to perform the operations described. The order in which some or all of the operations are described should not be construed as to imply that these operations are necessarily order dependent. Alternative ordering will be appreciated by one skilled in the art having the benefit of this description. Further, it will be understood that not all operations are necessarily present in each embodiment provided herein. Also, it will be understood that not all operations are necessary in some embodiments.
Further, unless specified otherwise, “first,” “second,” and/or the like are not intended to imply a temporal aspect, a spatial aspect, an ordering, etc. Rather, such terms are merely used as identifiers, names, etc. for features, elements, items, etc. For example, a first object and a second object generally correspond to object A and object B or two different or two identical objects or the same object.
Moreover, “exemplary” is used herein to mean serving as an example, instance, illustration, etc., and not necessarily as advantageous. As used herein, “or” is intended to mean an inclusive “or” rather than an exclusive “or”. In addition, “a” and “an” as used in this application are generally be construed to mean “one or more” unless specified otherwise or clear from context to be directed to a singular form. Also, at least one of A and B and/or the like generally means A or B or both A and B. Furthermore, to the extent that “includes”, “having”, “has”, “with”, and/or variants thereof are used in either the detailed description or the claims, such terms are intended to be inclusive in a manner similar to the term “comprising”.
Also, although the disclosure has been shown and described with respect to one or more implementations, equivalent alterations and modifications will occur to others skilled in the art based upon a reading and understanding of this specification and the annexed drawings. The disclosure includes all such modifications and alterations and is limited only by the scope of the following claims. In particular regard to the various functions performed by the above described components (e.g., elements, resources, etc.), the terms used to describe such components are intended to correspond, unless otherwise indicated, to any component which performs the specified function of the described component (e.g., that is functionally equivalent), even though not structurally equivalent to the disclosed structure. In addition, while a particular feature of the disclosure may have been disclosed with respect to only one of several implementations, such feature may be combined with one or more other features of the other implementations as may be desired and advantageous for any given or particular application.
