Patent Application
20160378871
This disclosure generally relates to the field of search, and more particularly to techniques for generating search results that correspond to states of software applications.
In recent years, the use of computers, smartphones, and other Internet-connected computing devices has grown significantly. Correspondingly, the number of software applications (apps) available for such devices has also grown. Today, many diverse software apps can be accessed on a number of different computing devices, including smartphones, personal computers, automobiles, and televisions. These software apps can include business driven apps, games, educational apps, news apps, shopping apps, messaging apps, media streaming apps, and social networking apps, as some examples. Because of the large number of software apps available today and the wide range of functionality they provide, computing device users often require the ability to search for and access specific software app functionality.
In one example, a method includes receiving a search query from a user device, receiving an indication of a current application (app) state of a native app executing on the device from the device, and identifying one or more app state records based on the query and the indication. In this example, each app state record includes an app access mechanism (AAM) and app state information (ASI). The AAM references a native app and indicates one or more operations for the app to perform. The ASI describes an app state of the native app after the app has performed the operations. The method further includes selecting the one or more AAMs from the identified app state records and transmitting the selected AAMs to the user device.
In another example, a method includes receiving a search query from a user device, receiving an indication of a current app state of a native app executing on the device from the device, and identifying one or more app state records based on the query. In this example, each app state record includes an AAM and ASI. The AAM references a native app and indicates one or more operations for the app to perform. The ASI describes an app state of the native app after the app has performed the operations. The method further includes generating a result score for each of the identified app state records based on the indication and selecting one or more of the records based on the one or more result scores. The method also includes selecting the one or more AAMs from the selected app state records and transmitting the AAMs to the user device.
In another example, a system includes one or more computing devices configured to receive a search query from a user device, receive an indication of a current app state of a native app executing on the device from the device, and identify one or more app state records based on the query and the indication. In this example, each app state record includes an AAM and ASI. The AAM references a native app and indicates one or more operations for the app to perform. The ASI describes an app state of the native app after the app has performed the operations. The computing devices are further configured to select the one or more AAMs from the identified app state records and transmit the selected AAMs to the user device.
In another example, a system includes one or more computing devices configured to receive a search query from a user device, receive an indication of a current app state of a native app executing on the device from the device, and identify one or more app state records based on the query. In this example, each app state record includes an AAM and ASI. The AAM references a native app and indicates one or more operations for the app to perform. The ASI describes an app state of the native app after the app has performed the operations. The computing devices are further configured to generate a result score for each of the identified app state records based on the indication and select one or more of the records based on the one or more result scores. The computing devices are also configured to select the one or more AAMs from the selected app state records and transmit the AAMs to the user device.
The details of one or more examples are set forth in the accompanying drawings and the description below. Other features, objects, and advantages will be apparent from the description and drawings, and from the claims.
The figures and the following description relate to example implementations by way of illustration only. It should be noted that from the following discussion, alternative implementations of the structures and methods disclosed herein will be readily recognized as viable alternatives that may be employed without departing from the scope of this disclosure.
The present disclosure generally relates to the field of search, and, more particularly, to techniques for generating and displaying search results at a computing device based on a current application (app) state of a software app executing on the device. Using the techniques described herein may, in some examples, improve search query understanding and search result relevance. For example, the techniques include, while performing a search for app states of software apps based on a user's search query, using an app state of a software app executing on the user's device at the time of the search (e.g., the user's “current app state”) to improve relevance of search results that are responsive to the query. In particular, the techniques include using information associated with (e.g., displayed as part of) the current app state as a textual relevance feature, a category relevance feature, an entity relevance feature, and/or as a search result scoring feature, e.g., as part of a machine-learned relevance (MLR) model.
According to the disclosed techniques, initially, a user of a user device may interact with a so-called “native” app executing on the device. For example, the user may set the native app into a particular app state via the user's interaction. The app state of the native app displayed on the user device may be referred to herein as a “current app state.” While the user device (e.g., the native app) displays the current app state, the user may further cause the device to launch a search app (e.g., a native or web-based app that enables the user to perform searches on the device). As a result, the user device may launch and execute the search app while continuing to execute the current app state. For example, the user device may display a GUI of the launched search app, with the current app state displayed below, or in the background of, the GUI. The user may then input a search query (e.g., a text string) into a search field of the GUI and cause the user device to transmit the query and an indication of the current app state to a search system. The search system may receive the search query and the indication from the user device and generate search results that are responsive to (e.g., that provide the functionality described by) the query and are relevant to the current app state. For example, as described herein, the search results may each specify an app state of a native app that provides the function described by the search query and is relevant to the current app state, as specified by the indication, based on one or more of text relevance, entity relevance, and category relevance considerations. Additionally, or alternatively, the search system may generate the search results using one or more aspects of the current app state as a scoring feature, as also described herein.
The search system may transmit the search results to the user device. The user device may receive the search results from the search system and display the results to the user (e.g., as one or more user selectable links). The user may select one or more of the search results (e.g., the user selectable links). In some examples, upon the user selecting a particular search result (e.g., an associated user selectable link), the user device may launch the corresponding native app and set the app into an app state specified by the result. The user may then interact with the app state (e.g., preview or perform a function provided by the state). In other examples, upon the user selecting the search result, the user device may download and install the native app. In these examples, the user device may then launch the native app and set the app into the app state specified by the search result. In still other examples, upon the user selecting the search result, the user device may launch a web browser app and access a URL (e.g., corresponding to a web equivalent of the app state of the native app) specified by the result.
In this manner, the techniques described herein may improve search query understanding and search result relevance. As one example, the current app state of the native app executing on the user device may indicate the user's activity with respect to the app prior to and/or at the time of invoking the search app. For instance, the current app state may indicate whether, at the time of invoking the search app, the user was interacting with a main page of the native app, or a specific page of (e.g., an entry for a particular business, product, or service in) the app. As such, the current app state may indicate the user's need for functionality associated with the current app state. Moreover, the user invoking the search app after interacting with the current app state may further indicate the user's dissatisfaction with the functionality provided by the state. Accordingly, the current app state may serve as contextual data related to the user and the user's behavior (e.g., the user's intent, or need) with respect to the search query. By performing searches for app states of native apps using both the user's search query and the indication of the current app state as search parameters, the techniques may improve understanding of the query. Additionally, by identifying app states of native apps that are both responsive to (e.g., match) the search query and are relevant to the current app state (e.g., based on text, category, and/or entity considerations), the techniques may further enable generating search results that are more relevant to the query than those generated using the query alone.
In the example of
In this disclosure, an app may refer to computer software that causes a computing device (e.g., one of the user device(s) 102) to perform a task. In some examples, an app may be referred to as a “program.” Example apps include word processing apps, spreadsheet apps, messaging apps, media streaming apps, social networking apps, and games. Apps can be executed on a variety of different computing devices, including mobile computing devices, such as smartphones, tablets, and wearable computing devices (e.g., smart headsets and/or smart watches). Apps can also be executed on other types of computing devices having other form factors, such as laptop computers, desktop computers, or other consumer electronic devices. In some examples, apps may be installed on a computing device prior to a user purchasing the device. In other examples, the user may download and install apps on the computing device after purchasing the device. A native app, as used herein, may refer to an app that is installed and executed on a user device 102. A web-based app, in turn, may refer to an app that is accessible from a user device 102 via a web browser app included on the device 102.
An AM, as used herein, may include any of a native app AM (hereinafter, “app AM,” or “AAM”), a web AM (hereinafter, “WAM”), and an app download address (ADA). As such, a user device 102 of the present disclosure may use an AM to access the functionality provided by a native or a web-based app. For example, a user of the user device 102 may select a user selectable link that includes the AM to access the functionality of the app.
An AAM may be a string that references a native app and indicates one or more operations for a user device 102 (e.g., the app) to perform. If a user of the user device 102 selects a user selectable link that includes the AAM, the device 102 may launch the native app and (e.g., cause the app to) perform the operations. In other words, the user selecting the user selectable link may cause the user device 102 to launch the native app and set the app into an app state (e.g., in which the app displays a GUI, or screen) that corresponds to the operations. As a result, the native app may be configured to display one or more products, services, or vendors, to the user, e.g., via a display device of the user device 102. In this manner, the AAM may specify the app state of the native app. The app state, in turn, may refer to the operations indicated by the AAM and/or the outcome of the native app performing the operations in response to the user selecting the user selectable link that includes the AAM.
A WAM may include a resource identifier that references a web resource (e.g., a page of a web-based app, or website). For example, the WAM may include a uniform resource locator (URL) (i.e., a web address) used with the hypertext transfer protocol (HTTP). If a user of a user device 102 selects a user selectable link that includes the WAM, the device 102 may launch a web browser app included on the device 102 and retrieve the web resource referenced by the resource identifier. Stated another way, if the user selects the user selectable link, the user device 102 may launch the web browser app and access an app state (e.g., a page) of a web-based app, or website, specified by the WAM. In some examples, a WAM included in an app state record with an AAM may specify an app state of a web-based app that is an equivalent of (e.g., analogous to) an app state of a native app specified by the AAM.
An ADA may specify a location (e.g., a digital distribution platform, such as Google Play® by Google Inc.) where a native app (e.g., a native app referenced by an AAM) may be downloaded. In some examples, an app state record may include an ADA with an AAM (and, e.g., a WAM). In these examples, the ADA may specify a location from which a native app referenced by the AAM may be downloaded.
In some examples, the search system 100 may transmit the search results, including the AMs, to the user device 102 with additional data. For example, the search system 100 may transmit link (e.g., text and/or image) data that the user device 102 may use to generate the user selectable links for the AMs included in the search results. Each user selectable link may include a portion of the link (e.g., text and/or image) data that the user of the user device 102 may select (e.g., touch, or “click on”). Each user selectable link may also be associated with one or more of the AMs included in the search results, such that when the user selects the link, the user device 102 launches a native or web-based app referenced by the corresponding AM(s) and causes the app to perform one or more operations indicated by the AM(s). The text and/or image data of the user selectable link may indicate the operations or function that the native or web-based app performs in response to selection of the link. For example, if the user selectable link is for a song in a native or web-based music player app, the text and/or image data may indicate that the user device 102 will launch the app and that the app will play the song when the user selects the link. In some examples, when the user selects the user selectable link, the user device 102 downloads a native app referenced by the corresponding AM(s) and installs the app on the device 102. Example user selectable links are described with reference to
As described herein, the search system 100 uses data (e.g., app state records) included in the search data store 112 to generate search results based on search queries and indications of current app states received from the user device(s) 102. The search data store 112 may include one or more databases, indices (e.g., inverted indices), tables, files, or other data structures that may be used to implement the techniques of this disclosure. In some examples, the search data store 112 may be included in one or more storage devices. The search data store 112 includes one or more app state records. Each app state record may include data related to a function of a native app and/or to an app state of the app resulting from the app performing the function. For example, each app state record may include, among other content, an app state ID, ASI, and one or more AMs. An app state ID of an app state record may uniquely identify the record among other app state records included in the search data store 112. ASI of an app state record may describe an app state into which an app is set according to one or more AMs included in the record. An AM (e.g., an AAM, or a WAM) of an app state record may include data (e.g., an alphanumeric string) that causes a user device 102 to launch a native or web-based app and perform a function associated with the app. In some examples, an AM of an app state record may include data (e.g., an ADA) that enables a user device 102 to download and install a native app on the device 102. Example app state records are described with reference to
As described herein, the search system 100 receives the search query and the indication of the current app state from the user device 102 and generates the search results based on the query and the indication. The search query may include text, numbers, and/or symbols (e.g., punctuation) entered into the user device 102 by the user. For example, the user may have entered the search query into a search field, or “search box,” of a search app included on the user device 102. The user may have entered the search query using a touchscreen keypad, a mechanical keypad, and/or via speech recognition techniques and transmitted the query to the search system 100 using the search app. In some examples, the search app may be a native app dedicated to search, or a more general app, such as a web browser app. The indication of the current app state may include text, numbers, and/or symbols (e.g., punctuation), as well as any machine-readable (e.g., binary) data used by the user device 102 (e.g., the search app) to represent the state (e.g., an app state ID, an AAM, and/or one or more operations associated with the state), as described herein. In some examples, the indication may include or reference any of terms (e.g., text), categories (e.g., types), entities, and/or other data associated with the current app state, as also described herein. In some instances, the search app may determine the current app state and transmit the indication to the search system 100 (e.g., with the search query).
In some examples, the user device 102 may transmit additional data to the search system 100 with the search query and the indication of the current app state. The search query, the indication, and the additional data may be referred to herein as a query wrapper. The additional data may include geo-location data associated with the user device 102, platform data for the device 102 (e.g., a type and/or a version of the device 102, an operating system (OS), and/or a web browser app of the device 102), an identity of the user (e.g., a username), partner specific data, and other data. The user device 102 may transmit the query wrapper to the search system 100, which may use the search query, the indication, and/or the additional data included in the wrapper to generate the search results and provide the results to the device 102.
To generate the search results, the search module 110 may identify one or more app state records included in the search data store 112 based on the search query and, e.g., the indication of the current app state. Initially, the search module 110 may analyze the search query. The search module 110 may then identify one or more app state records included in the search data store 112 based on the (e.g., analyzed) search query and, e.g., the indication. For example, the search module 110 may identify the app state records based on (e.g., text) matches between terms of the search query and terms of information included in the records. In some examples, the search module 110 may further identify the app state records based on matches between the indication (or, e.g., information generated using the indication) and information included in the records. The search module 110 may then process (e.g., score) the identified app state records. For example, the search module 110 may determine how well the identified app state records match the search query and, e.g., various aspects of the current app state, as specified by the indication. The search module 110 may then select one or more of the identified app state records that best match the search query and, e.g., various aspects of the current app state, and transmit indications of the selected records to the result generation module 114.
The result generation module 114 may identify the app state records selected by the search module 110 in the search data store 112 using the received indications. The result generation module 114 may then select one or more AMs (e.g., AAMs, WAMs, and/or ADAs) from the identified app state records. The result generation module 114 may transmit the selected AMs to the user device 102 as the search results. In some examples, the result generation module 114 may transmit additional data with the AMs to the user device 102. For example, as described herein, the search module 110 may generate result scores for the app state records from which the AMs are selected (e.g., using values of metrics associated with the persons, places, or things described in the records, various features of the search query, and, e.g., various features of the current app state as specified by the indication). As such, each AM may be associated with a result score that indicates a rank of the AM relative to the other AMs. In some examples, the result generation module 114 may transmit the result scores associated with the AMs to the user device 102 with the AMs. Additionally, or alternatively, the result generation module 114 may transmit link data and/or indications of user interaction distances associated with the AMs (e.g., with the corresponding app state records and app states specified by the records) to the user device 102, as also described herein.
The user device(s) 102 may be any computing devices capable of providing search queries and indications of current app states to the search system 100 (and, e.g., the app state determination system 108) and receiving search results from the search system 100. The user device(s) 102 may include any of smartphones, and tablet, laptop, or desktop computers. The user device(s) 102 may also include any computing devices having other form factors, e.g., computing devices included in vehicles, gaming devices, televisions, or other appliances (e.g., networked home automation devices and home appliances). The user device(s) 102 may use a variety of different operating systems or platforms. In instances where a user device 102 is a mobile device, the device 102 may operate using an OS such as ANDROID® by Google Inc., IOS® by Apple Inc., or WINDOWS PHONE® by Microsoft Corporation. In instances where the user device 102 is a laptop or desktop computing device, the device 102 may use an OS such as MICROSOFT WINDOWS® by Microsoft Corporation, MAC OS® by Apple Inc., or LINUX® (LINUX is the registered trademark of Linus Torvalds in the U.S. and other countries). The user device(s) 102 may interact with any of the systems 100, 108 using operating systems other than those described herein, whether presently available or developed in the future.
The user device(s) 102 can communicate with the search system 100 (and, e.g., the app state determination system 108) via the network 106. In general, the user device(s) 102 may communicate with any of the systems 100, 108 using any app that can transmit search queries and indications of current app states to the systems 100, 108 and receive search results from the search system 100. In some examples, the user device(s) 102 may include an app that is dedicated to interfacing with one or more of the systems 100, 108, such as an app dedicated to searches (e.g., a search app). In other examples, the user device(s) 102 may communicate with any of the systems 100, 108 using a more general app, such as a web browser app. An app included on a user device 102 to communicate with any of the systems 100, 108 may include a GUI with a search field, or “box,” into which users may enter search queries, e.g., using a touchscreen, physical keyboard, a speech-to-text program, or other form of user input available on the device 102. In some examples, the app may also be configured to determine (e.g., via a current app state determination module) current app states of native apps executing on the user device 102 and transmit indications of the states to one or more of the systems 100, 108.
The user device 102 may use a GUI of a search app, or a more general app, included on the device 102 to display search results to the user. The user device 102 may also use the GUI to receive search queries from the user and transmit the queries and indications of current app states to the search system 100 (and, e.g., the app state determination system 108). The GUI may display the search results to the user in a variety of different ways, depending on the information transmitted to the user device 102 from the search system 100. In examples where the search results include one or more AMs, the search system 100 may transmit the AMs to the user device 102 with result scores, link data, and/or other information (e.g., indications of user interaction distances) used to generate and display (e.g., rank) one or more user selectable links for the AMs. In some examples, the GUI may display the search results to the user as a list of the user selectable links, including text and/or image data. For example, the text and/or images data may include names of apps referenced by the AMs, descriptions of the AMs and/or operations indicated therein, and images associated with the apps, or app states thereof, specified by the AMs (e.g., app, or app state icons or “screens”). In some examples, the GUI may display the search results as the list of the user selectable links arranged under a search field into which the user has entered a search query. The GUI may arrange the user selectable links by result scores associated with the links, i.e., with the AMs for which the links are generated, or using other logic (e.g., based on user interaction distance). In some examples, the GUI may also group the user selectable links by the associated native apps (e.g., using native app headers).
The data source(s) 104 may be sources of data that the search system 100 may use to generate and/or update the search data store 112. For example, the search system 100 may use the data source(s) 104 to generate and/or update one or more databases, indices, tables, files, or other data structures included in the search data store 112. The search system 100 may generate new app state records and update existing app state records based on data retrieved from the data source(s) 104. For example, the search system 100 may include modules that generate new app state records and/or update existing app state records based on the data retrieved from the data source(s) 104. In some examples, some or all of the data included in the search data store 112 (e.g., one or more app state records) may be manually generated by a human operator.
The data source(s) 104 may include a variety of different data providers. For example, the data source(s) 104 may include data from app developers, such as app developer websites and data feeds provided by app developers. The data source(s) 104 may also include operators of digital distribution platforms configured to distribute apps to user devices 102. The data source(s) 104 may further include other websites, such as websites that include web logs (i.e., blogs), app reviews, or other data related to apps. Additionally, the data source(s) 104 may include social networking sites, such as “FACEBOOK®” by Facebook Inc. (e.g., Facebook posts) and “TWITTER®” by Twitter Inc. (e.g., text from tweets). The data source(s) 104 may also include online databases that include data related to movies, television programs, music, and restaurants. The data source(s) 104 may further include other types of data sources, which may have various types of content and update rates.
In some examples, the search system 100 may retrieve data from the data source(s) 104, including any type of data related to native app functionality and/or native app states. The search system 100 may then generate one or more app state records based on the data and store the records in the search data store 112. In other examples, some or all of the data included in the app state records (e.g., ASI) may be manually generated by a human operator. Additionally, in some examples, the data included in the app state records may be updated over time so that the search system 100 may provide up-to-date search results in response to user-specified search queries and indications of current app states received from the user device(s) 102.
In some examples, the search system 100 (e.g., the app state determination system 108, for example, as part of the app state determination module 116, as shown in
In the example of
In some examples, as shown in
In the examples described above, each app state record included in the search data store 112 may indicate one or more terms (e.g., ASI), categories, and/or entities associated with the app state specified by the record, as described with reference to
The information conveyed by the search results 134 may depend on how the set processing module 142 generates the result scores for the app state records of the consideration set. For example, for each app state record, the corresponding result score may be generated based on various features associated with the record, such as relevance of the app state of the native app specified by the record to the search query 130, popularity of the state, and/or other properties of the state, depending on the one or more parameters the set processing module 142 uses to score the app state records. The set processing module 142 may generate the result scores for the app state records in a variety of different ways. In some examples, the set processing module 142 generates a result score for an app state record based on one or more scoring features. The scoring features may be associated with the app state record, the search query 130, and/or other data. An app state record scoring feature (hereinafter, “record scoring feature”) may be based on any data associated with an app state record. For example, a record scoring feature may be based on any data included in ASI of an app state record. An example record scoring feature may be a popularity score (e.g., based on user ratings of a native app or a state of the app) associated with an app state record. A query scoring feature may include any data associated with the search query 130. For example, a query scoring feature may include any of a number of words in the search query 130, popularity of the query 130, and an expected frequency of the words in the query 130. A record-query scoring feature may include any data generated based on information associated with both an app state record and a search query 130 that resulted in identification of the record by the set generation module 140. For example, a record-query scoring feature may include any parameters that indicate how well terms of a search query 130 match terms of ASI (and/or an app state ID) of an app state record identified using the query 130. In some examples, as described herein, the set processing module 142 may generate a result score for an app state record based on an indication of a current app state 132 of a native app executing on a user device 102. In these examples, a “current app state” scoring feature may include one or more terms, categories, and/or entities associated with the current app state. As one example, a current app state scoring feature may include any parameters that indicate how well the terms associated with the current app state (e.g., included in, or generated based on, the indication 132) match terms of ASI (and/or an app state ID) of an app state record identified using the search query 130. As another example, a current app state scoring feature may include any parameters that indicate whether the categories associated with the current app state (e.g., included in, or generated based on, the indication 132) match one or more categories indicated by an app state record identified using the search query 130. As still another example, a current app state scoring feature may include any parameters that indicate whether the entities associated with the current app state (e.g., included in, or generated based on, the indication 132) match one or more entities indicated by an app state record identified using the search query 130. In general, the set processing module 142 may generate a result score for an app state record using the record, query, record-query, current app state, and/or any other scoring features.
In some examples, to generate the result scores for the app state records of the consideration set, the set processing module 142 may include one or more machine-learned models (e.g., a supervised learning model, for example, including regression) configured to receive one or more of the record, query, record-query, and/or current app state scoring features. For example, the set processing module 142 may pair the search query 130 with each app state record and calculate a vector of features for each (query, record) pair. The vector of features may include one or more record, query, record-query, and/or current app state scoring features. The set processing module 142 may then input the vector of features into a machine-learned relevance (MLR) model to calculate a result score for the app state record (e.g., simultaneously based on the features). In some examples, the MLR model may include a set of (e.g., gradient-boosted) decision trees. In other examples, the MLR model may be trained by a simple form of logistic regression. In still other examples, the machine-learned task described herein can be framed as a semi-supervised learning task, where a minority of training data is labeled with human-curated result scores and the rest of the data is used without such labels.
As described herein, the result scores associated with the app state records (e.g., the AMs included therein) may be used in various different ways. In some examples, the result scores may be used to rank (e.g., order) the AMs in a list. In these examples, a higher result score may indicate that the corresponding AM (e.g., an AAM specifying a state of a native app) is more relevant to the user than an AM (e.g., an AAM specifying a state of another native app) having a smaller result score. In examples where the search results 134 are displayed as a list of user selectable links on the user device 102, the links including AMs associated with larger result scores may be listed closer to the top of the list (e.g., near the top of the screen). In these examples, links including AMs having lower result scores may be located farther down the list (e.g., off screen) and may be accessed by scrolling down the screen of the user device 102.
As another example, as shown in
In the examples described above, the search system 100 (e.g., the set generation module 140 and/or the set processing module 142) may identify and/or rank one or more app state records included in the search data store 112 based on one or more matches between the terms, categories, and/or entities associated with the current app state and information included in the records (e.g., ASI, app state IDs, indications of categories, and/or indications of entities associated with the records and/or the app states specified by the records). As one example, the set generation module 140 may identify the app state records based on (e.g., text) matches between the terms associated with the current app state and terms of ASI and/or an app state ID included in each record. As another example, the set processing module 142 may generate result scores for (e.g., rank) the app state records based on how well the terms associated with the current app state match the terms of the ASI and/or the app state ID included in each record. As another example, the set generation module 140 may identify the app state records based on (e.g., text) matches between the categories and/or entities associated with the current app state and indications of categories and/or entities included in each record. As still another example, the set processing module 142 may generate the result scores for the app state records based on whether the categories and/or entities associated with the current app state match the indications of categories and/or entities included in each record.
In still other examples, the app state determination system 108 may receive the indication of the current app state 132 from the user device 102 and generate one or more other scoring features using the indication 132 (e.g., using one or more other aspects of the current app state, such as image data, formatting data, and/or other information associated with the state), and transmit an indication of the scoring features to the search system 100. In these examples, the search system 100 (e.g., the set generation module 140) may identify one or more app state records included in the search data 112 store based on the search query 130 and generate one or more results scores for the identified records using the other scoring features.
In other examples, the app state determination system 108 may receive the indication of the current app state 132 from the user device 102 and forward the indication 132 to the search system 100 (e.g., the set generation module 140 and/or the set processing module 142) without modifying the indication 132 or generating additional data based on the indication 132. In still other examples, the search system 100 may receive the indication 132 from the user device 102 and identify and/or rank one or more app state records included in the search data store 112 based on the indication 132 without the use of the app state determination system 108.
In some examples, the app state record 500A of
In block 606A, the search system 100 (e.g., the set generation module 140) may identify a consideration set of one or more app state records included in the search data store 112 based on the (e.g., analyzed) search query 130 and based on the indication of the current app state 132. As described herein, each identified app state record may specify an app state of a native app. In particular, each identified app state record may include an AAM that references a native app and indicates one or more operations for the app to perform, and ASI that describes an app state of the app after the app has performed the operations. For example, the search system 100 may identify the app state records based on matches between terms of the search query 130 and terms of information (e.g., ASI and/or app state IDs) included in the records. As described with reference to
To identify the app state records of the consideration set based on the indication of the current app state 132, the search system 100 (e.g., the set generation module 140) may use any of a variety of techniques. As one example, with reference to
As another example, with reference to
As another example, with reference to
Referring back to
With reference to
As one example, with reference to
As another example, with reference to
As another example, with reference to
Blocks 610E-616E are analogous to blocks 610A-616A of the method 600A.
In these examples, the indication of the current app state 132 may include one or more of textual data, numerical data, and machine-readable (e.g., binary) data that identifies and/or describes the state in some manner. For example, the indication 132 may include any of an app state ID of an app state record associated with the current app state (e.g., represented using textual and/or numerical data), a human-readable textual description of the state, and a machine-readable representation of the state. The search system 100 (e.g., the set processing module 142) may generate the result scores for the app state records of the consideration set based on one or more scoring features (e.g., record scoring features, query scoring features, and record-query scoring features) including a scoring feature associated with the current app state (e.g., a current app state scoring feature), as described herein. In other words, the search system 100 may generate the result scores for the app state records using the indication 132 (e.g., the textual, numerical, and/or machine-readable data) as a scoring feature, alone or in combination with one or more other scoring features. As also described herein, the search system 100 (e.g., the set processing module 142) may include one or more MLR models (e.g., supervised learning models) configured to receive the one or more scoring features, including the scoring feature associated with the current app state, and generate the result scores for the app state records of the consideration set using the scoring features. For example, as described herein, the search system 100 may pair the indication of the current app state 132 with each app state record of the consideration set and compute a vector of features for each (current app state, record) pair. The search system 100 may then input the vector of features into an MLR model to compute a result score for the corresponding app state record. As also described herein, the MLR model included in the search system 100 (e.g., the set processing module 142) may be created using training data (e.g., search queries 130, indications of current app states 132, and search results 134), some or all of which may be labeled with human-curated result scores. In this manner, the search system 100 may process (e.g., score) the app state records of the consideration set using an MLR model that takes into consideration the current app state as a scoring feature.
In additional examples (not shown), one or more terms (e.g., text) of the search query 130 may indicate or suggest the manner in which the user may interact with a native app (e.g., the native app associated with the current app state). Specifically, the search query 130 may indicate or suggest a function sought by the user by specifying the query 130. In these examples, the search system 100 (e.g., the set generation module 140) may identify the app state records of the consideration set based on these indications or suggestions. For example, the search system 100 may identify the consideration set of app state records based on matches between terms of the search query 130 that indicate or suggest a specific function desired by the user and terms (e.g., of ASI and/or app state IDs) that describe one or more functions associated with each record. In a specific example where the search query 130 includes the terms “random restaurant,” the search system 100 may identify an app state record included in the search data store 112 that specifies an app state of a native app that provides a function called “ShowRandomRestaurant” (e.g., via a “Show me a random restaurant” user selectable GUI element). In this manner, the search system 100 may identify the app state records of the consideration set using function matching between the search query 130 and the records.
In block 706A, the user device 102 may transmit the search query 130 and the indication of the current app state 132 to the search system 100. For example, the user device 100 may transmit the search query 130 and indication 132 jointly (e.g., as part of the query wrapper), or separately. In some examples, the indication 132 may include textual, numerical, and/or machine-readable (e.g., binary) data that specifies (e.g., describes) the current app state. As a specific example, the indication 132 may include an app state ID that identifies an app state record included in the search data store 112 that specifies the current app state. In other examples, the indication 132 may include an AAM that specifies the current app state (e.g., references the corresponding native app and indicates one or more operations for the app to perform, which sets the app into the state). In a specific example where the user device 102 displays a GUI associated with an entry in Yelp for the restaurant “Amarin Thai Cuisine” located in Mountain View, Calif., the indication 132 may include the string “yelp:///biz/amarin-thai-cuisine-mountain-view.” The user device 102 may generate the AAM as part of determining the current app state (e.g., by interacting with the corresponding native app and/or the OS 120) or retrieve the AAM (e.g., from the corresponding app state record included in the search data store 112) after determining the current app state.
In some examples, the current app state may not have a corresponding AAM. For example, the native app associated with the current app state may not be configured to receive AAMs that specify app states of the app (e.g., that reference the app and indicate operations for the app to perform, which sets the app into the states). In these examples, the indication 132 may include one or more operations that, when performed by the native app, set the app into the current app state. For example, the operations may be represented as one or more text strings (e.g., instructions, or code) or in another form (e.g., using a machine-readable representation). In some examples, the operations may correspond to one or more user interactions with (e.g., user inputs to) the native app that led a user from a main app state (e.g., a home page) of the app to the current app state. In these examples, the user device 102 may determine the operations by recording (e.g., monitoring) the user interactions as they occur at the device 102.
In additional examples, the indication 132 may include a compressed version of the textual, numerical, and machine-readable data, AAMs, and/or operations described herein. In these examples, the search system 100 may receive the indication 132 and decompress (e.g., decode) the compressed information described therein to determine the current app state. In other examples, the indication 132 may include other data associated with the current app state that indicates the state to the search system 100 in some manner.
As described herein, upon receiving the indication of the current app state 132 from the user device 102, the search system 100 may determine the current app state based on the indication 132 (and, e.g., other data). In some examples, the search system 100 may directly determine the current app state using textual, numerical, and/or machine-readable data (e.g., an app state ID) included in the indication 132. In other examples, the search system 100 may determine the current app state based on an AAM included in the indication 132 (e.g., by setting a version of the native app associated with the state into the state using the AAM). In still other examples, the search system 100 may determine the current app state based on one or more operations included in the indication 132 (e.g., by inputting the operations into a version of the native app associated with the state until the app is set into the state). In some examples, the search system 100 may use the AAM and/or the operations as inputs into a version (e.g., a copy) of the native app associated with the current app state, or a virtual machine (VM) implementation (e.g., a model, or an emulation) of the app, either of which may execute on the system 100 and/or at a remote computing resource accessible by the system 100. Like the native app itself, these versions of the app may be capable of receiving the AAM as part of the indication 132 and being set into the current app state as a result. Additionally, or alternatively, these native app versions may be capable of receiving the operations as part of the indication 132 and recreating (e.g., “playing back”) the corresponding user interactions to elucidate (e.g., via introspection) the current app state. In other examples, the search system 100 may use the AAM and/or the operations as inputs into a state machine present on the system 100 and/or at a remote computing resource accessible by the system 100. The state machine may represent (e.g., include therein) one or more popular app states of native apps and/or user interactions in the apps (e.g., transitions among the app states). In these examples, the state machine may be capable of receiving the AAM and/or the operations as part of the indication 132 and being set into a state that corresponds to the current app state as a result.
In the example of
As one example, the search system 100 may have generated the search results 134 by identifying one or more app state records included in the search state data store 112 based on matches between terms (e.g., text) associated with the current app state, as specified by the indication 132, and terms of information (e.g., ASI and/or app state IDs) included in each record. For example, the search system 100 may have identified one or more app state records that include some or all of the text (e.g., the text string “NFL”) displayed as part of the current app state. As another example, the search system 100 may have generated the search results 134 by identifying one or more app state records included in the search state data store 112 based on matches between one or more categories (e.g., types) associated with the current app state (e.g., with the native app associated with the state), as specified by the indication 132, and one or more categories associated with (e.g., specified in) each record. For example, the search system 100 may have initially determined the categories associated with the current app state based on the indication 132 (e.g., directly from the indication 132, or based on matches between terms associated with the current app state, as specified by the indication 132, and terms included in a category data store). The search system 100 may have then identified one or more app state records included in the search state data store 112 that are each associated with (e.g., that each specify) one or more of the determined categories (e.g., “football,” or “NFL”). As described herein, a category associated with a current app state may be a category that is associated with the state, specifically, or with a native app associated with the state, generally. As an example, a category associated with a current app state of NFL Now, such as an entry in NFL Now for “The New York Giants,” may be “The New York Giants,” specifically, or “NFL” or “football,” generally. Similarly, a category associated with (e.g., specified in) each of one or more of the identified app state records may be a category that is associated with the corresponding app state, specifically, or with the corresponding native app, generally. As still another example, the search system 100 may have generated the search results 134 by identifying one or more app state records included in the search state data store 112 based on matches between one or more entities (e.g., names of businesses, services, franchises, films, musical artists, and/or persons) associated with the current app state, as specified by the indication 132, and one or more entities associated with (e.g., specified in) each record. For example, the search system 100 may have initially identified the entities associated with the current app state based on the indication 132 (e.g., based on matches between terms associated with the current app state, as specified by the indication 132, and terms included in an entity data store). The search system 100 may have then identified one or more app state records included in the search state data store 112 that are each associated with (e.g., that each specify) one or more of the identified entities (e.g., “NFL”).
In still other examples, the search system 100 may have generated the search results 134 by further generating one or more result scores for the identified app state records using the indication of the current app state 132 as a scoring feature. For example, the search system 100 may have generated a result score for each identified app state record using various aspects (e.g., terms, categories, and/or entities) of the current app state, as specified by the indication 132, as a scoring feature input into an MLR model (e.g., also using any of the search query 130 and the record as one or more additional scoring features input into the same MLR model).
In the above-described examples, the search system 100 may have also identified the app state records in the search state data store 112 based on matches between terms (e.g., text) of the search query 130 and terms of information (e.g., ASI and/or app state IDs) included in each record. The search system 100 may have then generated one or more result scores for the identified app state records, as described herein, selected one or more (e.g., a subset) of the records based on the corresponding one or more result scores, and generated the search results 134 based on the selected records. As shown in
In some examples, the GUI element 115A and/or the GUI 115B of the search app 124 may be embedded in the native app (e.g., NFL Mobile) executing on the user device 102. In other words, in these examples, one or more GUIs associated with the search app 124 may be a part of the native app executing on the user device 102, rather than a part of a stand-alone search app 124. In these examples, instead of invoking a separate search app 124 while viewing the current app state on the user device 102, the user may interact with (e.g., enter search queries 130 into and submit the queries 130 via) a GUI that is displayed directly in the current app state.
In additional examples, the search system 100 may generate additional search results that include content that is not associated with app states of native apps (e.g., content related to apps, web sites, documents, and/or media files). In these examples, the search system 100 may identify one or more records (e.g., app records, or other data structures) stored in a data store that include the content based on the search query 130, in a similar manner as described above.
In further examples, the current app state may correspond to an app state (e.g., a web page) of a web-based app executing on the user device 102 (e.g., via the web browser app 122). In these examples, the search system 100 may generate search results that match both the search query 130, as described herein, and various aspects of the current app state (e.g., associated terms, categories, and/or entities), in a similar manner as described herein with reference to using an indication of a current app state 132 of a native app to generate search results 134.
The modules and data stores included in the search system 100 and app state determination system 108 represent features that may be included in these systems 100, 108 as they are described in the present disclosure. For example, the search module 110, search data store 112, and result generation module 114 may represent features included in the search system 100. Similarly, the app state determination module 116 and app state data store 118 may represent features included in the app state determination system 108. The modules and data stores described herein may be embodied by electronic hardware, software, and/or firmware components. Depiction of different features as separate modules and data stores does not necessarily imply whether the modules and data stores are embodied by common or separate electronic hardware, software, and/or firmware components. In some implementations, the features associated with the one or more modules and data stores depicted herein may be realized by common or separate electronic hardware, software, and/or firmware components.
The modules and data stores may be embodied by electronic hardware, software, and/or firmware components including, but not limited to, one or more processing units, memory components, input/output (I/O) components, and interconnect components. The interconnect components may be configured to provide communication between the processing units, memory components, and I/O components. For example, the interconnect components may include one or more buses configured to transfer data between electronic components. The interconnect components may also include control circuits (e.g., a memory controller and/or an I/O controller) configured to control communication between electronic components.
The processing units may include one or more central processing units (CPUs), graphics processing units (GPUs), digital signal processing units (DSPs), or other processing units. The processing units may be configured to communicate with the memory components and I/O components. For example, the processing units may be configured to communicate with the memory components and I/O components via the interconnect components.
A memory component, or memory, may include any volatile or non-volatile media. For example, the memory may include electrical media, magnetic media, and/or optical media, such as a random access memory (RAM), read-only memory (ROM), non-volatile RAM (NVRAM), electrically-erasable programmable ROM (EEPROM), Flash memory, solid state drives (SSDs), hard disk drives (HDDs), magnetic tape drives, optical storage technology (e.g., compact disc, digital versatile disc, and/or Blu-ray disc), and/or any other memory components.
The memory components may include (e.g., store) the data described herein. For example, the memory components may store the data included in the app state records of the search data store 112 and/or the data included in the app state data store 118. The memory components may also include instructions that may be executed by the processing units. For example, the memory components may include computer-readable instructions that, when executed by the processing units, cause the units to perform the various functions attributed to the modules and data stores described herein.
The I/O components may refer to electronic hardware, software, and/or firmware that provide communication with a variety of different devices. For example, the I/O components may provide communication between other devices and the processing units and memory components. In some examples, the I/O components may be configured to communicate with a computer network. For example, the I/O components may be configured to exchange data over a computer network using a variety of different physical connections, wireless connections, and protocols. The I/O components may include network interface components (e.g., a network interface controller), repeaters, network bridges, network switches, routers, and firewalls. In some examples, the I/O components may include hardware, software, and/or firmware configured to communicate with various human interface devices, including display screens, keyboards, pointer devices (e.g., a mouse), touchscreens, speakers, and microphones. In other examples, the I/O components may include hardware, software, and/or firmware configured to communicate with additional devices, such as external memory (e.g., external HDDs).
In some implementations, the search system 100 and/or app state determination system 108 may be a system of one or more computing devices (e.g., a computer search system) configured to implement the techniques described herein. Put another way, the features attributed to the modules and data stores described herein may be implemented by one or more computing devices. Each computing device may include any combination of electronic hardware, software, and/or firmware described herein. For example, each computing device may include any combination of the processing units, memory components, I/O components, and interconnect components described herein. The computing devices may also include various human interface devices, including display screens, keyboards, pointing devices (e.g., a mouse), touchscreens, speakers, and microphones. The computing devices may also be configured to communicate with additional devices, such as external memory (e.g., external HDDs).
The computing devices of the search system 100 and/or app state determination system 108 may be configured to communicate with the network 106. The computing devices may also be configured to communicate with one another via a computer network. In some examples, the computing devices may include one or more server computing devices configured to communicate with the user device(s) 102 (e.g., receive search queries 130 and indications of current app states 132, and transmit search results 134), gather data from the data source(s) 104, index the data, store the data, and store other documents. The computing devices may reside in a single machine or in multiple machines at a single geographic location, or be distributed across a number of geographic locations.
Additionally, the various implementations of the search system 100 and app state determination system 108 described herein (e.g., using one or more computing devices that include one or more processing units, memory components, I/O components, and interconnect components) are equally applicable to any of the user device(s) 102 and components thereof.
This application claims the benefit of U.S. Provisional Application No. 62/183,527 filed Jun. 23, 2015, which is hereby incorporated by reference in its entirety.
| Number | Date | Country | |
|---|---|---|---|
| 62183527 | Jun 2015 | US |
