Generating Software Application Search Results Using Application Connections

Information

  • Patent Application
  • 20170169022
  • Publication Number
    20170169022
  • Date Filed
    December 11, 2015
    9 years ago
  • Date Published
    June 15, 2017
    7 years ago
Abstract
Techniques include, for an application (app) record specifying a software app and including an app download address (ADA) for downloading the app, determining connections (e.g., links to and from other resources) associated with the app and generating a quality value indicating the quality of the app based on the connections. In some examples, the techniques further include receiving a search query from a user device and identifying the record based on the query and the quality value (e.g., using the value as a boost factor within Lucene® information retrieval software). Additionally, or alternatively, the techniques include identifying the record based on the search query, generating a result score for the record based on the quality value, and selecting the record from among other records based on the score. The techniques also include selecting the ADA from the record and transmitting the ADA to the user device as a search result.
Description
TECHNICAL FIELD

This disclosure generally relates to the field of software applications (apps), and more particularly to techniques for performing searches for apps.


BACKGROUND

In recent years, the use of computers, smartphones, and other Internet-connected computing devices has grown significantly. Correspondingly, the number of software applications (apps) available for such computing devices has also grown. Today, many diverse software apps can be accessed on a number of different computing devices, including, but not limited to, smartphones, personal computers, automobiles, and televisions. These software apps can include business driven apps, games, educational apps, news apps, shopping apps, messaging apps, media streaming apps, and social networking apps, as some examples. Because of the large number of software apps available today and the wide range of functionality they provide, computing device users often require the ability to search for and access specific software apps.


SUMMARY

In one example, a method includes, for each of a plurality of application (app) records each specifying a native app and including an app download address (ADA) for downloading the app, determining one or more connections associated with the app and generating a quality value indicating a degree of quality associated with the app based on the connections. The method further includes receiving a search query from a user device and identifying one or more of the plurality of app records based on the query and the quality value associated with each identified record. The method also includes selecting the one or more ADAs from the identified app records and transmitting the ADAs to the user device.


In another example, a method includes, for each of a plurality of app records each specifying a native app and including an ADA for downloading the app, determining one or more connections associated with the app and generating a quality value indicating a degree of quality associated with the app based on the connections. The method further includes receiving a search query from a user device and identifying one or more of the plurality of app records based on the query. The method still further includes, for each of the identified app records, generating a result score based on the quality value associated with the record. The method also includes selecting one or more of the identified app records based on the result score associated with each selected record, selecting the one or more ADAs from the selected records, and transmitting the ADAs to the user device.


In another example, a system includes one or more computing devices configured to, for each of a plurality of app records each specifying a native app and including an ADA for downloading the app, determine one or more connections associated with the app and generate a quality value indicating a degree of quality associated with the app based on the connections. The devices are further configured to receive a search query from a user device and identify one or more of the plurality of app records based on the query and the quality value associated with each identified record. The devices are also configured to select the one or more ADAs from the identified app records and transmit the ADAs to the user device.


In another example, a system includes one or more computing devices configured to, for each of a plurality of app records each specifying a native app and including an ADA for downloading the app, determine one or more connections associated with the app and generate a quality value indicating a degree of quality associated with the app based on the connections. The devices are further configured to receive a search query from a user device and identify one or more of the plurality of app records based on the query. The devices are still further configured to, for each of the identified app records, generate a result score based on the quality value associated with the record. The devices are also configured to select one or more of the identified app records based on the result score associated with each selected record, select the one or more ADAs from the selected records, and transmit the ADAs to the user device.


In another example, a non-transitory computer-readable storage medium includes instructions that cause one or more computing devices to, for each of a plurality of app records each specifying a native app and including an ADA for downloading the app, determine one or more connections associated with the app and generate a quality value indicating a degree of quality associated with the app based on the connections. The instructions further cause the devices to receive a search query from a user device and identify one or more of the plurality of app records based on the query and the quality value associated with each identified record. The instructions also cause the devices to select the one or more ADAs from the identified app records and transmit the ADAs to the user device.


In another example, a non-transitory computer-readable storage medium includes instructions that cause one or more computing devices to, for each of a plurality of app records each specifying a native app and including an ADA for downloading the app, determine one or more connections associated with the app and generate a quality value indicating a degree of quality associated with the app based on the connections. The instructions further cause the devices to receive a search query from a user device and identify one or more of the plurality of app records based on the query. The instructions still further cause the computing devices to, for each of the identified app records, generate a result score based on the quality value associated with the record. The instructions also cause the devices to select one or more of the identified app records based on the result score associated with each selected record, select the one or more ADAs from the selected records, and transmit the ADAs to the user device.





BRIEF DESCRIPTION OF DRAWINGS

The details of one or more examples are set forth in the accompanying drawings and the description below. Other features, objects, and advantages will be apparent from the description and drawings, and from the claims.



FIG. 1 illustrates an example environment that includes a search system, an application (app) connection analysis system, one or more data sources, and one or more user devices that communicate via a network.



FIG. 2 illustrates an example user device in communication with an example search system and an example app connection analysis system.



FIG. 3A is a functional block diagram of an example search system.



FIG. 3B is a functional block diagram of an example search module.



FIGS. 4A-4B illustrate example app records.



FIG. 5A illustrates another example user device in communication with an example search system and an example app connection analysis system.



FIG. 5B illustrates an example quality value generation module.



FIG. 5C illustrates an example app connection determination module.



FIGS. 6A-6D are conceptual diagrams of example connections associated with native apps and an example app connection graph.



FIGS. 7A-7C depict example graphical user interfaces (GUIs) that may be generated on a user device according to the present disclosure.



FIGS. 8-11 are flow diagrams that illustrate example methods for generating search results based on a search query and app connection data using a search system.



FIG. 12 is a flow diagram that illustrates an example method for generating search results based on a search query and app connection data using a user device.





DETAILED DESCRIPTION

The figures and the following description relate to example implementations by way of illustration only. It should be noted that from the following discussion, alternative implementations of the structures and methods disclosed herein will be readily recognized as viable alternatives that may be employed without departing from the scope of this disclosure.


The present disclosure generally relates to the field of search, and, more particularly, to techniques for generating and displaying search results that specify software applications (apps) based on connections (e.g., links to and from various resources) associated with the apps. In some implementations, the techniques of this disclosure may be used to enable users to perform searches for software apps within digital distribution platforms (e.g., Google Play by Google Inc., the App Store by Apple Inc., Amazon Appstore by Amazon Inc., and Windows Phone Store by Microsoft Corporation) configured to distribute software apps to user devices. In general, the techniques described herein include initially identifying one or more connections (e.g. links) associated with each of one or more software apps included in (e.g., downloadable from) a digital distribution platform. The techniques further include, for each of the software apps, generating a value of a quality metric (e.g., a “quality value”) that indicates a degree of quality associated with the app based on the connections identified for the app. The techniques also include, in response to receiving a search query from a user device, generating search results that specify one or more of the software apps included in the digital distribution platform based on the query and the quality value associated with each app. Using the techniques described herein may, in some examples, improve search result relevance and enhance user experience.


According to the disclosed techniques, a user of a user device (e.g., a mobile computing device) may input a search query (e.g., a text string) into a search field of a search app executing on the device. The user may then cause the user device (e.g., the search app) to transmit the search query to a software app search system (e.g., a digital distribution platform). The search system may receive the search query from the user device, generate search results that each specify a software app using the query, and transmit the results to the device. To generate the search results, the search system may initially generate app connection data that indicates connections associated with software apps specified by app records included in the system. In particular, the search system may, for each app record, determine one or more connections associated with the software app specified by the record. The search system may further generate a quality value indicating a degree of quality associated with the software app based on the determined connections. The search system may then identify one or more of the app records included in the system based on the search query received from the user device and based on the quality value associated with each identified record. For example, the search system may identify each app record by using the quality value associated with the record as a so-called “boost factor” within Lucene® information retrieval software developed by the Apache Software Foundation (hereinafter, “Lucene”), or as a similar input within other information retrieval software. Alternatively, the search system may identify one or more of the app records included in the system based on the search query, in a similar manner as described above. In this example, the search system may further rank (e.g., arrange in an order) the identified app records based on the quality values associated with the records and select one or more (e.g., highest-ranking ones) of the ranked records for further consideration.


The search system may generate the search results using the app records identified and/or ranked based on the quality values associated with the records. Specifically, the search system may select one or more application download addresses (ADAs) from the app records and generate the search results to include the ADAs and, e.g., other data associated with the records. The search system may then transmit the search results, including the ADAs and, e.g., the other data, to the user device. The user device may receive the search results from the search system and display the results to the user (e.g., as one or more user selectable links). In some examples, the user device may use the other data received from the search system with the search results to display the results to the user (e.g., to generate and/or arrange the user selectable links).


The user may select one or more of the search results (e.g., one or more of the associated user selectable links) on the user device. Upon the user selecting a particular search result (e.g., an associated user selectable link), the user device may download and install the software app specified by the selected result (e.g., by the associated user selectable link). For example, the user device may download the software app from a digital distribution platform using an ADA included in the selected result (e.g., in the associated user selectable link). In some examples, after downloading and installing the software app, the user device may launch the app on the device. Upon the user device launching the software app, the user may interact with the app on the device (e.g., preview and/or perform a function provided by the app).


In this manner, the techniques described herein may improve search result relevance and enhance user experience. As one example, by identifying app records specifying software apps that both match the search query and have relatively large quality values, the search results may be more relevant (e.g., useful) to the user than search results generated using the query alone. Additionally, by ranking app records identified using the search query such that records that specify software apps having relatively large quality values are ranked higher than records that specify software apps having relatively small quality values, those of the search results that are more relevant (e.g., useful) to the user may be displayed earlier than other results. As another example, by limiting the search results to those that specify software apps having relatively large quality values, or by ranking the results based on the corresponding quality values as described above, the user may more easily (e.g., quickly) access those of the software apps specified by the results having the highest quality, thereby enhancing the user's experience.



FIG. 1 illustrates an example environment that includes a search system 100, an app connection analysis system 108 (hereinafter, “analysis system 108”), one or more data sources 104, and one or more user devices 102 that communicate via a network 106. The network 106 through which the above-described systems and devices communicate may include any type of network, such as a local area network (LAN), a wide area network (WAN), and/or the Internet. As shown in FIG. 1, the search system 100 includes an app search module 110 (hereinafter, “search module 110”), an app search data store 112 (hereinafter, “search data store 112”), and an app result generation module 114 (hereinafter, “result generation module 114”), which are described in greater detail herein. As also shown, the analysis system 108 includes an app connection determination module 116 (hereinafter, “connection determination module 116”), an app connection data store 118 (hereinafter, “connection data store 118”), and an app connection record module 120 (hereinafter, “connection record module 120”), which are also described in greater detail herein. In some examples, the analysis system 108 may be a part of the search system 100, a part of another system or device, or a stand-alone system or device.


In this disclosure, a software app may refer to computer software that causes a computing device to perform a task. In some examples, a software app may be referred to as an “app,” or a “program.” Example apps include word processing apps, spreadsheet apps, messaging apps, media streaming apps, social networking apps, and games. Apps can be executed on a variety of different computing devices. For example, apps can be executed on mobile computing devices, such as smartphones, tablets, and wearable computing devices (e.g., smart watches, fitness bands, and headsets, such as smart glasses). Apps can also be executed on other types of computing devices having other form factors, such as laptop computers, desktop computers, and other consumer electronic devices (e.g., smart home appliances, home networking devices, and home automation devices). In some examples, apps may be installed on a computing device prior to a user purchasing the device. In other examples, the user may download and install apps on the computing device after purchasing the device. A native app, as used herein, may refer to an app that is installed and executed on a user device. A web-based app, in turn, may refer to an app that is accessible from a user device via a web browser app.


In some examples, the functionality of an app may be accessed on the computing device on which the app is installed. Additionally, or alternatively, the functionality of an app may be accessed via a remote computing device. In further examples, all of an app's functionality may be included on the computing device on which the app is installed. Such apps may function without communication with other computing devices (e.g., via the Internet). In additional examples, an app installed on a computing device may access information from other remote computing devices during operation. For example, a weather app installed on a computing device may access the latest weather information via the Internet and display the accessed information to the user. In still other examples, an app (e.g., a web-based app) may be partially executed by a user's computing device and partially executed by a remote computing device. For example, a web-based app may be executed, at least in part, by a web server and accessed by a web browser app of a user's computing device. Example web-based apps include web-based email sites, online auction sites, online retail sites, and other websites.


The search system 100 of this disclosure may be implemented as part of a digital distribution platform (e.g., a so-called “app store”) configured to distribute native apps to the user device(s) 102. In general, the search system 100 may perform searches for native apps included in (e.g., downloadable from) the system 100 based on user-specified search queries (e.g., text strings) received from the user device(s) 102 and connections (e.g., links to and from other resources) associated with the apps. According to the techniques of this disclosure, the search system 100 may be configured to receive a search query from one of the user device(s) 102 via the network 106. For example, the user device 102 may receive the search query from a user of the device 102 and transmit the query to the search system 100 via the network 106. The search system 100 may be further configured to, upon receiving the search query from the user device 102, perform a search for one or more native apps included in the system 100 based on the query and based on app connection data generated using the analysis system 108. In some examples, the app connection data may indicate one or more connections associated with each of one or more native apps included in the search system 100. Additionally, or alternatively, the app connection data may include a value of a quality metric, which may be referred to herein as a “quality value,” associated with each of the native apps that indicates a degree of quality associated with the app. For example, the analysis system 108 may generate the quality value for each of the native apps based at least in part on the connections associated with the app. Based on performing the search, the search system 100 may identify one or more native apps included in the system 100 and generate search results that specify the apps (e.g., that enable a user device 102 to download and install the apps). The search system 100 may also be configured to transmit the search results to the user device 102 via the network 106. The user device 102 may receive the search results from the search system 100 and display the results to the user as one or more user selectable links including image and/or text data. The user may select (e.g., touch, or “click on”) any of the user selectable links on the user device 102. In response to the user selecting a particular user selectable link, the user device 102 may download and install (and, e.g., launch) a native app specified by the selected link (e.g., by the associated search result).


In the example of FIG. 1, the search system 100 generates the search results based on the search query, the app connection data, and information included in one or more app records stored in the search data store 112. In this example, each app record may specify a native app. The information included in the app records may include one or more access mechanisms (AMs) that enable the user device(s) 102 to access (e.g., download and install) the native apps specified by the records. For example, each app record may include an ADA (e.g., an alphanumeric string, such as a download link, binary data, or another data structure) that indicates a location where the native app specified by the record may be downloaded. In some examples, the ADA may specify the search system 100 as the location where the native app may be downloaded. In other examples, the ADA may specify another location where the native app may be downloaded (e.g., a website associated with a developer of the app). The search system 100 identifies one or more of the app records based on the search query and app connection data, selects the AMs (e.g., ADAs) from the identified records, and transmits the selected AMs to the user device 102 as the search results. The user device 102 displays the search results to a user of the device 102 as one or more user selectable links that include the AMs. The information included in the app records may also include app attributes (AAs) (e.g., text), such as various features and metadata, and other information (e.g., app names/IDs) associated with the records, which the search system 100 may use to identify the records in the search data store 112 as described above. Example app records are described with reference to FIGS. 4A-4B. The search data store 112, including one or more app records, may include one or more databases, (e.g., inverted) indices, tables, files, or other data structures that may be used to implement the techniques of the present disclosure. In some examples, the search data store 112 may be included in one or more storage devices.


For example, to generate the search results, the search module 110 may identify one or more app records included in the search data store 112 based on the search query and the app connection data. Initially, the search module 110 may analyze the search query. The search module 110 may then identify one or more app records included in the search data store 112 based on the (e.g., analyzed) search query and the app connection data. For example, the search module 110 may identify the app records based on (e.g., text) matches between terms of the search query and terms of information included in the records. In some examples, the search module 110 may further identify the app records based on the quality value associated with the native app specified by each record, as indicated by the app connection data. For example, the search module 110 may identify each app record by using the quality value associated with the native app specified by the record as a boost factor as part of Lucene. Alternatively, the search module 110 may identify each app record by determining that the quality value associated with the native app specified by the record is greater than a threshold value, as described in greater detail herein. The search module 110 may then process (e.g., rank and select a subset of) the identified app records. Specifically, the search module 110 may generate a result score for each of the identified app records based on how well information included in the record matches the search query. In some examples, the search module 110 may further generate the result score for each identified app record based on one or more connections associated with the native app specified by the record and/or the quality value associated with the app, as further indicated by the app connection data. The search module 110 may then select one or more of the identified app records that have the highest one or more result scores and transmit indications of (e.g., app names/IDs associated with) the selected records to the result generation module 114.


The result generation module 114 may identify the app records selected by the search module 110 in the search data store 112 using the received indications (e.g., app names/IDs). The result generation module 114 may then select one or more AMs (e.g., ADAs) from the identified app records and transmit the AMs to the user device 102 as search results. In some examples, the result generation module 114 may transmit additional data to the user device 102. For example, as described herein, the search module 110 may generate result scores for the app records from which the AMs are selected (e.g., using various scoring features associated with the search query, the records, and/or the app connection data used to identify and/or rank the records). As such, each AM may be associated with a result score that indicates a rank of the AM relative to the other AMs. In these examples, the result generation module 114 may transmit the result scores associated with the AMs to the user device 102 along with the AMs. In other examples, the result generation module 114 may transmit link data and/or other information associated with the AMs (e.g., with the corresponding app records) to the user device 102.


The search query may include text, numbers, and/or symbols (e.g., punctuation) entered into the user device 102 by the user. For example, the user may have entered the search query into a search field, or “box,” of a search app executing on the user device 102. The user may have entered the search query into the search app using a touchscreen keypad, a mechanical keypad, and/or via speech recognition techniques and later caused the app to transmit the query to the search system 100. In some examples, the user may have entered the search query into the search app using various autosuggest (e.g., so-called “autocomplete”) techniques. Additionally, or alternatively, the search query may be generated or selected based on an interaction between the user and the user device 102, such as, e.g., in response to the user selecting a link that corresponds to a predefined search query within an app executing on the device 102. In some examples, the search app may be a native app dedicated to search, or a more general app, such as a web browser app. The app connection data, in turn, may include text, numbers, symbols, and/or machine-readable (e.g., binary) data used by the analysis system 108 to represent one or more connections associated with a native app and/or a quality value indicating a degree of quality associated with the app and generated based at least in part on the connections.


In some examples, the user device 102 may transmit additional data to the search system 100 along with the search query. The search query and the additional data may be referred to herein as a “query wrapper.” The additional data may include geo-location data associated with the user device 102, platform data for the device 102 (e.g., a type and/or a version, an operating system (OS), and/or a web browser app associated with the device 102), an identity of the user (e.g., a username), partner specific data, and/or other data (e.g., indications of one or more native apps that are installed on the device 102). The user device 102 may transmit the query wrapper to the search system 100. The search system 100 may receive the query wrapper and use the search query and, e.g., the additional data included in the wrapper, to generate the search results and provide the results to the user device 102.


In other examples, the search system 100 may transmit the search results, including the AMs (e.g., ADAs), to the user device 102 with additional data. For example, the search system 100 may transmit link (e.g., text and/or image) data that the user device 102 may use to generate the user selectable links for the AMs included in the search results. For instance, each user selectable link may include a portion of the link data that the user of the user device 102 may select (e.g., touch, or click on). Each user selectable link may also be associated with one of the AMs included in the search results, such that when the user selects the link, the user device 102 downloads and installs the native app specified by the AM. The link data included in the user selectable link may indicate (e.g., textually and/or graphically) the native app associated with the link. Example user selectable links are illustrated in FIG. 7B.


The user device(s) 102 may be any computing devices capable of providing search queries and, e.g., app interaction data, to the search system 100 (and, e.g., the analysis system 108) and receiving search results from the system 100. The user device(s) 102 may include any of smartphones, and tablet, laptop, and desktop computing devices. The user device(s) 102 may also include any computing devices having other form factors, e.g., those included in vehicles, gaming devices, televisions, or other appliances (e.g., networked home automation devices and home appliances). The user device(s) 102 may use a variety of different operating systems or platforms (e.g., an OS 200, as shown in FIG. 2). In the event the user device 102 is a mobile device, the device 102 may operate using an OS such as ANDROID® by Google Inc., IOS® by Apple Inc., or WINDOWS PHONE® by Microsoft Corporation. In the event the user device 102 is a laptop or desktop computing device, the device 102 may use an OS such as MICROSOFT WINDOWS® by Microsoft Corporation, MAC OS® by Apple Inc., or LINUX® (LINUX is the registered trademark of Linus Torvalds in the U.S. and other countries). In general, the user device(s) 102 may interact with any of the systems 100, 108 using operating systems other than those described herein, whether presently available or developed in the future.


The user device(s) 102 may communicate with the search system 100 (and, e.g., the analysis system 108) via the network 106. In general, the user device(s) 102 may communicate with any of the systems 100, 108 using any app that can transmit search queries and, e.g., app interaction data, to one or more of the systems 100, 108, and receive search results from the search system 100. In some examples, the user device(s) 102 may include an app that is dedicated to interfacing with one or more of the systems 100, 108, such as an app dedicated to searches (e.g., a search app 204, as also shown in FIG. 2). In other examples, the user device(s) 102 may communicate with any of the systems 100, 108 using a more general app, such as a web browser app (e.g., a web browser app 202, as further shown in FIG. 2). An app included on a user device 102 to communicate with one or more of the systems 100, 108 may display a graphical user interface (GUI) including a search field, or box, into which a user may enter search queries. For example, the user may enter the search queries using a touchscreen, a physical keyboard, a speech-to-text program, or another form of user input available on the user device 102. The app may be configured to transmit the search queries to the search system 100 (e.g., in response to user inputs). In some examples, the app may be further configured to determine (e.g., via an app interaction determination module 208, as shown in FIG. 2) interactions (e.g., data exchanges) between native apps that are installed on the user device 102 (e.g., one or more native apps 206, as also shown in FIG. 2) and other resources. In these examples, the app may be configured to transmit app interaction data indicating the interactions to one or more of the systems 100, 108 (e.g., with the search queries, or separately).


In some examples, the user device 102 may use the same (e.g., dedicated, or more general) app to display the search results received from the search system 100 to the user. For example, the user device 102 may display the search results via the GUI used to receive the search queries from the user and transmit the queries to one or more of the systems 100, 108, as described herein. The GUI may display the search results to the user in a variety of different ways, depending on the information transmitted by the search system 100 to the user device 102 as part of the results. As previously described, the search results may include one or more AMs (e.g., ADAs), along with link data, result scores, and/or other information used to generate user selectable links for the AMs. The GUI may display the search results to the user as a list of the user selectable links, including text and/or images. For instance, the text and/or images may include names of native apps referenced by the AMs, descriptions of the apps, and/or images associated with the apps (e.g., app “icons,” or “screenshots”). In additional examples, the GUI may display the search results as the list of the user selectable links arranged under the search field, or box, into which the user has entered a search query. For example, the GUI may arrange the user selectable links by result scores associated with the links, i.e., with the AMs for which the links are generated, or using other logic. In still other examples, the GUI may also group the user selectable links by the associated native app category (e.g., using app category headers). In additional examples, the search system 100 may transmit the search results to the user device 102 via an app programming interface (API). In these examples, the GUI used to display the search results on the user device 102 may be determined (e.g., defined) by a third-party app (e.g., that is associated with the API). For example, the GUI may implement visual (e.g., include text and/or image data), audible (e.g., include a text-to-speech output), and/or any other techniques of presenting the search results to the user on the user device 102. The data source(s) 104 may be any sources of data that the search system 100 may use to generate and/or update the search data store 112. For example, the search system 100 may use the data source(s) 104 to generate and/or update one or more databases, indices, tables, files, or other data structures (e.g., app records) included in the search data store 112. As an example, the search system 100 may generate new app records and/or update existing app records based on data retrieved from the data source(s) 104. For instance, the search system 100 may include one or more modules (not shown) that generate new app records and/or update existing app records based on the data. In some examples, some or all of the data included in the search data store 112 (e.g., one or more app records) may be manually generated by a human operator.


The data source(s) 104 may include a variety of different data providers. For example, the data source(s) 104 may include data from app developers, such as app developer websites and data feeds provided by app developers. The data source(s) 104 may also include operators of digital distribution platforms configured to distribute apps to user devices. The data source(s) 104 may further include other websites, such as websites that include web logs (i.e., blogs), app reviews, or other data related to apps. Additionally, the data source(s) 104 may include social networking sites, such as “FACEBOOK®” by Facebook Inc. (e.g., Facebook posts) and “TWITTER®” by Twitter Inc. (e.g., text from tweets). The data source(s) 104 may also include other types of data sources, which may have various types of content and update rates. In some examples, the search system 100 may retrieve data from the data source(s) 104, including any type of data related to native apps and/or native app functionality. The search system 100 may then generate one or more app records based on the data and store the records in the search data store 112. In other examples, some or all of the data (e.g., AAs) included in the app records of the search data store 112 may be manually generated by a human operator. Additionally, in some examples, the data included in the app records may be updated over time so that the search system 100 provides up-to-date search results in response to user-specified search queries received from the user device(s) 102.



FIG. 2 illustrates an example of one of the user device(s) 102 in communication with the search system 100 and analysis system 108. Specifically, FIG. 2 depicts example interactions and data exchanged among the user device 102, search system 100, and analysis system 108. As shown in FIG. 2, the user device 102 may transmit a query wrapper to the search system 100. The query wrapper may include a search query 210, app interaction data 216, geo-location data, platform data, and/or other data (e.g., an IP address) associated with the user, the user device 102, and/or the query 210. For example, the user may have entered the search query 210 into a search field 212 of a GUI of a search app 204 included on the user device 102. The user may have then caused the search app 204 to submit the search query 210 to the search system 100 (i.e., as part of the query wrapper) by selecting a search button 214 of the GUI. In this example, the app interaction data 216 may indicate one or more interactions between one or more native apps 206 included on the user device 102 and one or more other resources. The user device 102 (e.g., the search app 204) may have generated the app interaction data 216 and submitted the data 216 to the search system 100 (e.g., also as part of the query wrapper). For example, the user device 102 may have generated the app interaction data 216 using an app interaction determination module 208 included on the device 102 (e.g., as part of the search app 204). In some examples, the user device 102 may determine the app interaction data 216 prior to, during (e.g., in response to), or after the user enters and/or submits the search query 210.


Upon receiving the query wrapper from the user device 102, the search system 100 may generate one or more search results 220 based on the search query 210 and app connection data 218 generated by the analysis system 108. For example, the analysis system 108 may generate the app connection data 218 based at least in part on the app interaction data 216 received from the user device 102 (e.g., via the search system 100). To generate the search results 220, the search system 100 may identify one or more app records included in the search data store 112 based on the search query 210 and, e.g., the app connection data 218. The search system 100 may further generate results scores for (e.g., rank) the identified app records, e.g., also based on the app connection data 218. The search system 100 may then select one or more of the identified and ranked app records based on the corresponding results scores, select one or more ADAs from the selected records, and transmit the selected ADAs as the search results 220 to the user device 102 (e.g., along with link data, one or more result scores, and/or other information associated with and/or selected from the records).


In the example of FIG. 2, upon receiving the search results 220 from the search system 100, the user device 102 may display the results 220 to the user as one or more user selectable links. For example, the user device 102 may generate the user selectable links such that each link is associated with (e.g., includes) one of the ADAs included in the search results 220. As described herein, each ADA included in the search results 220 may specify a native app (e.g., reference the app and indicate a location where the app may be downloaded). As a result, when the user selects (e.g., touches, or clicks on) each user selectable link, the user device 102 may download and install the native app specified by the ADA included in the link. Upon downloading and installing the native app, the user device 102 may optionally launch the app (e.g., into a default, or main, state of the app). In some examples, the user device 102 may generate the user selectable links using the link data also included in the search results 220. For example, the link data may include any of text (e.g., describing a name of a native app) and image data (e.g., an app icon, or screenshot associated with the app). In this manner, the link data included in (e.g., used to generate) each user selectable link may describe the native app associated with the link (e.g., specified by the ADA included in the link). The user device 102 may further arrange (e.g., order) the user selectable links as part of displaying the links to the user based on the result scores also included in the search results 220. For example, the user device 102 may assign each user selectable link the result score associated with the app record from which the ADA included in the link was selected. The user device 102 may then order the user selectable links based on the corresponding result scores (e.g., display higher-ranking links higher within a list of user selectable links). Example search results 220 displayed to a user of a user device 102 as user selectable links are described with reference to FIGS. 7A-7C.



FIG. 3A illustrates an example of the search system 100. As described herein, the search system 100 generates one or more search results 220 based on a search query 210 received from one of the user device(s) 102, app connection data 218 (e.g., an indication of one or more connections associated with a native app and/or a quality value associated with the app) generated by the analysis system 108, and data included in app records of the search data store 112. Specifically, the search module 110 identifies one or more app records included in the search data store 112 based on the search query 210 and, e.g., the app connection data 218. In some examples, the search system 100 further ranks the identified app records, e.g., also based on the app connection data 218. The search module 110 then transmits one or more app names/IDs 222 that identify the identified and, e.g., ranked, app records to the result generation module 114. The result generation module 114 receives the app names/IDs 222 from the search module 110, identifies the app records in the search data store 112 using the names/IDs 222, and selects one or more ADAs from the identified records. The result generation module 114 then transmits the selected ADAs to the user device 102 as the search results 220 (e.g., with link data, result scores, and/or other data associated with the identified app records).



FIG. 3B is a functional block diagram of an example search module 110. FIG. 3B also depicts examples of the search data store 112 and analysis system 108. The search module 110 of FIG. 3B includes a query analysis module 300, a consideration set generation module (hereinafter, “set generation module”) 302, and a consideration set processing module (hereinafter, “set processing module”) 304. The query analysis module 300 receives a search query 210 from one of the user device(s) 102 (e.g., as part of a query wrapper) and analyzes the query 210 (e.g., performs any of tokenization, filtering, stemming, synonymization, and stop word removal with respect to the query 210). The set generation module 302 identifies one or more app records included in the search data store 112 based on the (e.g., analyzed) search query 210 and, e.g., app connection data 218 received from the analysis system 108. As described herein, the app connection data 218 may indicate connections associated with native apps specified by app records included in the search data store 112 and/or information generated based on the connections (e.g., quality values indicating a degree of quality associated with the apps). For example, the set generation module 302 may identify one or more app records included in the search data store 112 based on one or more (e.g., text) matches between one or more terms of the search query 210 and one or more terms of information (e.g., AAs and/or app names/IDs) included in the records. In a specific example, the set generation module 302 may identify the app records using the search query 210 as an input to Lucene. In some examples, the set generation module 302 may further identify at least one of the app records based on the quality value associated with (e.g., included in) the record, as indicated by the app connection data 218. In a specific example, the set generation module 302 may identify the app record by using the quality value as a boost factor in Lucene. Alternatively, the set generation module 302 may identify the app record by determining that the quality value is greater than a threshold value. For example, the search module 110 may generate the threshold value dynamically, e.g., based on the search query 210. As a specific example, the search module 110 may generate a relatively high threshold value for some search queries 210 (e.g., “banking apps”) and a relatively low threshold value for other search queries 210 (e.g., “games”). The identified app records may be referred to herein as a “consideration set.”


The set processing module 304 may process (e.g., score and select a subset of) the consideration set. For example, the set processing module 304 may generate a result score for each app record of the consideration set, thereby ranking the records, and select one or more records from the set having the highest one or more result scores. In some examples, the set processing module 304 may generate the result score for at least one of the app records of the consideration set using the app connection data 218 (e.g., an indication of connection(s) and/or a quality value associated with the native app specified by the record). The set processing module 304 may then transmit one or more app names/IDs 222 associated with the (e.g., selected) app records of the consideration set to the result generation module 114, as described above.


The information conveyed by the search results 220 may depend on how the set processing module 304 generates the result scores for the app records of the consideration set. For example, for each app record, the corresponding result score may be generated based on various features associated with the record, such as relevance of the native app specified by the record to the search query 210, popularity of the app, and/or other properties of the app, depending on the one or more parameters the set processing module 304 uses to score the app records. The set processing module 304 may generate the result scores for the app records in a variety of different ways. In some examples, the set processing module 304 generates a result score for an app record based on one or more scoring features. The scoring features may be associated with the app record, the search query 210, and/or other data (e.g., app connection data 218). An app record scoring feature (hereinafter, “record scoring feature”) may be based on any data associated with an app record. For example, a record scoring feature may be based on any data included in AAs of an app record. An example record scoring feature may be a popularity score (e.g., based on user ratings of a native app) associated with an app record. A query scoring feature may include any data associated with the search query 210. For example, a query scoring feature may include any of a number of words in the search query 210, the popularity of the query 210, and an expected frequency of the words in the query 210. A record-query scoring feature may include any data generated based on information associated with both an app record and a search query 210 that resulted in identification of the record by the set generation module 302. For example, a record-query scoring feature may include any parameters that indicate how well terms of a search query 210 match terms of AAs (and/or an app name/ID) of an app record identified using the query 210. In some examples, as described herein, the set processing module 304 may generate a result score for an app record based on the app connection data 218. In these examples, an “app connection” scoring feature may include any data associated with the app connection data 218 (e.g., an indication of one or more connections associated with a native app and/or a quality value indicating a degree of quality associated with the app and generated based at least in part on the connections). In some examples, the set processing module 304 may generate a result score for an app record based on whether the native app specified by the record is, or is not, associated with one or more connections, as indicated by the app connection data 218. In other examples, the set processing module 304 may generate the result score based on a quality value indicating a degree of quality associated with the native app and generated based at least in part on the connections, as also indicated by the app connection data 218. In still other examples, the set processing module 304 may generate the result score based on any combination of the connections and the quality value. In general, the set processing module 302 may generate a result score for an app record using any of the record, query, record-query, app connection scoring features, and/or any additional scoring features not explicitly listed.


In some examples, to generate the result scores for the app records of the consideration set, the set processing module 304 may include one or more machine-learned models (e.g., a supervised learning model, for example, including regression) configured to receive one or more of the record, query, record-query, and app connection scoring features described herein. For example, the set processing module 304 may pair the search query 210 with each app record and calculate a vector of features for each (query, record) pair. The vector of features may include one or more record, query, record-query, and app connection scoring features. The set processing module 304 may then input the vector of features into a machine-learned relevance (MLR) model to calculate a result score for the app record (e.g., simultaneously based on the features). In some examples, the MLR model may include a set of (e.g., gradient-boosted) decision trees. In other examples, the MLR model may be trained by a simple form of logistic regression. In still other examples, the machine-learned task described herein can be framed as a semi-supervised learning task, where a minority of training data is labeled with human-curated result scores and the rest of the data is used without such labels.


As described herein, the result scores associated with the app records (e.g., the ADAs included therein) may be used in various different ways. In some examples, the result scores may be used to rank (e.g., order) the ADAs in a list. In these examples, a higher result score may indicate that the corresponding ADA (e.g., native app) is more relevant to the user than an ADA having a smaller result score. In examples where the search results 220 are displayed as a list of user selectable links on the user device 102, the links including ADAs associated with larger result scores may be listed closer to the top of the list (e.g., near the top of the screen). In these examples, links including ADAs having lower result scores may be located farther down the list (e.g., off screen) and may be accessed by scrolling down the screen of the user device 102.



FIGS. 4A-4B illustrate example app records 400A, 400B that may be included in the search data store 112. FIG. 4A illustrates a general example of an app record 400A. The app record 400A of FIG. 4A includes information related to (e.g., specifying) a native app (e.g., any native app included in a digital distribution platform). The app record 400A may generally represent data stored in the search data store 112 that is related to a native app. The search data store 112 may include one or more app records each having a similar structure as that of the app record 400A. In other words, the search data store 112 may include one or more app records each having an app name/ID, one or more AAs, one or more ADAs, and, e.g., link and/or app connection data associated with the record, which are described in greater detail herein.


As shown in FIG. 4A, the app record 400A includes an app name/ID 402A that uniquely identifies the record 400A among other app records included in the search data store 112. In some examples, the app name/ID 402A may correspond to a name of the native app specified by the data included in the app record 400A. For example, the app name/ID 402A may include any of “Google Maps,” “Facebook,” “Twitter,” “Microsoft Word,” and “Angry Birds.” Additionally, or alternatively, the app name/ID 402A may include an alphanumeric ID (e.g., an index) associated with the native app specified by the app record 400A (e.g., assigned to the app by the search system 100). In general, the app name/ID 402A may be a string of alphabetic, numeric, and/or symbolic characters (e.g., punctuation marks) that uniquely identify the app record 400A among other app records included in the search data store 112.


As further shown, the app record 400A includes AAs 404A (e.g., text) that describe the native app specified by the record 400A, and which may be used to identify the record 400A in the search data store 112. The AAs 404A may include data fields for any of a name of a developer (e.g., a publisher) of the native app, a genre (e.g., a category) of the app, a description of the app, user reviews for the app, user ratings for the app, and a number of downloads associated with the app. For example, the name of the developer of the native app may be “Rovio Entertainment Limited.” The genre of the native app may be “games.” In some examples, the developer of the native app may provide the description and/or the genre associated with the app. In other examples, the search system 100 may provide any of the description and genre of the native app. The data field associated with the user reviews may include text from reviews of the native app by users. The data field associated with the user ratings may indicate ratings (e.g., a number of stars, such as 0-5 stars) given to the native app by users. The data field associated with the number of downloads may indicate a total number of times users have downloaded the native app to a user device 102. In some examples, the AAs 404A may also describe one or more functions provided by the native app, such as, e.g., “search for travel destinations,” “make restaurant reservations,” and “buy movie tickets.”


In additional examples, the AAs 404A may include other information relating to the native app specified by the app record 400A, such as any of a version of the app, an OS associated with the app, a price of the app, security and/or privacy data regarding the app, and battery and/or bandwidth usage of the app. The AAs 404A may also include various numeric data (e.g., statistics) associated with the native app, such as any of a download rate (e.g., a number of downloads per month) and a number of user ratings and/or user reviews associated with the app. The AAs 404A may also include information retrieved from websites, such as user reviews associated with the native app, articles associated with the app (e.g., wiki articles), and/or other information. In some examples, the AAs 404A may further include digital media related to the native app, such as images (e.g., icons and/or screenshots) associated with the app.


In general, the AAs 404A may include any type of data associated with the native app specified by the app record 400A, including various different types of data, such as structured, semi-structured, and/or unstructured data. For example, the AAs 404A may include information extracted or inferred from documents retrieved from the data source(s) 104. Additionally, or alternatively, the AAs 404A may include data generated manually by a human operator. In some examples, the AAs 404A may be updated so that the search system 100 may provide up-to-date search results 220 in response to receiving a search query 210 from one of the user device(s) 102.


As also shown, the app record 400A includes one or more ADAs 406A that enable a user device 102 to access (e.g., download and install) the native app specified by the record 400A. In some examples, the app record 404A may include multiple ADAs 406A that are each configured to enable a user device 102 to access the native app on a different OS or platform. In these examples, to generate search results 220 based on a search query 210 received from one of the user device(s) 102, the search system 100 (e.g., the result generation module 114) may select one of the multiple ADAs 406A that corresponds to the OS or platform associated with the device 102. As further shown in FIG. 4A, the app record 400A may also include link data 408A, which may include text indicating a name of the native app specified by the record 400A and/or image data (e.g., one or more app icons, or screenshots) associated with the app.


As shown in FIG. 4A, the app record 400A may optionally include app connection data 410A. The app connection data 410A may indicate one or more connections associated with the native app specified by the record 400A. Additionally, or alternatively, the app connection data 410A may indicate a quality value indicating a degree of quality associated with the native app and generated based at least in part on the connections. As described herein, the analysis system 108 may initially generate app connection data 218 indicating the connections and/or the quality value. The search system 100 may then store the app connection data 218 in the app record 400A as the app connection data 410A and later use the data 410A to identify and/or rank the record 400A in response to receiving a search query 210 from one of the user device(s) 102. In some examples, the search system 100 may store the app connection data 218 in the app record 400A as part of the AAs 404A (e.g., by augmenting the AAs 404A to include the indication of the connections and/or the quality value).


In additional examples (not shown), the app record 400A may also include information that describes values of one or more metrics associated with the native app specified by the record 400A. Example metrics include popularity of (e.g., a number of downloads associated with) the native app and/or user ratings associated with the app. The information included in the app record 400A may also be based on measurements associated with the record 400A, such as how often the record 400A is retrieved during a search and how often user selectable links generated for any of the ADA(s) 406A of the record 400A are selected by a user.



FIG. 4B illustrates a specific example of an app record 400B that specifies the native application “ANGRY BIRDS®” by Rovio Entertainment Limited, (hereinafter, “Angry Birds”). As shown in FIG. 4B, the app record 400B includes an app name/ID “Angry Birds/ID#1” 402B that uniquely identifies the record 400B among other app records included in the search data store 112. In other examples, the app name/ID 402B may be a numeric value, or have another (e.g., machine-readable) representation. As further shown, the app record 400B includes AAs 404B that describe the native app specified by the record 400B (i.e., Angry Birds), and which (e.g., along with the app name/ID 402B) the search system 100 (e.g., the set generation module 302) may use to identify the record 400B in the search data store 112. For example, as described herein, the search system 100 may identify the app record 400B in the search data store 112 based on (e.g., text) matches between terms of a search query 210 received from one of the user device(s) 102 and terms of the AAs 404B. The AAs 404B of FIG. 4B describe a developer, a genre, a description, user reviews, user ratings, and a number of downloads associated with Angry Birds, in a similar manner as described with reference to the app record 400A.


As also shown, the app record 400B further includes one or more ADAs 406B that enable a user device 102 to access Angry Birds. Specifically, the ADA(s) 406B each indicate a location where the user device 102 may download Angry Birds. As explained herein, each of the ADA(s) 406A may specify a location where the user device 102 may download a particular version of Angry Birds corresponding to the specific OS or platform associated with the device 102. As further shown, the app record 400B also includes link data 408B, which may include a text string “Angry Birds” and an app icon, or a screenshot, associated with Angry Birds.


The app record 400B also optionally includes app connection data 410B indicating one or more connections associated with Angry Birds and/or a quality value indicating a degree of quality associated with Angry Birds generated based at least in part on the connections. As described herein, the analysis system 108 may initially generate app connection data 218 indicating the connections and/or quality value. The search system 100 may then store the app connection data 218 in the app record 400B as the app connection data 410B and later use the data 410B to identify and/or rank the record 400B in the manner described herein.



FIG. 5A illustrates example interactions and data exchanged among one of the user device(s) 102, the search system 100, the analysis system 108, and one or more APIs (e.g., servers and data stores) 500-1 . . . 500-N associated with native apps specified by app records included in the search data store 112. As described herein, the analysis system 108 generates app connection data 218 for each of one or more native apps specified by app records included in the search data store 112. In other words, the analysis system 108 generates the app connection data 218 for each of one or more of the app records. As further described herein, app connection data 218 associated with a particular native app (e.g., with a corresponding app record included in the search data store 112) may indicate one or more connections associated with the app and/or information (e.g., a quality value) generated for the app (e.g., for the app record) based on the connections. In some examples, the analysis system 108 may generate app connection data 218 for a subset of the native apps specified by the app records included in the search data store 112 (e.g., some native apps may not be associated with connections). In other examples, for a particular native app specified by an app record included in the search data store 112, the analysis system 108 may generate app connection data 218 that indicates a subset of the connections associated with the app (e.g., the system 108 may be unable to identify all connections associated with the app).


In this disclosure, a connection associated with a native app may include any of a variety of connection types. In some examples, the connection may refer to a so-called “outbound” link included in the native app that is used by the app to retrieve data from another resource (e.g., a “deep link,” or a passed intent to an activity that is not part of the app, as in the case of ANDROID® by Google). In other examples, the connection may refer to a so-called “inbound” link included in another resource that is used by the resource to retrieve data from the native app. In further examples, the connection (e.g., the outbound or inbound link) may be between the native app and another, different native app. In other words, the connection may facilitate the exchange of data between two native apps. In other examples, the connection may be between the native app and a resource associated with a native app, such as an API associated with a native app, or a “native API” (e.g., one or more servers and data stores used by a native app), a native app library (e.g., portions of code associated with a native app), or another resource, such as a website associated with a native app (e.g., a web-equivalent of a native app). For example, the connection may be between the native app and an API, an app library, and/or a website associated with another native app. In some examples, the native app may include the native app library (e.g., instructions) associated with the other native app. In these examples, the native app library being included in the native app may constitute a connection between the app and the other native app. In still other examples, the connection may be between the native app and a non-app resource, such as a website (e.g., a web-based app not associated with a native app), an advertisement (ad) network, or another web resource (e.g., a multiplayer game server). As such, the connection may be directional and have an associated type. In this manner, a connection associated with a native app, as described herein, may be a connection between the app and various different resources.


As shown in FIG. 5A, the analysis system 108 includes the connection determination module 116, connection data store 118, and connection record module 120. The connection determination module 116 may generate the app connection data 218 for each of the one or more native apps specified by the app records included in the search data store 112. The connection record module 120 may generate one or more connection records used to store the app connection data 218 generated by the connection determination module 116 and store the records, including the data 218, in the connection data store 118. In some examples, upon the connection determination module 116 generating the app connection data 218, the module 116 may further generate a so-called “app connection graph” (e.g., as shown in FIG. 6D) representing the data 218, namely, one or more connections associated with each native app. In these examples, the connection record module 120 may generate a connection record used to store the app connection graph and store the record, including the graph, in the connection data store 118.


In this example, to generate app connection data 218 for a native app specified by a particular app record included in the search data store 112, the connection determination module 116 may be configured to perform any of the following actions. Initially, the connection determination module 116 may determine (e.g., identify) one or more connections associated with the native app. For example, to determine the connections, the connection determination module 116 may perform any combination of static and dynamic connection analyses with respect to the native app. As one example, to perform the static connection analysis, the connection determination module 116 may analyze one or more commands (e.g., human-readable programming language instructions, or machine-readable instructions specifying operations to be performed by a processing unit) associated with the native app (e.g., with an executable binary object associated with the app). In some examples, the commands may be a part of various code components of the native app having various levels of abstraction, and which may interoperate via messages, events, and/or so-called “intents.” For example, the connection determination module 116 may analyze the commands associated with the native app by accessing an API (e.g., an API data store) associated with the app, or another resource that stores the commands and where the commands are not being executed by a processing unit (e.g., in an “off-line” manner). For instance, as shown in FIG. 5A, the connection determination module 116 may access one of the API(s) 500-1 . . . 500-N (e.g., one of API data store(s) 502-1 . . . 502-N included therein) associated with the native app. In this example, the API(s) 500-1 . . . 500-N may be associated with one or more native apps 504-1 . . . 504-N specified by one or more app records 400A-1 . . . 400A-N included in the search data store 112, as indicated by dashed line 506. As a result of performing the analysis, the connection determination module 116 may identify one or more commands configured to invoke communication (e.g., data exchange) between the native app and one or more other resources. For example, as shown in FIG. 5C, the connection determination module 116 may include a static analysis module 510 that, in turn, includes a command analysis module 512 configured to analyze commands associated with native apps without fully executing the apps and determine connections for each app based on the commands. In this manner, the connection determination module 116 may determine one or more connections corresponding to one or more outbound links associated with the native app. To determine connections corresponding to inbound links associated with the native app, the connection determination module 116 may perform static connection analysis with respect to another native app that retrieves data from the app, in a similar manner as described above.


As another example, to perform the dynamic connection analysis, the connection determination module 116 may monitor one or more live interactions (e.g., data exchanges) between the native app and one or more other resources. For example, the connection determination module 116 may detect the interactions between the native app (e.g., in response to a user input) and resources while the app executes on a computing device, such as a user device 102, or an API associated with the app (e.g., in a “real-time” manner). Alternatively, the connection determination module 116 may receive an indication of the interactions from a computing device that executes the native app. For example, as shown in FIG. 5A, the connection determination module 116 may receive app interaction data 216 indicating the interactions from the user device 102 (e.g., assuming the native app is included on the device 102). In other examples, the connection determination module 116 may receive the app interaction data 216 from another location (e.g., from one of the API(s) 500-1 . . . 500-N associated with the native app). By monitoring the interactions, the connection determination module 116 may identify one or more resources with which the native app is configured to communicate (e.g., exchange data). For example, as also shown in FIG. 5C, the connection determination module 116 may include a dynamic analysis module 514 that, in turn, includes a data exchange detection module 516 configured to monitor interactions (e.g., data exchanges) between executing native apps and other resources and determine connections for each app based on the interactions. Additionally, or alternatively, as also shown, the data exchange detection module 516 may be configured to receive an indication of the interactions (e.g., as app interaction data 216) and determine the connections for each native app based on the data 216. In this manner, the connection determination module 116 may determine one or more connections corresponding to one or more outbound and/or inbound links associated with the native app. In general, the connection determination module 116 may determine one or more connections associated with the native app by performing any combination of the static and dynamic connection analyses, or by receiving equivalent or analogous inputs from a parallel system (or a set of multiple systems) tasked with performing the static and/or dynamic connection analysis.


In some examples, another system or device, rather than the connection determination module 116, may perform any of the static and dynamic connection analyses described above. In these examples, the connection determination module 116 may receive an output of this system or device and determine the one or more connections associated with the native app based on the output. In some examples, the connection determination module 116 may determine the connections based on known markers included in the output. In other examples, the connection determination module 116 may determine the connections based on outputs corresponding to static and/or dynamic connection analyses performed for other native apps.


In some examples, to generate the app connection data 218 for the native app, the connection determination module 116 may be configured to identify any combination of one or more so-called “explicit” and “inferred” connections associated with the app. As one example, to identify an explicit connection associated with the native app, the connection determination module 116 may identify a connection between the app and another resource (e.g., another native app, or a non-app resource). For instance, the connection determination module 116 may identify a user selectable link included in the native app that links to (e.g., opens a state of) another native app, or a website that does not correspond to a native app. As another example, to identify an inferred connection associated with the native app, the connection determination module 116 may identify a connection between the app and a website (e.g., a user selectable link included in the app that links to the website). In this example, the website may correspond to another native app (e.g., the website may be a web-based app that is a web-equivalent of the other app). Upon identifying the connection between the native app and the website, the connection determination module 116 may infer the connection between the app and the other native app. In other examples, the connection determination module 116 may similarly infer the connection between the native app and the other native app based on a connection between the app and any of an API, an app library, and another resource associated with the other app.


In a specific example, the connection determination module 116 may initially analyze each of one or more native apps, e.g., each represented by a native app binary, and generate app connection data 218 that indicates (e.g., using one or more alphanumeric strings) one or more connections associated with each app. For example, an input to the connection determination module 116 may be the string “YouTube_1_Android_OS2.3, YouTube_2_Android_OS5.2, Facebook . . . ” indicating the native apps. In this example, the app connection data 218 may indicate various relationships between each native app and one or more associated connections as follows: “App1: -->URL2, App1 includes Library 3, App2: -->URL4 . . . .” The connection determination module 116 may then assign indicators (e.g., IDs) to the determined connections, e.g., as follows: “URL2===APP_id=‘YouTube_1232323,’ Library 3===Unknown, URL4===Unknown, Library 5===App_id=‘YouTube_1232323’ . . . ” The connection determination module 116 may optionally normalize the data described above. The connection determination module 116, or a component of the search system 100 (e.g., a record generation/update module) may then augment the corresponding app records included in the search data store 112 to include the (e.g., normalized) data. For example, the app records may be augmented to include various features derived from the determined connections, such as “a number of inbound links,” “a number of outbound links,” “link quality” (e.g., as determined using one or more algorithms), and a list of other app records (e.g., app IDs included therein) that specify other native apps connected with the native apps specified by the records. As a specific example, an app record (e.g., “App_record_1”) specifying the native app YouTube (e.g., corresponding to the canonical app YouTube) may be augmented to include the following data: “App_record_1 (All YouTube): num_in_links=1, num_out_links=12, connected-app_record_ids: [facebook, netflix, IMDB].”


In the example of FIG. 5A, upon determining the one or more connections associated with the native app, the connection determination module 116 may be further configured to generate a quality value (e.g., a numeric value between 0 and 1) indicating a degree of quality associated with the app based on the connections. In general, a relatively larger quality value generated for a native app may indicate a relatively higher degree of quality associated with the app compared to a relatively smaller quality value generated for the app. For example, upon identifying N connections associated with the native app, where N is an integer value greater or equal to 1, the connection determination module 116 may generate a quality value for the app based on one or more (e.g., a subset) of the N connections. In some examples, the connection determination module 116 may generate the quality value for the native app based on a single one of the N connections. In other examples, the connection determination module 116 may generate the quality value for the native app based on multiple ones of the N connections. In this example, the quality value being closer to 0 than 1 (e.g., less than 0.5) may indicate a relatively lower quality of the native app (e.g., the associated one or more connections), whereas the quality value being closer to 1 than 0 (e.g., greater than 0.5) may indicate a relatively higher quality of the app (e.g., the connections).


In some examples, to generate the quality value for the native app based on the one or more connections associated with the app, the connection determination module 116 may use a set of one or more rules. For example, the connection determination module 116 may apply the set of rules to an indication of the connections associated with the native app and compute the quality value for the app in response to applying the set of rules to the indication. In these examples, the set of rules may be manually generated based on user inputs and/or automatically generated based on data indicating past user behavior. For example, the set of rules may be defined by one or more human operators of the analysis system 108. In this example, each rule may be associated with one or more connections (e.g., with one or more specific resources linked to a native app by the connections) and configured to influence the quality value based on whether or not the connections are present with respect to a particular native app. For example, each rule may be configured to contribute positively or negatively to the quality value based on whether the connections are associated with the native app. In other examples, each rule may be configured to contribute positively or negatively to the quality value based on the number, the types, and/or the relative (e.g., individual) quality of the connections associated with the native app (e.g., based on the number of inbound and/or outbound links, or the popularity or common use of a particular connection). Additionally, or alternatively, the set of rules may be defined using data indicating past user behavior with respect to native apps having particular connections to other resources. For example, the set of rules may be defined using so-called “click data” indicating whether and/or how often users select search results 220 specifying native apps associated with specific connections. In general, the set of rules may be defined using any (e.g., historical) user behavioral data describing one or more of a degree of quality a user perceives to be associated with a particular connection associated with a given native app, a degree of quality the user perceives to be associated with the app itself, and whether and/or how often the user has previously selected a search result 220 that specifies the app. In additional examples, the set of rules may be defined using data derived by correlating other indicators of quality of native apps (e.g., user ratings) with particular connections associated with the apps. For example, the data may indicate that one or more native apps each having one or more specific connections (e.g., links, or app libraries) frequently receive low user ratings indicative of low quality in a particular distribution platform (e.g., Google Play by Google Inc.).


In other examples, to generate the quality value for the native app based on the one or more connections associated with the app, the connection determination module 116 may use a machine-learned model. For example, the connection determination module 116 may include one or more supervised learning models configured to receive an indication of one or more connections associated with a native app and generate a quality value indicating a degree of quality associated with the app using the indication. For example, the connection determination module 116 may provide the indication of the connections to the machine-learned model as one or more inputs. The machine-learned model may compute the quality value based on the indication and output the value to the connection determination module 116. In some examples, the machine-learned model included in the connection determination module 116 may be created using training data (e.g., indications of native apps and indications of connections associated with the apps), some or all of which may be labeled with human-curated quality values. In this manner, the training data may include one or more indications of one or more training connections associated with each of one or more training native apps, and one or more training quality values each indicating a degree of quality associated with one of the training apps. In general, the training data may be generated using any of the various types of user behavioral data described above with reference to the set of rules. In some examples, the set of rules and/or the machine-learned model described herein may be embodied in one or more software instructions and included in a quality value generation module 508, as depicted in FIG. 5B. In the example of FIG. 5B, the quality value generation module 508 may be a part of the connection determination module 116, another module included in the analysis system 108, or within another stand-alone system or device.


In some examples, the quality value associated with the native app may indicate popularity of the one or more resources linked with the app by the one or more connections used to generate the value. For example, the quality value may indicate whether and/or how many other native apps connect with the resources and/or whether the resources are used by many users. In other examples, the quality value may indicate quality of the resources (e.g., whether the resources are updated frequently and/or whether the resources generally have positive user reviews and/or relatively high user ratings). In still other examples, the quality value may indicate one or more other properties of the resources determined using data associated therewith. In this manner, the quality value may indicate a degree of quality associated with the native app that links with the resources via the connections. In other words, the various properties associated with the resources (e.g., popularity, quality, and so forth) may be imputed onto the native app by virtue of the app being connected with the resources via the connections.


Upon the connection determination module 116 generating the app connection data 218 for the native app, including an indication of the connections and/or the quality value associated with the app, the analysis system 108 transmits the data 218 to the search system 100. The search system 100 may use the app connection data 218 as described herein. In some examples, the search system 100 may store some or all of the app connection data 218 (e.g., the indication of the connections and/or quality value) in the corresponding app record included in the search data store 112. For example, as described herein with reference to FIGS. 4A-4B, the search system 100 may store the app connection data 218 in a designated field of the app record.



FIGS. 6A-6C illustrate various types of connections that may be associated with a native app. FIG. 6A depicts a native app (“Native App 1”) that is connected with any combination of 1) another native app (e.g., “Native App 2”), 2) an API associated with the other app (e.g., “Native App 2 API”), 3) an app library associated with the other app (e.g., “Native App 2 Library”), 4) a website associated with the other app (e.g., a web equivalent of the other app), and 5) another resource associated with the other app that is not explicitly listed. In the example of FIG. 6A, Native App 1 is connected with one or more resources each of which is either a native app, or is associated with a native app. FIG. 6B depicts an example in which a connection between a native app (“Native App 1”) and another native app (“Native App 2”) is inferred based on a connection between the app and any of 1) an API associated with the other app, 2) an app library associated with the other app, 3) a website associated with the other app, and 4) another resource associated with the other app that is not explicitly listed. FIG. 6C depicts a native app (“Native App 1”) that is connected with any combination of 1) a website not associated with another native app, 2) an ad network, 3) a game server, 4) a statistics service (e.g., Google Analytics® by Google Inc.) and 5) another non-app resource not explicitly listed. In the example of FIG. 6C, Native App 1 is connected with one or more non-app resources.



FIG. 6D illustrates an example representation of connections associated with native apps. The representation shown in FIG. 6D may be referred to herein as an “app connection graph” and may be included as part of app connection data 218 generated by the analysis system 108. The app connection graph of FIG. 6D indicates one or more connections (“C1” . . . “C16”) associated with each of one or more native apps (“Native App 1” . . . “Native App 3”) specified by one or more app records included in the search data store 112. As shown in FIG. 6D, the connections are configured to link (e.g., facilitate data exchange between) each of the native apps and one or more other resources, such as another native app (e.g., another one of the apps), an API, an app library, and/or a non-app resource (e.g., a website, an ad network, or another resource). The analysis system 108 may generate the app connection graph of FIG. 6D by identifying the connections associated with the native apps using any of the static and dynamic connection analysis techniques described herein. In some examples (not shown), the analysis system 108 may further include indications of (e.g., app names/IDs included in app records specifying) the native apps within the app connection graph. Upon generating the app connection graph, the analysis system 108 may store the graph in a connection record included in the connection data store 118 for later retrieval. At a later point in time (e.g., in response to the search system 100 receiving a search query 210), the analysis system 108 may retrieve and traverse the app connection graph to determine the one or more connections associated with a particular one of the native apps 600. Traversing the app connection graph may enable the analysis system 108 to determine the connections associated with the native app 600 relatively quicker than by performing the static and/or dynamic connection analysis for the app 600.



FIGS. 7A-7C depict example GUIs that may be generated on one of the user device(s) 102 according to this disclosure. In particular, the examples of FIGS. 7A-7C depict the user device 102 performing a search for native apps using a search query 210 specified by a user of the device 102 and app connection data 218 associated with the apps. As shown in FIG. 7A, the user initially enters a search query “movies” 210 into a search field 212 of a GUI of a search app 204 executing on the user device 102. As also shown, the user then interacts with a search button 214 of the GUI to cause the search app 204 to transmit the search query 210 to the search system 100. As described herein, in some examples, the user device 102 (e.g., the search app 204) may also transmit app interaction data 216 to the search system 100 (e.g., with the search query 210, or separately). As also described herein, the user device 102 (e.g., the app interaction determination module 208) may generate the app interaction data 216 by monitoring interactions (e.g., data exchanges) between the native app(s) 206 included on the device 102 and other (e.g., native app and/or non-app) resources. In these examples, the app interaction data 216 may indicate the interactions between the native app(s) 206 and the other resources.


The search system 100 may receive the search query 210 and, e.g., the app interaction data 216, from the user device 102. The search system 100 may then generate search results 220 that each specify a native app based on the search query 210 and app connection data 218 generated by the analysis system 108, as described herein. In some examples, the analysis system 108 may generate the app connection data 218 using the app interaction data 216 received from the user device 102, as also described herein. In the example of FIGS. 7A-7C, the search results 220 specify the native apps Flixster, Toy Story 3, YouTube, IMDb, and Fandango. In this example, the app connection data 218 used by the search system 100 to generate the search results 220 may indicate one or more connections associated with each of these native apps. Additionally, or alternatively, the app connection data 218 may indicate quality values each associated with one of the native apps, where each value is generated based at least in part on the connections associated with the corresponding native app. To generate the search results 220, the search system 100 may identify app records included in the search data store 112 that specify Flixster, Toy Story 3, YouTube, IMDb, and Fandango. As described herein, the search system 100 may identify the app records based on the search query 210. In some examples, the search system 100 may further identify the app records based on the app connection data 218 (e.g., the quality values), as also described herein. Additionally, or alternatively, the search system 100 may rank the identified app records, e.g., based on the app connection data 218 (e.g., the indications of the connections and/or quality values), as further described herein. The search system 100 may select one or more ADAs from the identified and, e.g., ranked, app records and transmit the search results 220, including the ADAs, to the user device 102.


As shown in FIG. 7B, the user device 102 receives the search results 220 from the search system 100 in response to transmitting the search query 210 and, e.g., the app interaction data 216, to the system 100. As also shown, the user device 102 displays the search results 220 to the user as user selectable links 226-1 . . . 226-5 (collectively, the “links 226”). For example, the user device 102 may generate each of the links 226 using link (e.g., text and/or image) data also received from the search system 100 as part of the search results 220. In this example, the search results 220 are responsive to the search query 210 (i.e., the text string “movies”). In particular, the search results 220 specify the native apps Flixster, Toy Story 3, YouTube, IMDb, and Fandango, which are each associated with movies and related services. As further shown, the user device 102 may order the links 226 within a list. For example, the user device 102 may order each link 226 based on the result score associated (e.g., received) with the corresponding one of the search results 220. As also shown, the user device 102 may display the links 226 such that one or more of the links 226 each indicate to the user (e.g., via any of GUI elements 224A . . . 224E) that the corresponding native app is accessible on the device 102 free of charge.


As also shown in FIG. 7B, the user may select (e.g., touch, or click on) one of the links 226 on the user device 102, namely the user selectable link 226-4 referencing IMDb. As shown in FIG. 7C, upon the user selecting the user selectable link 226-4, the user device 102 may download (e.g., from a digital distribution platform using an ADA included in the link 226-4) and install IMDb. As also shown, upon downloading and installing IMDb, the user device 102 may further launch IMDb. Specifically, as depicted in FIG. 7C, the user device 102 may configure IMDb to display a GUI 700 corresponding to the main, or default screen of IMDb. Upon the user device 102 downloading, installing, and launching IMDb in the manner described herein, the user may interact with IMDb (e.g., search for movie entries within IMDb).



FIG. 8 is a flow diagram that illustrates an example method 800 for generating search results 220 based on a search query 210 and app connection data 218 using the search system 100. As shown in FIG. 8, in block 802, the analysis system 108 may initially, for each of a plurality of app records included in the search data store 112 each specifying a native app and including an ADA for downloading the app, determine one or more connections associated with the app and generate a quality value indicating a degree of quality associated with the app based on the connections. As described herein, to determine the connections associated with the native app, the analysis system 108 may perform any of static connection analysis (e.g., analyze software instructions associated with the app) and dynamic connection analysis (e.g., detect interactions between the app and other resources). As further described herein, to generate the quality value based on the connections, the analysis system 108 may use any of a set of rules and a machine-learned model. Upon determining the connections and generating the quality value for each of the plurality of app records, the analysis system 108 may transmit app connection data 218, including the value and, e.g., an indication of the connections, to the search system 100. As described herein, in some examples, the analysis system 108 may also store the app connection data 218 in a connection record included in the connection data store 118. As also described herein, in other examples, upon receiving the app connection data 218 from the analysis system 108, the search system 100 may store the data 218 in the corresponding one of the plurality of app records included in the search data store 112.


In block 804, (e.g., at a later point in time following the analysis system 108 generating the app connection data 218, as described with reference to block 802), the search system 100 may receive a search query 210 specified by a user from one of the user device(s) 102 (e.g., as part of a query wrapper). In block 806, the search system 100 (e.g., the query analysis module 300) may optionally perform an analysis of the search query 210. For example, the search system 100 may perform any of tokenization, filtering, stemming, synonymization, and stop word removal with respect to the search query 210. In some examples, the search system 100 may receive additional information from the user device 102 (e.g., as part of the query wrapper, or separately), such as user information and/or geo-location, platform, and IP address information associated with the device 102.


In block 808, the search system 100 (e.g., the set generation module 302) may identify a consideration set of one or more of the plurality of app records included in the search data store 112 based on the (e.g., analyzed) search query 210 and based on the quality value associated with each identified record, as indicated by the app connection data 218. For example, the search system 100 may identify each app record of the consideration set based on (e.g., text) matches between terms of the search query 210 and terms of information (e.g., AAs and/or an app name/ID) included in the record. In this example, the search system 100 may further identify each app record based on the quality value associated with (e.g., included in) the record (e.g., by using the value as a boost factor in Lucene). As such, the search system 100 may identify the app records of the consideration set such that each record both matches the search query 210 and specifies a native app having a particular degree of quality.


In some examples, the search system 100 may identify at least one of the app records of the consideration set in the manner described herein using a quality value associated with the record that has been previously generated. For example, the analysis system 108 may generate the quality value prior to the search system 100 receiving the search query 210 from the user device 102. As one example, the analysis system 108 may store the quality value in a connection record included in the connection data store 118. As another example, the search system 100 may store the quality value in the one of the plurality of app records included in the search data store 112 that corresponds to (e.g., that will be identified as) the app record of the consideration set. In these examples, upon the search system 100 receiving the search query 210 from the user device 102, the system 100 may retrieve the quality value from any of these records and use the value as described herein to identify the corresponding app record of the consideration set.


In other examples, the search system 100 may identify at least one of the app records of the consideration set using a dynamically-generated quality value associated with the record. For example, the analysis system 108 may generate the quality value in response to the search system 100 receiving the search query 210 from the user device 102. In a specific example, to generate the quality value for the app record, the analysis system 108 may analyze (e.g., traverse) an app connection graph (e.g., as shown in FIG. 6D), which the system 108 may have previously generated. In this example, the app connection graph may indicate the native app specified by the app record and one or more connections between the app and other resources (e.g., native apps specified by other app records included in the search data store 112 and/or non-app resources). As a result of analyzing the app connection graph, the analysis system 108 may identify the connections associated with the native app and generate the quality value based on the connections. Upon the analysis system 108 generating the quality value for the app record, the search system 100 may use the value as described herein to identify the record.


In blocks 810-812, the search system 100 (e.g., the set processing module 304) may optionally process the consideration set of app records. Specifically, in block 810, the search system 100 may generate one or more result scores for the app records included in the consideration set. For example, the search system 100 may generate a result score for each app record of the consideration set. In block 812, the search system 100 may select one or more app records from (e.g., select a subset of) the consideration set based on the one or more result scores associated with the selected records. For example, the search system 100 may select one or more app records of the consideration set having the highest (e.g., largest) one or more result scores.


In block 814, the search system 100 (e.g., the result generation module 114) may select one or more ADAs from the (e.g., selected) app records of the consideration set. For example, the search system 100 may select an ADA from each (e.g., selected) app record of the consideration set. In some examples, the search system 100 may also select other information from the (e.g., selected) app records of the consideration set, such as link data, result scores, and/or other data associated with the records. In block 816, the search system 100 (e.g., the result generation module 114) may generate one or more search results 220 that include the selected ADAs. For example, the search system 100 may generate the search results 220 such that each result 220 includes one of the ADAs and, e.g., the other information, selected from each (e.g., selected) app record. In block 818, the search system 100 (e.g., the result generation module 114) may transmit the search results 220, including the selected ADAs and, e.g., the selected other information, to the user device 102.



FIG. 9 is a flow diagram that illustrates another example method 900 for generating search results 220 based on a search query 210 and app connection data 218 using the search system 100. Blocks 902-906 of the method 900 are analogous to blocks 802-806 of the method 800. In block 908, the search system 100 (e.g., the set generation module 302) may identify a consideration set of one or more of the plurality of app records included in the search data store 112 based on the (e.g., analyzed) search query 210. For example, the search system 100 may identify each app record of the consideration set based on (e.g., text) matches between terms of the search query 210 and terms of information (e.g., AAs and/or an app name/ID) included in the record, in a similar manner as described with reference to the method 800.


In block 910, the search system 100 (e.g., the set processing module 304) may, for each app record of the consideration set, generate a result score based on the quality value associated with the record. As described herein, to generate the result score based on the quality value, the search system 100 may use the value as a scoring feature (e.g., along with one or more other scoring features associated with the search query 210, the app record, and/or other parameters) in conjunction with an MLR model. As also described herein, in other examples, to generate the result score, the search system 100 may use the connections as one or more scoring features (e.g., along with one or more other scoring features) in conjunction with the MLR model. In some examples, the search system 100 may generate the result score for at least one of the app records of the consideration set using a quality value that has been previously generated, in a similar manner as described with reference to the method 800. Alternatively, the search system 100 may generate the result score using a dynamically-generated quality value, also in a similar manner as described with reference to the method 800.


In block 912, the search system 100 (e.g., the set processing module 304) may select one or more app records (e.g., a subset) of the consideration set based on the result score associated with each selected record. For example, the search system 100 may select any app record of the consideration set that is associated with a result score that is greater than a predetermined threshold score. Alternatively, the search system 100 may select N app records of the consideration set (e.g., where N is an integer value that is greater than 0) that are associated with the highest (e.g., largest) one or more results scores. In a specific example, the search system 100 may rank the app records of the consideration set based on the result scores associated with the records. For example, the search system 100 may arrange the app records in an order of decreasing result scores. The search system 100 may then select one or more of the arranged app records based on the order. For instance, the search system 100 may select one or more of the app records that are associated with result scores that are greater than a predetermined threshold score, or with the highest (e.g., largest) results scores.


In block 914, the search system 100 (e.g., the result generation module 114) may select one or more ADAs from the selected app records. For example, the search system 100 may select an ADA from each selected app record. In some examples, the search system 100 may also select other information from the selected app records, such as link data, result scores, and/or other data associated with the records. In block 916, the search system 100 (e.g., the result generation module 114) may generate one or more search results 220 that include the selected ADAs. For example, the search system 100 may generate the search results 220 such that each result 220 includes one of the ADAs and, e.g., the other information, selected from each selected app record. In block 918, the search system 100 (e.g., the result generation module 114) may transmit the search results 220, including the selected ADAs and, e.g., the selected other information, to the user device 102.


In some examples, the transmission of the search results 220 from the search system 100 to the user device 102, as described in the methods 800 and 900, may be intermediated by any number of other services, systems, and/or devices.



FIGS. 10-11 are flow diagrams that each illustrate an example of a particular aspect of the methods 800 and 900 described herein relating to generating app connection data 218 that indicates connections associated with native apps. FIG. 10 depicts an example method 1000 for generating a quality value indicating a degree of quality associated with a native app based on one or more connections associated with the app using a set of rules. FIG. 11 depicts an example method 1100 for generating the quality value using a machine-learned model.


As shown in FIG. 10, in block 1002, the analysis system 108 may initially generate a set of one or more rules (e.g., software instructions and associated parameters) configured to compute a quality value indicating a degree of quality associated with a native app based on one or more connections associated with the app. For example, one or more human users may specify the set of rules and transmit an indication of the set to the analysis system 108. Additionally, or alternatively, the analysis system 108 may automatically define the set of rules using various considerations. In some examples, the analysis system 108 may generate the set of rules using data indicating a degree of quality users perceive to be associated with specific native app connections. In other examples, the analysis system 108 may generate the set of rules using data indicating a degree of quality users perceive to be associated with native apps having particular connections. In still other examples, the analysis system 108 may generate the set of rules using data indicating whether and/or how often users select search results specifying native apps associated with particular connections. As one example, the set of rules may specify that the presence of one or more particular connections associated with the native app results in the quality value being increased or decreased by a predetermined amount (e.g., 0.1). As another example, the set of rules may indicate that the absence of one or more specific connections associated with the native app results in the quality value being increased or decreased by a predetermined amount. As still another example, the set of rules may dictate that the presence and/or absence of one or more specific combinations of connections associated with the native app results in the quality value being increased or decreased by a predetermined amount.


As described with reference to the methods 800 (e.g., block 802) and 900 (e.g., block 902), the analysis system 108 may determine one or more connections associated with a native app specified by an app record and generate a quality value indicating a degree of quality associated with the app based on the connections. To generate the quality value based on the connections, as shown in block 1004, the analysis system 108 may apply the set of rules described above to the connections. As further shown in block 1006, in response to (e.g., as a result of) applying the set of rules to the connections, the analysis system 108 may compute the quality value. For example, the analysis system 108 (e.g., one or more processing units included therein) may execute software instructions (e.g., stored in a memory) that embody the set of rules, causing the system 108 to receive an indication of the connections as one or more inputs, process the connections, generate the quality value, and transmit the value as an output.


As shown in FIG. 11, in block 1102, the analysis system 108 may initially generate training data that includes an indication of one or more training connections associated with each of one or more training native apps and one or more training quality values each indicating a degree of quality associated with one of the apps. For example, in a similar manner as described with reference to FIG. 10, one or more human users may specify the training data and transmit the data to the analysis system 108. Additionally, or alternatively, the analysis system 108 may automatically generate the training data using any of various considerations, e.g., those described with reference to FIG. 10. In a specific example, the analysis system 108 may generate the training data by providing the indication of the training connections associated with the training native apps to one or more human users, and, in response to providing the indication, receiving the training quality values from the users. In block 1104, the analysis system 108 may generate a machine-learned model (e.g., software instructions and associated parameters) based on the training data. Stated another way, the analysis system 108 may train the machine-learned model using the training data. As a result, the machine-learned model may be configured to compute a quality value indicating a degree of quality associated with a native app based on one or more connections associated with the app.


As described with reference to the methods 800 (e.g., block 802) and 900 (e.g., block 902), the analysis system 108 may determine one or more connections associated with a native app specified by an app record and generate a quality value indicating a degree of quality associated with the app based on the connections. To generate the quality value, as shown in block 1106, the analysis system 108 may provide an indication of the connections to the machine-learned model described above as one or more inputs. As further shown in block 1108, in response to (e.g., as a result of) providing the indication of the connections to the machine-learned model, the analysis system 108 may receive the quality value from the model. For instance, the analysis system 108 (e.g., one or more processing units included therein) may execute software instructions (e.g., stored in a memory) that embody the machine-learned model, causing the system 108 to receive the indication of the connections as the inputs, process the connections, generate the quality value, and transmit the value as an output.



FIG. 12 is a flow diagram that illustrates an example method 1200 for generating search results 220 based on a search query 210 and app connection data 218 using a user device 102. As shown in FIG. 12, in block 1202, one of the user device(s) 102 may initially receive a search query 210 from a user of the device 102. As described herein, the user device 102 may receive the search query 210 from the user via a search app 204 executing on the device 102. In block 1204, the user device 102 may transmit the search query 210 to the search system 100. As also described herein, the user device 102 may transmit the search query 210 to the search system 100 in response to receiving an input from the user via the search app 204. In this example, the search system 100 may receive the search query 210 from the user device 102 and generate one or more search results 220 based on the query 210 and app connection data 218 generated by the analysis system 108. As explained herein, the search results 220 may include one or more ADAs, link data, result scores, and/or other information. The search system 100 may then transmit the search results 220 to the user device 102.


In block 1206, the user device 102 may receive the search results 220, including the ADAs, from the search system 100 in response to transmitting the search query 210 to the system 100. In block 1208, the user device 102 may display the ADAs to the user as one or more user selectable links. As described herein, the user device 102 may display the ADAs as the user selectable links via the search app 204. For example, the user device 102 may generate each user selectable link to include one of the ADAs and, e.g., the corresponding link data and/or other information also received as part of the search results 220. In some examples, the user device 102 may further rank (e.g., arrange within an order) the user selectable links including the ADAs based on the corresponding result scores also received along with the search results 220.


In block 1210, the user device 102 may determine that the user has selected one of the user selectable links displayed to the user. In block 1212, in response to making this determination, the user device 102 may download a native app referenced by the selected user selectable link (e.g., from a digital distribution platform using the ADA included in the link) and, in block 1214, install the app on the device 102. In block 1216, upon downloading and installing the native app, the user device 102 may optionally launch the app on the device 102.


In further examples, the search system 100 may generate one or more additional search results (not shown) that include content that does not specify native apps (e.g., content related to native app states, websites, documents, and/or media files). In these examples, the search system 100 may identify one or more records (e.g., app state records, or other data structures) stored in a data store that include the content based on the search query 210, in a similar manner as described herein. The search system 100 may then select the content from the identified records and transmit the content to the user device 102 with the search results 220.


The modules and data stores included in the search system 100 and analysis system 108 represent features that may be included in these systems 100, 108 as they are described in the present disclosure. For example, the search module 110, search data store 112, and result generation module 114 may represent features included in the search system 100. Similarly, the connection determination module 116, connection data store 118, and connection record module 120 may represent features included in the analysis system 108. The modules and data stores described herein may be embodied by electronic hardware, software, and/or firmware. Depiction of different features as separate modules and data stores does not necessarily imply whether the modules and data stores are embodied by common or separate electronic hardware, software, and/or firmware components. In some implementations, the features associated with the modules and data stores depicted herein may be realized by one or more common or separate electronic hardware, software, and/or firmware components.


The modules and data stores may be embodied by electronic hardware, software, and/or firmware components including, but not limited to, one or more processing units, memory components, input/output (I/O) components, and interconnect components. The interconnect components may be configured to provide communication between the processing units, memory components, and I/O components. For example, the interconnect components may include one or more buses configured to transfer data between electronic components. The interconnect components may also include control circuits (e.g., a memory controller and/or an I/O controller) configured to control communication between electronic components.


The processing units may include one or more central processing units (CPUs), graphics processing units (GPUs), digital signal processing units (DSPs), or other processing units. The processing units may be configured to communicate with the memory components and I/O components. For example, the processing units may be configured to communicate with the memory components and I/O components via the interconnect components.


A memory component (memory) may include any volatile or non-volatile media. For example, the memory may include electrical media, magnetic media, and/or optical media, such as a random access memory (RAM), read-only memory (ROM), non-volatile RAM (NVRAM), electrically-erasable programmable ROM (EEPROM), Flash memory, hard disk drives (HDD), magnetic tape drives, optical storage technology (e.g., compact disc, digital versatile disc, and/or Blu-ray disc), or any other memory components. The memory components may include (e.g., store) the data described herein. For example, the memory components may store the data included in the app records 400A of the search data store 112 and/or the data included in (e.g., connection records of) the connection data store 118. The memory components may also include instructions executed by the processing units. For example, the memory components may include computer-readable instructions that, when executed by the processing units, cause the units to perform the various functions attributed to the modules and data stores described herein.


The I/O components may refer to electronic hardware, software, and/or firmware that provide communication with a variety of different devices. For example, the I/O components may provide communication between other devices and the processing units and memory components. In some examples, the I/O components may be configured to communicate with a computer network. For example, the I/O components may be configured to exchange data over a computer network using a variety of different physical connections, wireless connections, and protocols. The I/O components may include network interface components (e.g., a network interface controller), repeaters, network bridges, network switches, routers, and firewalls. In some examples, the I/O components may include hardware, software, and/or firmware configured to communicate with various human interface devices, including display screens, keyboards, pointer devices (e.g., a mouse), touchscreens, speakers, and microphones. In other examples, the I/O components may include hardware, software, and/or firmware configured to communicate with additional devices, such as external memory (e.g., external HDDs).


In some examples, the search system 100 and/or analysis system 108 may be a system of one or more computing devices (e.g., a computer search system) configured to implement the techniques described herein. Put another way, the features attributed to the modules and data stores described herein may be implemented by one or more computing devices. Each computing device may include any combination of electronic hardware, software, and/or firmware described herein. For example, each computing device may include any combination of the one or more processing units, memory components, I/O components, and interconnect components described herein. The computing devices may also include various human interface devices, including display screens, keyboards, pointing devices (e.g., a mouse), touchscreens, speakers, and microphones. The computing devices may also be configured to communicate with additional devices, such as external memory (e.g., external HDDs).


The computing devices of the search system 100 and/or analysis system 108 may be configured to communicate with the network 106. The computing devices may also be configured to communicate with one another via a computer network. In some examples, the computing devices may include one or more server computing devices configured to communicate with the user device(s) 102 (e.g., receive search queries 210 and app interaction data 216, and transmit search results 220), gather data from the data source(s) 104, index the data, store the data, and store other documents. In other examples, the computing devices may reside within a single machine at a single geographic location, within multiple machines at a single geographic location, or be distributed across a number of geographic locations.


Additionally, the various implementations of the search system 100 and analysis system 108 described herein (e.g., using one or more computing devices that include one or more processing units, memory components, I/O components, and interconnect components) are equally applicable to any of the user device(s) 102, as well as to the various components thereof.

Claims
  • 1. A method comprising: for each of a plurality of application (app) records each specifying a native app and including an app download address (ADA) for downloading the app, determining one or more connections associated with the app, and generating a quality value indicating a degree of quality associated with the app based on the one or more connections;receiving a search query from a user device;identifying one or more of the plurality of app records based on the search query and based on the quality value associated with each identified record;selecting the one or more ADAs from the identified one or more of the plurality of app records; andtransmitting the one or more ADAs to the user device.
  • 2. The method of claim 1, wherein the one or more connections associated with the native app specified by at least one of the plurality of app records comprise one or more of the following: an outbound link configured to enable the native app to access another resource; andan inbound link configured to enable another resource to access the native app.
  • 3. The method of claim 1, wherein the one or more connections associated with the native app specified by at least one of the plurality of app records comprise one or more of the following: a link between the native app and another native app;a link between the native app and a native app programming interface (API);a native app library included in the native app; anda link between the native app and a web resource.
  • 4. The method of claim 1, wherein determining the one or more connections associated with the native app specified by at least one of the plurality of app records comprises performing a static connection analysis including identifying one or more software instructions associated with the app that cause the app to communicate with another resource.
  • 5. The method of claim 1, wherein determining the one or more connections associated with the native app specified by at least one of the plurality of app records comprises performing a dynamic connection analysis including detecting that the app is communicating with another resource.
  • 6. The method of claim 1, wherein generating the quality value based on the one or more connections associated with the native app specified by at least one of the plurality of app records comprises: generating a set of one or more rules including one or more software instructions configured to compute the quality value based on the one or more connections;applying the set of one or more rules to the one or more connections; andin response to applying the set of one or more rules, computing the quality value.
  • 7. The method of claim 6, wherein generating the set of one or more rules comprises: receiving an indication of one or more of a degree of quality a user perceives to be associated with a connection associated with a native app, a degree of quality the user perceives to be associated with the app, whether the user has selected a search result specifying the app, and how often the user has selected a search result specifying the app; andgenerating the set of one or more rules based on the received indication.
  • 8. The method of claim 1, wherein generating the quality value based on the one or more connections associated with the native app specified by at least one of the plurality of app records comprises: generating training data including an indication of one or more training connections associated with each of one or more training native apps and one or more training quality values each indicating a degree of quality associated with one of the one or more training native apps;generating a machine-learned model based on the training data, wherein the machine-learned model includes one or more software instructions configured to compute the quality value based on the one or more connections;providing an indication of the one or more connections to the machine-learned model as one or more inputs; andin response to providing the indication, receiving the quality value from the machine-learned model.
  • 9. The method of claim 8, wherein generating the training data comprises: receiving an indication of one or more of a degree of quality a user perceives to be associated with a connection associated with a native app, a degree of quality the user perceives to be associated with the app, whether the user has selected a search result specifying the app, and how often the user has selected a search result specifying the app; andgenerating the training data based on the received indication.
  • 10. The method of claim 1, wherein identifying at least one of the one or more of the plurality of app records based on the quality value associated with the record comprises using the quality value as a boost factor within Lucene® information retrieval software developed by the Apache Software Foundation.
  • 11. The method of claim 1, wherein identifying at least one of the one or more of the plurality of app records based on the quality value associated with the record comprises determining that the quality value is greater than a threshold value.
  • 12. The method of claim 1, wherein each of the plurality of app records further includes one or more app attributes (AAs) that describe the native app specified by the record, and wherein identifying the one or more of the plurality of app records based on the search query comprises identifying each record based on one or more matches between one or more terms of the search query and one or more terms of the one or more AAs included in the identified record.
  • 13. A method comprising: for each of a plurality of application (app) records each specifying a native app and including an app download address (ADA) for downloading the app, determining one or more connections associated with the app, and generating a quality value indicating a degree of quality associated with the app based on the one or more connections;receiving a search query from a user device;identifying one or more of the plurality of app records based on the search query;for each of the identified one or more of the plurality of app records, generating a result score based on the quality value associated with the record;selecting one or more app records from the identified one or more of the plurality of app records based on the result score associated with each selected record;selecting the one or more ADAs from the selected one or more app records; andtransmitting the one or more ADAs to the user device.
  • 14. The method of claim 13, wherein the one or more connections associated with the native app specified by at least one of the plurality of app records comprise one or more of the following: an outbound link configured to enable the native app to access another resource;an inbound link configured to enable another resource to access the native app;a link between the native app and another native app;a link between the native app and a native app programming interface (API);a native app library included in the native app; anda link between the native app and a web resource.
  • 15. The method of claim 13, wherein determining the one or more connections associated with the native app specified by at least one of the plurality of app records comprises performing one or more of a static connection analysis including identifying one or more software instructions associated with the app that cause the app to communicate with another resource, and a dynamic connection analysis including detecting that the app is communicating with another resource.
  • 16. The method of claim 13, wherein generating the quality value based on the one or more connections associated with the native app specified by at least one of the plurality of app records comprises performing one or more of the following: generating a set of one or more rules including one or more software instructions configured to compute the quality value based on the one or more connections, applying the set of one or more rules to the one or more connections, and, in response to applying the set of one or more rules, computing the quality value; andgenerating training data including an indication of one or more training connections associated with each of one or more training native apps and one or more training quality values each indicating a degree of quality associated with one of the one or more training native apps, generating a machine-learned model based on the training data, wherein the machine-learned model includes one or more software instructions configured to compute the quality value based on the one or more connections, providing an indication of the one or more connections to the machine-learned model as one or more inputs, and, in response to providing the indication, receiving the quality value from the machine-learned model.
  • 17. The method of claim 13, wherein generating the result score for at least one of the identified one or more of the plurality of app records based on the quality value associated with the record comprises: generating training data including, for each of one or more training native apps, a training quality value and a training result score each associated with the training native app;generating a machine-learned relevance (MLR) model based on the training data, wherein the MLR model includes one or more software instructions configured to compute the result score based on the quality value;providing the quality value to the MLR model as an input; andin response to providing the quality value, receiving the result score from the MLR model.
  • 18. The method of claim 13, wherein each of the plurality of app records further includes one or more app attributes (AAs) that describe the native app specified by the record, and wherein identifying the one or more of the plurality of app records based on the search query comprises identifying each record based on one or more matches between one or more terms of the search query and one or more terms of the one or more AAs included in the identified record.
  • 19. A system comprising one or more computing devices configured to: for each of a plurality of application (app) records each specifying a native app and including an app download address (ADA) for downloading the app, determine one or more connections associated with the app, and generate a quality value indicating a degree of quality associated with the app based on the one or more connections;receive a search query from a user device;identify one or more of the plurality of app records based on the search query and based on the quality value associated with each identified record;select the one or more ADAs from the identified one or more of the plurality of app records; andtransmit the one or more ADAs to the user device.
  • 20. A system comprising one or more computing devices configured to: for each of a plurality of application (app) records each specifying a native app and including an app download address (ADA) for downloading the app, determine one or more connections associated with the app, and generate a quality value indicating a degree of quality associated with the app based on the one or more connections;receive a search query from a user device;identify one or more of the plurality of app records based on the search query;for each of the identified one or more of the plurality of app records, generate a result score based on the quality value associated with the record;select one or more app records from the identified one or more of the plurality of app records based on the result score associated with each selected record;select the one or more ADAs from the selected one or more app records; andtransmit the one or more ADAs to the user device.