Patent Application
20140372412
Search engines utilize index data structures to perform user search queries for search results. Index data structures index documents in a single index order. A user may require that search results be returned quickly with the search results identifying the most relevant matching documents sorted in a different order. However, conventional index data structures and related algorithms fail to provide search engines with an index data structure that supports identifying the most relevant documents ordered based on an attribute different from the ordering of the documents within the index data structure. As such, the most relevant documents for a particular attribute sort order may not be returned, particularly in cases with a constrained time period for performing the search query.
This summary is provided to introduce a selection of concepts in a simplified form that are further described below in the detailed description. This summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used in isolation as an aid in determining the scope of the claimed subject matter.
Embodiments of the present invention provide a method and system for, among other things, using dynamic filtering to search content items associated with augmented indexes. This may be accomplished by receiving a search query. The search query includes instructions to sort the results by some attribute; the ordering of results by this attribute is different from the ordering of content items within the search index. A plurality of search query elements for the search query is identified. The plurality of search query elements includes the sort attribute having one or more attribute-range-elements each associated with an attribute-range of the sort attribute. An augmented index for searching the query elements is referenced. The augmented index comprises the one or more attribute-range-elements that map to one or more content items having an attribute-value in the attribute-range of the one or more attribute-range elements. Matching content items that satisfy the search query are identified. Identifying the matching content items comprises: selecting matching content items in the augmented index based on the plurality of search query elements determined for the search query; and dynamically filtering the matching content items into a matched-group using the one or more attribute-range-elements mapping to the matching content items. The matching content items of the search query are provided as search results.
The present invention is described in detail below with reference to the attached drawing figures, wherein:
The subject matter of embodiments of the invention is described with specificity herein to meet statutory requirements. However, the description itself is not intended to limit the scope of this patent. Rather, the inventors have contemplated that the claimed subject matter might also be embodied in other ways, to include different steps or combinations of steps similar to the ones described in this document, in conjunction with other present or future technologies. Moreover, although the terms “step” and/or “block” may be used herein to connote different elements of methods employed, the terms should not be interpreted as implying any particular order among or between various steps herein disclosed unless and except when the order of individual steps is explicitly described.
For purposes of this disclosure, the word “including” has the same broad meaning as the word “comprising.” In addition, words such as “a” and “an,” unless otherwise indicated to the contrary, include the plural as well as the singular. Thus, for example, the requirement of “a feature” is satisfied where one or more features are present. Also, the term “or” includes the conjunctive, the disjunctive and both (a or b thus includes either a or b, as well as a and b).
For purposes of a detailed discussion below, embodiments of the present invention are described with reference to, for exemplary purposes, a search engine in the context of the searching for mobile device apps. Further, while embodiments of the present invention may generally refer to searching and the components described herein, it is understood that the techniques described may be extended to other implementations contexts performing the steps described herein.
In this description “content items” may refer to information retrieved based on keywords, queries, or their reformulations. The term “content items” is defined to include typical documents that are retrieved based on queries. More generally, content items include retrieved information resources and aids in identifying additional resources. Content items may be retrieved using an index data structure. Search engines maintain index data structures of content items that are matched to search queries in order to receive content items as search results.
Search engines are generally designed to search index data structures for search results relevant to a search query. Index data structures may include indexed documents i.e., sorted and/or ordered by some rank, for example, a static rank representing an approximation of the likelihood that the content item would be returned in a search query. This static rank may also be referred to as the index attribute or index ordering. A search engine may perform a search on an index data structure based on loose constraints and return a large number of search results (e.g., matching documents). Returning the matching documents may be optimized by implementing a time limit. The matching documents are identified in order of the index ordering, ranked by some unspecified algorithm, the top ranked documents selected and then re-ordered by this rank value, and then the documents are returned as search results; typically a user browses through only a handful of the matching documents in the search results. Further, if the search query requires ordering the matching documents based on a different attribute of the content items, search engines may first identify matching documents based on the index ordering and then separately sort based on the particular attribute to provide search results. With the time constraints imposed on search queries, not every matching document is examined and ranked. In this case, after sorting the partial results by the specified sort attribute, there may be visible missing results even in the top results that the user can be expected to see. To avoid this problem with traditional methods, all matching documents must be identified and retained prior to resorting by the attribute. This may lead to computationally expensive, slow searches.
In various embodiments, systems and methods are provided for searching content items using dynamic filtering. The systems and methods may allow for searching of content items in an index data structure. As discussed, the index data structure indexes the content items based on a single ordering on some attribute. The index data structure may be implemented as an inverted index with posting lists. Content items indexed in the index data structure may be associated with a sort attribute. The sort attribute is different from the index ordering of the index data structure; however search results may be efficiently returned based on the sort attribute in embodiments of the present invention. At a basic level, the sort attribute may have a range of attribute values identified for all content items. Multiple synthetic elements (e.g., attribute-range-elements) associated with different ranges (e.g., attribute-ranges) may be then be added into the inverted index. The synthetic elements map to content items that match the attribute-range based on a corresponding attribute value of the content item. Using dynamic filtering, content items are identified using the attribute-range-elements of the search query. In particular, dynamic filtering periodically updates the attribute-range-elements that are searched based on the search query attribute sort order and a target count of content items to be identified. In this regard, only the most relevant documents are identified for processing.
To further illustrate embodiments of the present invention by example, an index data structure may include content items (e.g., apps for a mobile device). Each content item may include several different attributes (e.g., popularity, title, rating, category, number of downloads, and price). An attribute may be an integer attribute, string, alphanumeric, or some other like formats. A user may search the content items to identify search results based on the Price, in ascending order. In this regard, a search query (e.g., “CARD GAME”—sorted based on ascending Price) may be received. It is contemplated that the search query may be received via an interface that provides a selection option for a sort attribute. The search query and any additional input may be parsed for query elements (e.g., search query terms). In particular, a sort attribute different from the index attribute of the search index is identified. As discussed below, the sort attribute is associated with synthetic elements i.e., one or more attribute-range-elements having an attribute-range of the sort attribute values of the content items. In embodiments, an expression of the search query may be reformulated to perform the search (e.g., [GAME]+[CARD]). The sort attribute may also be a reformulated query expression (e.g., [SORT]=[ASC]: [PRICE]).
Further, an augmented index (e.g., inverted index with synthetic elements in a posting list) of the content items may be utilized to execute the search query. The content items are partitioned into subsets of the Price (e.g., discrete groups Gi). The partitioned content items are each associated with an attribute-range-element associated with an attribute-range. For example, the inverted index may include attribute-range-elements for Price values from $0-$MAX. The Price attribute may be associated with several different attribute-range-elements e.g., _group—0 [$0 to $99] _group—1 [$100 to $199] and _group—2 [$200 to $MAX]. It is contemplated that the attribute-range-elements are inserted as additional posting lists into the inverted index. Other methods of associating the attribute-range-elements with the inverted index are contemplated within the scope of this invention. The entries in an attribute-range-element posting list map to all content items that have an attribute-value that corresponds to a value in the range of the attribute-range-element. For instance, _group—0 [$0 to $99] entries map to all content items with Price values between $0 and $99. It is contemplated that the attribute-range-element of a content item may be tagged to the content item attribute.
In executing the search query i.e., identifying matching content items in the posting list, a matched content item is assigned to a matched-group using the one or more attribute-range-elements mapping to the matching content items during a process referred to as dynamic filtering. As such, embodiments of the present invention further dynamically filter content items into a matched-group based on the attribute-range-element during execution of the query. Content items that are identified for the search query are filtered using the attribute value of the content items that corresponds to the sort attribute of the search query. The attribute-range-elements in the index, each associated with an attribute range, facilitate this process with the mapping to the content items.
In operation, the dynamic filtering process may be associated with a target count. The target count designates how many content items should be identified. As content items are matched, the matched count is incremented. When the target count is reached, the inclusive range of attribute values encompassed by the content items matched so far is examined, and a determination is made to stop matching on attribute-range-elements that do not overlap this range. As such, more relevant documents are matched throughout the dynamic filtering process. This process of updating the attribute-range-elements that are included in the dynamic filtering process is repeated periodically as additional content items are matched, and over time the content items included in the dynamic filter converge towards an optimal set. This additional filtering may cause a significant reduction in the total number of documents matched by the query, and if that happens, the chances of the query timing out before retrieving all relevant matched documents is greatly reduced.
Accordingly, in a first aspect of the present invention, one or more computer-readable media storing computer-useable instructions that, when used by one or more computing devices, cause the one or more computing devices to perform a method for searching content items associated with augmented indexes, using dynamic filtering. The method includes receiving a search query, the search query is associated with a sort attribute. The method further includes identifying a plurality of search query elements for the search query, the plurality of search query elements includes the sort attribute having one or more attribute-range-elements each associated with an attribute-range of the sort attribute. The method also includes referencing an augmented index for searching the query elements, the augmented index comprises the one or more attribute-range-elements that map to one or more content items having an attribute-value in the attribute-range of the one or more attribute-range elements. The method further includes identifying matching content items that satisfy the search query; identifying the matching content items comprises: (a) selecting matching content items in the augmented index based on the plurality of search query elements determined for the search query; and (b) dynamically filtering the matching content items into a matched-group using the one or more attribute-range-elements mapping to the matching content items. The method also includes providing the matching content items of the search query as search results.
In a second aspect of the present invention, a system for searching content items associated with augmented indexes, using dynamic filtering. The system includes a query receiving component for receiving a search query, the query is associated with a sort attribute. The system also includes a parsing component for: identifying a plurality of search query elements for the search query, the plurality of search query elements includes the sort attribute having one or more attribute-range-elements each associated with an attribute-range of the sort attribute. The parsing component is also configured for: reformulating the search query with one or more syntax elements that support dynamic filtering. The system further includes a data store for storing the content items. The system also includes a query execution component for: referencing an augmented index for searching the query elements, the augmented index comprises the one or more attribute-range-elements that map to one or more content items having an attribute-value in the attribute-range of the one or more attribute-range-elements. The query execution component is also configured for: identifying matching content items that satisfy the search query based on selecting matching content items in the augmented index that are dynamically filtered into a matched-group based on the one or more attribute-range-elements mapping to the matching content items. The system also includes a presenting component for: providing for display the matching content items as search results.
In a third aspect of the present invention, one or more computer-readable media storing computer-useable instructions that, when used by one or more computing devices, cause the one or more computing devices to perform a method for searching content items associated with augmented indexes. The method includes receiving a search query; the search query is associated with a sort attribute, the sort attribute is different from an index attribute. The method also includes identifying a plurality of search query elements for the search query, the plurality of search query elements includes the sort attribute having one or more attribute-range-elements each associated with an attribute-range of the sort attribute. The method further includes reformulating the search query to include one or more syntax elements that support dynamic filtering. The method also includes referencing an augmented index for searching the query elements, the augmented index comprises the one or more attribute-range-elements that map to one or more content items having an attribute-value in the attribute-range of the one or more attribute-range-elements. Each of the attribute-range-elements is associated with an attribute-range based on predefined ranges derived from the attribute-values of the one or more content items. The method further includes identifying matching content items that satisfy the search query, identifying the matching content items comprises: (a) selecting matching content items in the augmented index based on the plurality of search query elements determined for the search query; and (b) dynamically filtering the matching content items into a matched-group using the one or more attribute-range-elements mapping to the matching content items. The method also includes providing a target count of matching content items from the matched-group as search results for the search query, wherein the matching content items in the target count are sorted based on the sort attribute.
Having briefly described an overview of embodiments of the present invention, an exemplary operating environment in which embodiments of the present invention may be implemented is described below in order to provide a general context for various aspects of the present invention. Referring initially to
The invention may be described in the general context of computer code or machine-useable instructions, including computer-executable instructions such as program modules, being executed by a computer or other machine, such as a personal data assistant or other handheld device. Generally, program modules including routines, programs, objects, components, data structures, etc., refer to code that perform particular tasks or implement particular abstract data types. The invention may be practiced in a variety of system configurations, including hand-held devices, consumer electronics, general-purpose computers, more specialty computing devices, etc. The invention may also be practiced in distributed computing environments where tasks are performed by remote-processing devices that are linked through a communications network.
With reference to
Computing device 100 typically includes a variety of computer-readable media. Computer-readable media can be any available media that can be accessed by computing device 100 and includes both volatile and nonvolatile media, removable and non-removable media. By way of example, and not limitation, computer-readable media may comprise computer storage media and communication media.
Computer storage media include volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer-readable instructions, data structures, program modules or other data. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by computing device 100. Computer storage media excludes signals per se.
Communication media typically embodies computer-readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media. The term “modulated data signal” means a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal. By way of example, and not limitation, communication media includes wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, RF, infrared and other wireless media. Combinations of any of the above should also be included within the scope of computer-readable media.
Memory 112 includes computer storage media in the form of volatile and/or nonvolatile memory. The memory may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state memory, hard drives, optical-disc drives, etc. Computing device 100 includes one or more processors that read data from various entities such as memory 112 or I/O components 120. Presentation component(s) 116 present data indications to a user or other device. Exemplary presentation components include a display device, speaker, printing component, vibrating component, etc.
I/O ports 118 allow computing device 100 to be logically coupled to other devices including I/O components 120, some of which may be built in. Illustrative components include a microphone, joystick, game pad, satellite dish, scanner, printer, wireless device, etc.
With additional reference to
In some embodiments, one or more of the illustrated components/modules may be implemented as stand-alone applications. In other embodiments, one or more of the illustrated components/modules may be implemented with the index server node 230, as an Internet-based service. Any number of client computing devices, search engine interfaces, index server nodes and search engine interface components may be employed in the computing system 200 within the scope of embodiments of the present invention. Each may comprise a single device/interface or multiple devices/interfaces cooperating in a distributed environment. For instance, the index server node 230 may comprise multiple devices and/or modules arranged in a distributed environment that collectively provide the functionality of the index server node 230 described herein. Additionally, other components/modules not shown also may be included within the computing system 200.
It should be understood that this and other arrangements described herein are set forth only as examples. Other arrangements and elements (e.g., machines, interfaces, functions, orders, and groupings of functions, etc.) can be used in addition to or instead of those shown, and some elements may be omitted altogether. Further, many of the elements described herein are functional entities that may be implemented as discrete or distributed components or in conjunction with other components, and in any suitable combination and location. Various functions described herein as being performed by one or more entities may be carried out by hardware, firmware, and/or software. For instance, various functions may be carried out by a processor executing instructions stored in memory.
With continued reference to
The index server node 230 of
In embodiments, several different index server nodes (e.g., index server node 230) may support searching the content items in the data store 250 associated with the index server 230 node. In this regard, an embodiment of the present invention may include a plurality of index server nodes each associated with its own data store, where a search query is processed at the index server node for the portion of the content items within the data store 250 of the index server node. The portion of the content items may be searched via an index that may also be stored in the data store 250. The index server node 230 may further process a search query relative to the content items for that index server node 230. Thus index server node 230 may generate content items (e.g., search results) that are intended for aggregation with results from other similar search services as discussed later in more detail.
The query receiving component 232 of the index server node 230 is configured to receive requests for presentation of content items (e.g., algorithmically-identified search results) that satisfy an input search query. Typically, such a request is received via the browser 212, search engine interface 220, aggregation component 270, or combination thereof. For example, with reference to
The parsing component 234 is configured to analyze and expand an expression of the search query for supporting dynamic filtering. The parsing component 234 may receive the search query with the additional search query elements that define the scope and structure of the search. For example, a user at the client computing device 210 may identify a sort attribute upon which a search query is performed and also provide the order (e.g., descending or ascending) in which the content items are presented. The search query elements selections may be identified explicitly through user interface elements on the computing device. Several different types of search query elements that are used to dynamically filter content items are contemplated with embodiments of the present invention. The parsing component 230 may also identify the search query elements based on implicit associations with the search query. A person of ordinary skill in the art may recognize several different methods of explicit and implicit search query elements based on a received search query.
In embodiments, the parsing component 230 may further provide syntax elements in the search query to form a reformulated expression that supports different aspects of the invention. The reformulated expression syntax may support, among other things, sorting and partitioning for dynamic filtering. The search query may be reformulated to include syntax elements for search query elements (e.g., sort attributes, search ordering, and ranges). With reference to
Further in embodiments, the reformulated expression syntax may include the path to the range mapping data structure that contains range mappings to attribute-range-elements as discussed in detail later. For example, a path (e.g., a file path) to the data structure may be implemented as a syntax element when executing the query. It is contemplated that parts of the syntax format may be optionally entered as required by the search. As shown, several types of syntax elements may be appended to the search query such that the query expression includes additional information for executing the query, and in particular dynamically filtering the identified content items. Any and all such information, and any combination thereof, is considered within the scope of embodiments of the present invention.
The query execution component 236 is configured to determine one or more matching content items relevant to the received query. The query execution component 236 includes the attribute range mapper 238 and the dynamic filter 240. The attribute range mapper 238 is configured to map attribute-values of the content items to specific attribute-range-elements. In embodiments, the attribute-range-element is included as an additional posting list within the inverted index and maps to each content item that has an attribute-value that corresponds to an attribute-value in the range of the attribute-range-element. Attribute range mapping occurs online while executing the search query. Attribute range mapping also occurs offline or online while indexing the content items. Range mapping may also occur by analyzing attribute values of the content items during index load time. Embodiments of the present invention may advantageously also use the attribute range mapper 238 to predefine ranges based on the attribute-value of content items in the index data structure. For example, the distribution of attribute-values, may determine how each range is defined. The predefined range may be defined in a data structure the include mappings of contiguous, distinct attribute-values to attribute-range-elements. The data structure may be implemented as a file, database, or other range mapping service. Any variations and combinations thereof may are contemplated with embodiments of the present invention. The data structure of mappings may provide a serialized representation of the predefined ranges. Content items having attribute-values defined in the data structure may be tagged accordingly, as such; during execution of the search query the tags facilitate dynamic filtering of the matched content items into a matched-group.
It is contemplated that the attribute range mapper 238 may use a secondary or tertiary key to be part of a sort when an attribute cluster exists where several different content items share the same attribute value. For example, the $0.00 to $0.99 price point may be a popular price point for mobile device apps, which then creates a cluster of contents items at that range. The secondary or tertiary key may be a unique key that facilitates further distinguishing the content items. As such, the content items are still returned in some deterministic and consistent order.
The dynamic filter 240 is generally responsible for identifying matching content items. The dynamic filter 240 assigns matching content items to a matched-group using the one or more attribute-range-elements mapping to the matching content items. As discussed, the attribute-range-elements map to one or more content items based on an attribute-value of the content item that corresponds to an attribute range of the attribute-range-element. The dynamic filter 240 may further be associated with a target count. The target count designates how many content items should be identified. The dynamic filter is configured to determine when to stop matching on attribute-range-elements (e.g., irrelevant attribute-range-elements) that do not overlap a range relevant to the search query based on reaching the target count. As discussed further below, excluding matching on irrelevant attribute-range-elements limits the matching to content items most relevant to the search query. The dynamic filter 240 may perform this function periodically as additional content items are matched.
With reference to
In operation, the dynamic filter (e.g., dynamic filter 240) has all the attribute-range-elements. In
The dynamic filter further continues matching and matches content item—DOC ID #7 on the search terms and _GROUP—0. The dynamic filter again makes a determination to drop _GROUP—1 is from the further matching. _GROUP—1 also is added to the set of irrelevant attribute-range-elements. Matching the content item continues with content item—DOC ID #16 of _GROUP —0 and content item—DOC ID #22 of GROUP—0. As shown in the matched-group, the dynamic filter technique progressively converged into the matched-group an optimal set of content items most relevant to the search query content items—DOC ID #1, 16, 7, 22, and 3 in contrast to matching all content items—DOC ID #1, 16, 7, 22, 3, 12, 19, and 21. The matched-group may be sorted and order by the sort attribute i.e., Price in ascending order. Content item—DOC ID #12, content item—DOC ID #19, and content item—DOC ID #21 each match on the search query elements; however these content items are excluded from processing because they all match on the irrelevant attribute-range-items _GROUP—2 which was removed from processing. Thus, the dynamic filtering process resulted in a reduction in the total number of content items matched by the query, matching only on the most relevant content items, based on the sort attribute and order. In particular, instead of matching on 8 content items out of 22 (approximately 36%) with dynamic filtering the dynamic filter matched 5 content items out of 22 (approximately 23%). The addition of dynamic filtering, limits the number of content items that the index server ultimately matches. In search engines with many more groups and content items this translates to significant saving in costs. Further, the top two contents may be further communicated for additional processing, for example, in cases with more than one index node as discussed further below.
With reference to
Turning to
It is further contemplated that the aggregation component 270 may also provide additional functionality to align content items across attribute ranges. Index server nodes may provide results that may not be aligned across attribute ranges and that result in missing values in the results. For, example,
With continued reference to
Turning now to
Turning now to
At block 818, matching content items that satisfy the search query are identified. Identifying the matching content items comprises: at block 820, selecting matching content items in the augmented index based on the plurality of search query elements determined for the search query. In embodiments, selecting matching content items further includes identifying the search query elements that are search query terms of the search query; identifying the search query elements that are attribute-range-elements of the search query, the attribute-range-elements associated with the sort attribute; and selecting matching content items based on an algorithm finding matching content items for both the search query terms and the attribute-range-elements.
Identifying the matching content items also comprises: at block 822, dynamically filtering the matching content items into a matched-group using the one or more attribute-range-elements mapping to the matching content items. In embodiments, dynamic filtering that matching content items further includes: identifying one or more excluded attribute-range-elements; and excluding one or more content items from dynamic filtering based on the excluded attribute-range-elements. At block 824 a target count of matching content items is provided as search results for the search query.
Embodiments of the present invention have been described in relation to particular embodiments which are intended in all respects to be illustrative rather than restrictive. Alternative embodiments will become apparent to those of ordinary skill in the art to which the present invention pertains without departing from its scope.
From the foregoing, it will be seen that this invention in one well adapted to attain all the ends and objects hereinabove set forth together with other advantages which are obvious and which are inherent to the structure.
It will be understood that certain features and sub-combinations are of utility and may be employed without reference to other features or sub-combinations. This is contemplated by and is within the scope of the claims.
