The present application claims the benefit and priority of Chinese Patent Application No. 201110043030.7, filed on Feb. 18, 2011 and named “method and apparatus for clustering search terms”. The entire disclosures of the previous Chinese application are incorporated herein by reference.
The present invention relates to network search technology, and particularly to a method and apparatus for clustering search terms.
In network search technology, a user usually searches out a result through a corresponding search term. In a bid advertising system, the search term may be an identifier of an advertisement provided by an advertiser, and be referred to as a purchase word. The purpose is to facilitate the user to search out the corresponding advertisement through the search term.
In the bid advertising system, in order to improve the advertisement display quality, it is necessary to cluster search terms provided by the advertiser. A process for clustering the search terms can be abstracted as a process for performing clustering to a set of short text strings.
Currently, the most commonly-used method for clustering includes operations as follows: for a search term provided by an advertiser, search terms which are the most literally similar to the provided search term are searched out from existing search terms provided by all advertisers, and the search term provided by the advertiser is clustered together with the searched out search terms. As such, when a user of a search engine retrieves a corresponding advertisement through a search term, the advertisement corresponding to the search term are displayed to the user together with advertisements corresponding to search terms clustered with the search term.
However, there are some search terms that substantially relate to the advertisement corresponding to the search term provided by the advertiser although the search terms are not provided by the advertisers. The aforesaid method for clustering is just to literally cluster the search terms provided by the advertiser without considering other search terms which semantically relate to the search term provided by the advertiser and have not currently been provided by the advertiser, thereby reducing the accuracy of clustering search terms.
A method and apparatus for clustering search terms are provided by the present invention, so as to improve the accuracy and relevance of clustering the search terms.
A technical solution provided by the present invention includes:
A method for clustering search terms includes:
establishing a candidate search term set, wherein the candidate search term set includes a first search term provided by a user, and a second search term related to the first search term; and
performing a clustering operation on the first search term and the second search term related to the first search term in the candidate search term set according to text characteristic and/or semantic characteristic of search term.
An apparatus for clustering search terms includes:
an establishing unit, to establish a candidate search term set, wherein the candidate search term set includes a first search term provided by a user, and a second search term related to the first search term; and
a clustering unit, to perform a clustering operation on the first search term and the second search term related to the first search term in the candidate search term set according to text characteristic and/or semantic characteristic of search term.
As can be seen from the above technical solution, in the method and apparatus provided by embodiments of the present invention, when search terms are clustered, a search term provided by a user and other search terms related to the search term provided by the user are taken into account rather than only performing the clustering for the search term provided by the user just according to a literal relationship in prior art, and the clustering is performed for the search term provided by the user and other search terms related to the search term provided by the user according to text characteristic and/or semantic characteristic of search term, which obviously increases the accuracy and relevance of search term clustering.
a is a flowchart illustrating a process of step 102 in accordance with an embodiment of the present invention;
b is a flowchart illustrating a process for exploiting a potential clustering relationship in accordance with an embodiment of the present invention;
a is a schematic diagram illustrating a first structure of a topological graph among search terms in accordance with an embodiment of the present invention;
b is a schematic diagram illustrating a second structure of a topological graph among search terms in accordance with an embodiment of the present invention;
c is a schematic diagram illustrating a potential clustering relationship among search terms in accordance with an embodiment of the present invention;
d is a schematic diagram illustrating a third structure of a topological graph when a search term is added in accordance with an embodiment of the present invention;
Hereinafter, the present invention will be described in further detail with reference to the accompanying drawings and examples to make the objective, technical solution and merits therein clearer.
In the present invention, when search terms are clustered, a search term provided by a user like an advertiser is clustered together with search terms related to the search term according to the text characteristic and/or the semantic characteristic of search term rather than is clustered just according to a literal relationship as in conventional technologies, so that the accuracy of clustering search terms is improved. A method provided by an embodiment of the present invention is described hereinafter.
In step 101, a candidate search term set is established. The candidate search term set includes a first search term provided by a user, and a second search term related to the first search term.
In step 101, the second search term related to the first search term may be specifically determined according to any one of two ways shown as follows. In a first way, a search term matching the first search term provided by the user is determined, and the determined search term is determined as the second search term related to the first search term; in a second way, the first search term provided by the user is taken as a keyword word for search, and a search term in the search result is determined as the second search term related to the first search term provided by the user.
The search term obtained through the first way may be a search term obtained through performing a simple string conversion for the first search term provided by the user, or may be a search term that usually used together with the first search term, which is determined based on actual experiences. For example, if the first search term provided by the user is a coffee pot, based on experiences, it may know that the coffee pot is usually used together with a coffee mug and so on. Based on this, it may be determined that the search term matching the coffee pot provided by the user may be the coffee mug and so on.
Specifically, the search term obtained through the second way may be a search term in a search result when the first search term provided by the user is taken as a keyword for search. The search may be implemented through a user Query Bidterm Merge (QBM). In a specific implementation, the QBM may be as follows: taking the first search term provided by the user as an input for search; obtaining the search term from the search result; determining the obtained search term as the search term related to the first search term provided by the user.
So far, the candidate search term set may be obtained through step 101. It should be noted that in the embodiment of the present invention, it is necessary to ensure that there are not any repeated search terms in the candidate search term set obtained in step 101.
In step 102, a clustering operation is performed for the first search term and the second search term related to the first search term in the candidate search term set according to text characteristic and/or semantic characteristic of search term.
When step 102 is implemented, a similarity value between the first search term and the second search term related to the first search term in the candidate search term set may be calculated according to the text characteristic and/or the semantic characteristic of the first search term. The first search term is clustered together with the second search term which has a high similarity value with the first search term. Specifically, step 102 may be illustrated through a flowchart shown in
As shown in
In step 201a, a similarity value between a first search term and each second search term related to the first search term is calculated according to text characteristic and/or semantic characteristic of the first search term.
In step 202a, when the similarity value between the first search term and the second search term is greater than or equal to a first preset threshold, the first search term and the second search term are clustered together.
Through step 202a, the first search term and the second search term may be clustered together, wherein the second search term is related to the first search term and the similarity value between the first search term and the second search term is greater than or equal to the first preset threshold. Therefore, the basic clustering in the present invention can be implemented.
Preferably, in order to ensure a more complete clustering relationship, an embodiment of the present invention also provides a process for exploiting a potential clustering relationship, which may be illustrated through a process shown in
As shown in
In step 201b, second search terms are selected from all of second search terms related to a first search term, wherein a similarity value between the first search term and each selected second search term is greater than or equal to a second preset threshold.
As an extension of the embodiment of the present invention, in order to reduce the complexity for exploiting the potential clustering relationship, step 201b may alternatively be replaced as: selecting the second search terms from all of the second search terms clustered together with the first search term, wherein the similarity value with the first search term and each second search term is greater than or equal to the second preset threshold.
The second preset threshold in step 201b is unrelated with the first preset threshold in step 202a, these two thresholds may be equal, or may be not equal.
In step 202b, a similarity value between any two selected second search terms is calculated. When the calculated similarity value is greater than or equal to the first preset threshold, the two second search terms are clustered together.
The exploitation of the potential clustering relationship can be implemented through steps 201b to 202b.
Thus, in the embodiment of the present invention, a total clustering result may be formed through combining the first search term and the second search term clustered together in step 202a (i.e., the clustering relationship exists between the first search term and the second search term), as well as the second search term clustered together in step 202b. In the embodiment of the present invention, the clustering in step 202a and the clustering in step 202b may be implemented in accordance with an existing machine learning model, and are not specifically limited herein.
To make the process shown in
It is assumed that first search terms provided by a user are b1, b3, b4 and b5, respectively. Second search terms related to b1 are b2, b3 and b4It may be obtained through step 101. Second search terms related to b3 are b5, b6 and b4. Second search terms related to b4 are b7, b8 and b9. A second search term related to b5 is b3. All of the search terms are illustrated by a graph data structure shown in
Thereafter, based on step 201a, it can be obtained that: for b1, according to the text characteristic and/or the semantic characteristic of b1, a similarity value w12 between b1 and b2, a similarity value w13 between b1 and b3, and a similarity value w14 between b1 and b4 are calculated. For b3, according to the text characteristic and/or the semantic characteristic of b3, a similarity value w14 between b3 and b4, a similarity value w35 between b3 and b5, and a similarity value w36 between b3 and b6 are calculated. For b4, according to the text characteristic and/or the semantic characteristic of b4, a similarity value w47 between b4 and b7, a similarity value w48 between b4 and b8, and a similarity value w49 between b4 and b9 are calculated. For b5, a similarity value w53 between b5 and b3 are calculated according to the text characteristic and/or the semantic characteristic of b5.
Afterwards, step 202a is performed for each first search term provided by the user in
In the topological graph shown in
When it is determined that two search terms connected with a dashed line in
In the embodiment of the present invention, clustering relationships among search terms are denoted by a solid line (also called an edge relationship) between two search terms, therefore, only edge relationships may be traversed in the embodiment of the present invention, so that the complexity in the embodiment of the present invention is reduced to O(n+e), wherein n denotes the number of the search terms, and e denotes the number of the edge relationships.
It should be noted that as an extension of the embodiment of the present invention, a potential clustering relationship among second search terms related to a first search term provided by a user and “descendant” nodes of the second search terms within N hops (such as N=3) in
In addition, in a bid advertising system, a candidate search term set is not constant all the time, and search terms may be progressively added to the candidate search term set with the passage of time. For example, at a certain time point, a new first search term provided by a user is added to the candidate search term set. Compared with a previous search term, the newly-added first search term occurs newly. It is necessary to perform a similar clustering operation shown in
As shown in
In step 401, one or more second search terms related to a first search term are determined, the first search term to be added and a second search term are added to a candidate search term set, wherein the second search term is within the determined one or more second search terms and differs from any search term in the candidate search term set.
For example, before step 401 is performed, search terms stored in the candidate search term set are b1 to b9, as shown in
In step 402, the clustering operation is performed on the newly-added first search term and the determined one or more second search terms related to the first search term in the candidate search term set in accordance with text characteristic and/or semantic characteristic of search term.
The clustering operation is similar to the process shown in
Based on step 401, for n1, the second search terms related to n1 are determined as b5 and b6. Thus, when step 402 is to be performed, based on a process shown in
In step 403, a potential clustering relationship is exploited for the one and more second search terms, wherein the one or more second search term are in the candidate search term set and relate to the newly-added first search term.
In step 403, the potential clustering relationship may be exploited according to the process shown in
The newly-added first search term n1 is still taken as an example. The second search terms related to n1 have already been determined as b5 and b6 in step 401. Therefore, when step 403 is to be performed, if a similarity value between b5 and n1 and a similarity value between b6 and n1 are all greater than the second preset threshold, a similarity value between b5 and b6 may be calculated. If the calculated similarity value is greater than or equal to the first preset threshold, the two search terms b5 and b6 are clustered together. Otherwise, b5 and b6 are not clustered together.
So far, a clustering relationship between the newly-added first search term (referred to as an incremental search term) and an existing search term (referred to as an old search term) (referred to hereinafter as an incremental clustering result) may be implemented through above-mentioned steps 401 to 403. The incremental clustering result and the previous existing total clustering result are collectively referred to as a final clustering result in the present invention.
It should be noted that in an embodiment of the present invention, a second search term related to a first search term is not fixed and may be changed according to search term addition or deletion by a user. Based on this, the method provided by the embodiment of the present invention should be able to reflect the change. This change is implemented by periodically updating a candidate search term set (referred to as a total update). The specific implementation is: when a configured total update time arrives, determining the second search term related to the first search term in a candidate search term set, adding both the first search term and the determined second search term related to the first search term to a new candidate search term set, afterwards performing the clustering operation on the first search term and the determined second search term related to the first search term according to the processes as shown in
It is assumed that a first search term provided by a user in the first day is B1, a QBM extension result corresponding to the first search term is Q1=Q(B1), the extension result mainly consists of a a set of second search term related to the first search term. A clustering result is C1=C(Q(B1)), which is obtained by performing clustering for the first search term and the second search term based on the processes shown in
As can be seen from Table 1, the total update starts in the i-th day and ends in the k-th day; in the (k+1)-th (i.e., L-th) day, a synchronization for total data and incremental data are performed, i.e., the process shown in
An apparatus provided by an embodiment of the present invention is hereinafter described.
As shown in
establishing unit 501, to establish a candidate search term set, wherein the candidate search term set includes a first search term provided by a user, and a second search term related to the first search term.
clustering unit 502, to perform a clustering operation on the first search term and the second search term related to the first search term in the candidate search term set according to text characteristic and/or semantic characteristic of search term.
In specific implementation, the apparatus shown in
As shown in
Preferably, as shown in
adding unit 603, to determine one or more second search terms related to the first search term, adding the first search term to be added and a second search term to the candidate search term set when a user adds the first search term, wherein the second search term is within the determined one or more second search terms and differs from any search term in the candidate search term set.
Based on this, clustering unit 602 is further to perform the clustering operation on the newly-added first search term and the one or more second search terms related to the first search term in the candidate search term set according to the text characteristic and/or the semantic characteristic of search term.
Preferably, as shown in
updating unit 604, to determine the second search term related to the first search term in the candidate search term set, add both the first search term and the determined second search term related to the first search term to a new candidate search term set when a configured total update time arrives.
Based on this, clustering unit 602 is further to perform the clustering operation for the first search term and the second search term related to the first search term in the new candidate search term set in accordance with the text characteristic and/or the semantic characteristic of search term.
Specifically, clustering unit 602 performs the clustering operation through the following sub-units:
calculating sub-unit 6021, to calculate a similarity value between a first search term and a second search term related to the first search term in accordance with text characteristic and/or semantic characteristic of the first search term.
clustering sub-unit 6022, to cluster the first search term together with the second search term, when the similarity value between the first search term and the second search term is greater than or equal to a first preset threshold.
Preferably, clustering sub-unit 6022 is further to select second search terms from all of second search terms related to the first search term or from all of seconds search terms clustered with the first search term, wherein a similarity value between the first search term and each second search terms is greater than or equal to a second preset threshold respectively, calculate a similarity value between any two selected second search terms, and cluster the two second search terms together when the calculated similarity value is greater than or equal to the first preset threshold. The first preset threshold is unrelated to the second preset threshold.
The above is the description of the apparatus provided by the embodiment of the present invention.
As can be seen from the above technical solution, in the method and apparatus provided by embodiments of the present invention, when search terms are clustered, a search term provided by a user and another search term related to the search term provided by the user are taken into account rather than only performing clustering of a literal relationship for the search term provided by the user just in prior art. The clustering is performed for the search term provided by the user and the another search term related to the search term provided by the user according to text characteristic and/or semantic characteristic of search term, thereby increasing the accuracy of the search term clustering obviously.
Furthermore, clustering relationships among second search terms related to a first search term provided by the user are exploited in embodiments of the present invention, which can deeply exploit clustering relationships among search terms and make the search term clustering more accurate compared to the prior art.
The above are just preferable embodiments of the present invention, and are not used for limiting the protection scope of the present invention. Any modifications, equivalents, improvements, etc., made under the spirit and principle of the present invention, are all included in the protection scope of the present invention.
Number | Date | Country | Kind |
---|---|---|---|
201110043030.7 | Feb 2011 | CN | national |
Filing Document | Filing Date | Country | Kind | 371c Date |
---|---|---|---|---|
PCT/CN2012/070824 | 2/1/2012 | WO | 00 | 9/4/2013 |