Patent Application
20120330986
The present disclosure relates to an information processing apparatus, an information processing method, and a program and particularly, to an information processing apparatus, an information processing method, and a program that enable information associated with a search keyword to be provided to a user.
Conventionally, in addition to web pages and blogs, the Internet has become flooded with a variety of information using various social networking services (SNS), a representative of which is Twitter. In addition, a system that extracts information including an arbitrary keyword from the variety of information is known.
Specifically, if an existing search system is used, information including a search condition can be provided to a user using a keyword set arbitrarily by the user as the search condition. In addition, new information or frequently searched information can be provided to the user, according to the freshness or the search frequency of the information including the search keyword (for example, refer to Japanese Laid-Open Patent Publication No. 2009-15407).
As described above, the information including the search keyword can be searched in the related art. However, technology for providing information (which may not include a search keyword) associated with the search keyword or extracting information that has become a popular topic in the world from the information associated with the search keyword has not been established.
The present disclosure has been made in view of the above circumstances and enables extraction of information that has become a popular topic in the world.
According to an embodiment of the present disclosure, there is provided an information processing apparatus which includes an evaluation value calculating unit that acquires time-series data of a discrete system including sampling values xi in a measurement period i, calculates a movement deviation vt based on a movement mean mt of N sampling values xt, xt−1, xt−2, . . . , and xt−N+1 corresponding to predetermined periods before a predetermined measurement period t, and calculates an evaluation value st showing a rapid change in the time-series data of the discrete system in the measurement period t, on the basis of the movement deviation vt corresponding to the measurement period t and a movement deviation vt−1 corresponding to a measurement period t−1.
The evaluation value calculating unit may calculate the evaluation value st=movement deviation vt/movement deviation vt−1.
The evaluation value calculating unit may total continuous time-series data for each measurement period and convert the continuous time-series data into the time-series data of the discrete system.
The evaluation value calculating unit may set the measurement periods to temporally overlap and total the continuous time-series data for each measurement period and convert the continuous time-series data into the time-series data of the discrete system.
According to another embodiment of the present disclosure, there is provided an information processing method performed by an information processing apparatus which includes acquiring time-series data of a discrete system including sampling values xi in a measurement period i, calculating a movement deviation vt based on a movement mean mt of N sampling values xt, xt−1, xt−2, . . . , and xt−N+1 corresponding to predetermined periods before a predetermined measurement period t, and calculating an evaluation value st showing a rapid change in the time-series data of the discrete system in the measurement period t, on the basis of the movement deviation vt corresponding to the measurement period t and a movement deviation vt−1 corresponding to a measurement period t−1.
According to another embodiment of the present disclosure, there is provided a program for causing a computer to function as an evaluation value calculating unit that acquires time-series data of a discrete system including sampling values xi in a measurement period i, calculates a movement deviation vt based on a movement mean mt of N sampling values xt, xt−1, xt−2, . . . , and xt−N+1 corresponding to predetermined periods before a predetermined measurement period t, and calculates an evaluation value st showing a rapid change in the time-series data of the discrete system in the measurement period t, on the basis of the movement deviation vt corresponding to the measurement period t and a movement deviation vt−1 corresponding to a measurement period t−1.
According to the embodiments of the present disclosure described above, the time-series data of the discrete system including the sampling values xi in the measurement period i is acquired, the movement deviation vt based on the movement mean mt of the N sampling values xt, xt−1, xt−2, . . . , and xt−N+1 corresponding to the predetermined periods before the predetermined measurement period t is calculated, and the evaluation value st showing the rapid change in the time-series data of the discrete system in the measurement period t is calculated, on the basis of the movement deviation vt corresponding to the measurement period t and the movement deviation vt−1 corresponding to the measurement period t−1.
According to the embodiments of the present disclosure described above, information that has become a popular topic in the world can be extracted.
Hereinafter, preferred embodiments of the present disclosure will be described in detail with reference to the appended drawings.
First, the outline of a search apparatus that corresponds to an embodiment to which an information processing apparatus according to the present disclosure is applied will be described. The search apparatus sets various documents shown on the Internet or an intranet as search objects, searches for documents including a search keyword, and extracts a character string (hereinafter referred to as co-occurrence keyword or topic) included commonly in the search documents. The search apparatus provides information that has become a popular topic in the world (a trending topic) at a predetermined point of time among the documents on the Internet including the search keyword and the co-occurrence keyword as information associated with search information.
For example, the search apparatus sets tweets (showing short sentences of 140 characters or less which users of Twitter post (input)) of Twitter shown on the Internet as search objects, searches the tweets including a search keyword, and extracts a co-occurrence keyword included commonly in the searched tweets. The search apparatus calculates an evaluation value showing popularity with respect to each extracted co-occurrence keyword, displays a list of evaluation values such that the co-occurrence keyword is selected by the user, and provides the tweets including the selected co-occurrence keyword and the search keyword to the user. Thereby, the tweets regarding the information that has become a popular topic in the world can be provided to the user.
For example, if the search keyword is set as “Sensoji Temple,” “Taito Ward,” “Gokokuji,” “quake,” “in Asakusa,” and “intersection” are extracted as the co-occurrence keywords. If the user selects “quake” from the extracted co-occurrence keywords, the tweets that include the selected co-occurrence keyword “quake” and the search keyword “Sensoji Temple” are provided to the user.
The search keyword may be input by the user or may be automatically set on the basis of an operation history of the user. For example, a character string that appears frequently in a document created by the user, an artist name or a song title that is included in a play list created by the user, and a name of a star that appears frequently in a television program watched by the user may be extracted and may be set as the search keywords.
One or more contrast keywords may be set to be contrasted with the search keyword. Similar to the search keyword, the contrast keyword may be input by the user or may be automatically set on the basis of an operation history of the user.
When the contrast keyword is automatically set, the contrast keyword may be determined on the basis of the set search keyword. For example, when the search keyword is the artist name, another artist who is from the same nation may be searched from information on the Internet and an artist name of the other artist may be determined as the contrast keyword.
For example, when AAA is set as the search keyword and BBB is set as the contrast keyword, the co-occurrence keyword is extracted from the plurality of tweets including the search keyword AAA. However, the keyword having the high appearance frequency in the plurality of tweets including the contrast keyword BBB is excluded.
A plurality of character strings may be set as the search keyword and the contrast keyword and an AND search may be performed.
Hereinafter, in the present disclosure, an example of the case in which each tweet of Twitter is set as the search object will be described. However, the search objects of the search apparatus to be the embodiment are not limited to tweets.
The search object document and the search keyword are not limited to a search object document and a search keyword represented by a natural language such as Japanese and English, as long as the search object document and the search keyword can be represented by a character sting or a symbol string. For example, DNA information, phonemes, musical score information, data that is obtained by representing real number values to be quantized and included in a symbol string with a one-dimensional arrangement, and data that is obtained by representing data obtained by representing real number values to be quantized and included in a symbol string with a multi-dimensional arrangement with a one-dimensional arrangement may be set as the search object document and the search keyword.
The keyword setting unit 11 sets a character string input by the user as a search keyword. The keyword setting unit 11 sets a character string input by the user as a contrast keyword. The keyword setting unit 11 can automatically set at least one of the search keyword and the contrast keyword.
The document searching unit 12 sets each tweet of Twitter shown on the Internet as the search object and searches the tweets including the search keyword. The document searching unit 12 sets each tweet of Twitter shown on the Internet as the search object and searches the tweets including the contrast keyword. A period of a mentioned date and time of each tweet that is set as the search object may be limited from the present time to one month ago. The tweets that are obtained as the search result of the document searching unit 12 are associated with the search keyword or the contrast keyword and the association result is stored in the search document storage database 21 of the database 20.
The noise removing unit 13 removes a character string (hereinafter referred to as noise) not becoming the co-occurrence keyword from the tweets obtained as the search result. This will be specifically described below with reference to
The search index creating unit 14 creates a search index based on Suffix Array with respect to the tweets to be stored in the search document storage database 20 and obtained as the search result. The created search index is stored in the document search index database 22 of the database 20. By creating the search index, the appearance frequency DF (Document Frequency) of a topic (co-occurrence keyword) candidate character string necessary when the co-occurrence keyword is extracted in each tweet can be counted at a high speed.
When the search keyword or the contrast keyword is automatically set, the popularity determining unit 15 determines popularities of candidates of the search keyword or the contrast keyword. The popularity determining unit 15 determines the popularity of the extracted co-occurrence keyword (topic).
The topic extracting unit 16 extracts the co-occurrence keyword (topic) from each tweet of the search result from which the noise is removed. The extracted co-occurrence keyword (topic) is stored in the topic storage database 23 of the database 20.
The topic output unit 17 outputs the extracted co-occurrence keyword (topic). The topic output unit 17 may have a bot creating function for creating a tweet automatically on the basis of the extracted co-occurrence keyword (topic) and posting the tweet on Twitter.
The topic document output unit 18 acquires the tweets including the extracted co-occurrence keyword (topic) from the search document storage database 21 and outputs the tweets.
Next, an operation of the search apparatus 10 will be described.
In step S1, the keyword setting unit 11 sets the character string input by the user as the search keyword. A character string that appears frequently in a document created by the user, an artist name or a song title that is included in a play list created by the user, and a name of a star that appears frequently in a television program watched by the user may be extracted and set as the search keywords. In this case, an evaluation value of the popularity to be described below may be calculated with respect to the extracted artist name and the artist name of which the evaluation value is a predetermined threshold value or more may be adopted as the search keyword.
In step S1, the keyword setting unit 11 sets the character string input by the user or the automatically determined character string as the contrast keyword. Setting of the contrast keyword may be omitted.
In step S2, the document searching unit 12 sets each tweet of Twitter shown on the Internet as the search object and searches the tweets including the search keyword. The tweets of the search result are associated with the search keyword and the association result is stored in the search document storage database 21. When the contrast keyword is set, the document searching unit 12 sets each tweet of Twitter shown on the Internet as the search object and searches the tweets including the contrast keyword. The tweets of the search result are associated with the contrast keyword and the association result is stored in the search document storage database 21.
In step S3, the noise removing unit 13 removes a noise not becoming the co-occurrence keyword from the tweets obtained as the search result.
Returning to
In step S5, the topic extracting unit 16 executes topic extraction processing for extracting the co-occurrence keyword (topic) from each tweet of the search result from which the noise is removed. The extracted co-occurrence keyword (topic) is stored in the topic storage database 23 of the database 20.
In step S11, the topic extracting unit 16 extracts a character string group other than partial character strings appearing as only a part of other partial character strings among all partial character strings appearing in a tweet group of the search result from which the noise is removed. This corresponds to extracting a longest partial character string group in a range in which the appearance frequency DF does not change. This processing can be executed at a high speed using the search index based on the Suffix Array.
The character strings that conform to the rule by the kind of the characters to be described below are excluded from the topic candidate character strings and the remaining character strings are extracted as the topic candidate character strings.
As the kinds of the characters, a space (blank), a half-size English character, a Roman character expansion, hiragana, katakana, a full-size symbol, a macron, a half-size symbol, a control character, an invalid character, kanji, a half-size number, a punctuation mark, a Hangul character, a Thai character, an Arabic character, a Hebrew character, a Cyrillic character, and a Greek character are assumed.
[Rule to Exclude Token from Topic Candidate Character String]
When a character before a token (last character of a previous token) is a macron, the token is not designated as a topic candidate character string.
When a first character of the token is a space, the token is not designated as the topic candidate character string.
When the first character of the token is a full-size symbol, the token is not designated as the topic candidate character string.
When the first character of the token is a macron, the token is not designated as the topic candidate character string.
When the first character of the token is a half-size symbol, the token is not designated as the topic candidate character string.
When the first character of the token is a control character and an invalid character, the token is not designated as the topic candidate character string.
When the first character of the token is a punctuation mark, the token is not designated as the topic candidate character string.
When a character after a token (first character of a later token) is a macron, the token is not designated as a topic candidate character string.
When a last character of the token is a space, the token is not designated as the topic candidate character string.
When the last character of the token is a full-size symbol, the token is not designated as the topic candidate character string.
When the last character of the token is a half-size symbol, the token is not designated as the topic candidate character string.
When the last character of the token is a control character and an invalid character, the token is not designated as the topic candidate character string.
When the last character of the token is a punctuation mark, the token is not designated as the topic candidate character string.
When both the character before the token (final character of the previous token) and the first character of the token or the character after the token (first character of the later token) and the final character of the token are a half-size English character and a Roman character expansion, the token is not designated as the topic candidate character string.
When both the character before the token (final character of the previous token) and the first character of the token or the character after the token (first character of the later token) and the final character of the token are katakana, the token is not designated as the topic candidate character string.
When both the character before the token (final character of the previous token) and the first character of the token or the character after the token (first character of the later token) and the final character of the token are half-size numbers, the token is not designated as the topic candidate character string.
When both the character before the token (final character of the previous token) and the first character of the token or the character after the token (first character of the later token) and the final character of the token are Hangul characters, the token is not designated as the topic candidate character string.
When both the character before the token (final character of the previous token) and the first character of the token or the character after the token (first character of the later token) and the final character of the token are Cyrillic characters, the token is not designated as the topic candidate character string.
For example, as illustrated in HG 6, when the noise removed tweet is “People who stock up on chocolate raise your hands,” first, the character string group other than the partial character strings appearing as only the part of other partial character strings among all of the partial character strings in the tweet group of the search result is extracted. For example, when the appearance frequencies DF of “cho,” “chocolate,” and “chocolate” are 10, 10, and 4, respectively, “chocolate” is extracted. However, “cho” is not extracted. Then, the topic candidate character strings are extracted by applying a rule to exclude the token from the topic candidate character strings.
As such, the topic extracting unit 16 can extract the topic candidate character strings on the basis of a change point of the appearance frequency DF and the difference of the kinds of the characters, without depending on languages of the search object documents. However, the topic extracting unit 16 may extract the topic candidate character strings using morphological analysis based on characteristics of the languages of the documents.
When similar character strings are extracted as the topic candidate character strings, the similar character strings may be collected as one character string. In this case, similar means that a similarity degree of the character string is high and that a similarity degree of an appearing document is high.
In step S12, the topic extracting unit 16 calculates the appearance frequency DF of each topic character string in the tweets of the search result from which the noise is removed, using the search index stored in the document search index database 22.
In step S13, the topic extracting unit 16 adopts a topic candidate character string in which the appearance frequency DF satisfies a predetermined condition as the topic (co-occurrence keyword). That is, when both the search keyword and the contrast keyword are set, the topic extracting unit 16 adopts a topic candidate character string where a value obtained by dividing the appearance frequency DF in the tweets of the search result using the search keyword by the appearance frequency DF in the tweets of the search result using the contrast keyword is the predetermined threshold value or more as the topic. When only the search keyword is set, the topic extracting unit 16 adopts a topic candidate character string where the appearance frequency DF in the tweets of the search result using the search keyword is the predetermined threshold value or more as the topic.
When it is determined whether the topic candidate character string is adopted as the topic, instead of using the appearance frequency DF described above, Information Gain, Mutual Information, Bi-Normal separation, Fold Change, and a correlation coefficient may be calculated and used. A test such as a chi-squared test to measure specificity of the topic may be performed.
After the topic is extracted as described above, the topic extraction processing ends and the process returns to step S6 of
In step S6, the popularity determining unit 15 calculates an evaluation value of the popularity with respect to each co-occurrence keyword (topic) extracted in step S5. A calculation method will be described below with reference to
In step S7, the topic output unit 17 provides the extracted co-occurrence keyword (topic) and the evaluation value of the popularity thereof to the user. In step S8, when the search apparatus automatically sets the topic, the topic output unit 17 may not provide the extracted co-occurrence keyword (topic) and the evaluation value of the popularity thereof to the user.
If the provided co-occurrence keyword (topic) is selected by the user or the co-occurrence keyword where the evaluation value of the popularity is the threshold value or more is selected automatically by the search apparatus, in step S8, the topic document output unit 18 acquires the tweets including the extracted co-occurrence keyword (topic) and the search keyword from the search document storage database 21 and provides the tweets as the information associated with the search keyword to the user. When the plurality of acquired tweets are similar to each other, the plurality of tweets may be collected as one tweet and the tweet may be provided to the user. In this way, the series of operations that is executed as the associated information search processing ends.
The user can input the search keyword to the search keyword input column 51. If the user operates the Get Tweets button 52, the tweets including the search keyword are searched from the tweets of Twitter shown on the Internet.
If the user operates the Get Topic Words from Tweets button 53, the co-occurrence keyword (topic) is extracted from the tweets of the search result and the co-occurrence keyword and the evaluation value of the popularity are displayed on the topic display column 55. If the user selects the co-occurrence keyword (topic) displayed on the topic display column 55, a temporal transition of the evaluation value of the popularity with respect to the selected co-occurrence keyword (topic) is displayed on the evaluation value display column 56.
If the user operates the Show Tweets button 54 in a state in which the co-occurrence keyword (topic) is selected, the tweets including the search keyword and the selected co-occurrence keyword (topic) are displayed on the tweet display column 57.
For example, as illustrated in
If the user selects “Taito Ward” from the co-occurrence keywords (topics) displayed on the topic display column 55, a temporal transition of the evaluation value of the popularity with respect to the selected co-occurrence keyword (topic) is displayed on the evaluation value display column 56.
If the user operates the Show Tweets button 54 in a state in which “Taito Ward” is selected as the co-occurrence keyword (topic), the tweets including the search keyword “Sensoji Temple” and the selected co-occurrence keyword (topic) “Taito Ward” are displayed on the tweet display column 57. In
For example, as illustrated in
If the user selects “of shipment limitation” from the co-occurrence keywords (topics) displayed on the topic display column 55, a temporal transition of the evaluation value of the popularity with respect to the selected co-occurrence keyword (topic) is displayed on the evaluation value display column 56.
If the user operates the Show Tweets button 54 in a state in which “of shipment limitation” is selected as the co-occurrence keyword (topic), the tweets including the search keyword “vegetables” and the selected co-occurrence keyword (topic) “of shipment limitation” are displayed on the tweet display column 57. In FIG 8, however, sentences of the tweets are replaced with * (asterisks) in the tweet display column 57.
As described above, the search apparatus 10 can collect the tweets including the topic in which a user is interested for each topic and can provide the tweets to the user. If the search keyword is automatically set, the search apparatus 10 can collect the tweets including the estimated topic in which a user is interested for each topic and can provide the tweets to the user.
Next, a method of calculating an evaluation value of the popularity of the co-occurrence keyword in step S6 of the associated information search processing will be described.
First, the appearance frequency DF of the co-occurrence keyword in the tweets of the search result is converted into time-series data of a discrete system on the basis of a posting date and time of the tweet in which the co-occurrence keyword appears. Specifically, the appearance frequency DF of the co-occurrence keyword is converted into the frequency in a predetermined measurement period (for example, 24 hours).
When the measurement periods of the frequencies are set not to overlap at the time axis T, a sum of the frequencies in each measurement interval becomes the appearance frequency DF. When the measurement periods of the frequencies are set to overlap at the time axis T, samples of the plurality of frequencies can be acquired in a short period.
When the frequency in a certain measurement period t is set as xt, an evaluation value st of the popularity in the measurement period t is calculated using the frequencies xt, xt−1, xt−2, . . . , and xt−N−1 in N previous measurement periods t, t−1, t−2, . . . , and t−N+1 from the measurement period t.
Specifically, a movement mean mt, a movement deviation vt, and an evaluation value st are sequentially calculated.
Movement Mean mt=(Σxi)/N (1)
Movement Deviation vt=√((Σ(mt−xi))/N) (2)
Evaluation Value st=vt/vt−1 (3)
Σ means a sum of N values corresponding to i=t to i=t−N+1.
For example, when the frequency xt functioning as the time-series data of the discrete system transits as illustrated in
As can be seen from
The evaluation value st shows a short-term popularity trend when the measurement period t is short and shows a long-term popularity trend when the measurement period t is long. Therefore, an evaluation value st (one day) when the measurement period t is short (for example, one day=24 hours) and an evaluation value St (30 days) when the measurement period t is long (for example, one month=30 days) may be calculated and a weighted mean of the evaluation values may be calculated as a final evaluation value. The calculated final evaluation value may be used as an index to show the short-term popularity tendency and the long-term popularity tendency on whether the keyword has become a popular topic in the world (is trending).
The evaluation value st may be variously used in addition to the determination of the popularity of the co-occurrence keyword.
For example, if a sales volume of each of various products in a predetermined period is set as the frequency xt and the evaluation value st is calculated, the evaluation value st may be used as an index to determine a hit product.
If the number of times of searches by the search keyword is set as the frequency xt and the evaluation value st is calculated, the evaluation value st may be used as an index to determine a keyword that has become a popular topic in the world.
The series of processes described above can be realized by hardware or software. When the series of processes is executed by software, a program forming the software is installed in a computer embedded in dedicated hardware and a general-purpose computer in which various programs can be installed and various functions can be executed, from a program recording medium.
In a computer 100, a central processing unit (CPU) 101, a read only memory (ROM) 102, and a random access memory (RAM) 103 are connected mutually by a bus 104.
An input/output interface 105 is connected to the bus 104. An input unit 106 that includes a keyboard, a mouse, and a microphone, an output unit 107 that includes a display and a speaker, a storage unit 108 that is configured using a hard disk or a non-volatile memory, a communication unit 109 that is configured using a network interface, and a drive 110 that drives removable media 111 such as a magnetic disk, an optical disc, a magneto optical disc, or a semiconductor memory are connected to the input/output interface 105.
In the computer 100 that is configured as described above, the CPU 101 loads the programs stored in the storage unit 108 to the RAM 103 through the input/output interface 105 and the bus 104 and executes the programs, and the series of processes is executed.
The programs that are executed by the computer may be processed in time series according to the order described in the present disclosure and may be processed in parallel or at necessary timing when calling is performed.
One computer may process the programs and a plurality of computers may perform distributed processing on the programs. The programs may be transmitted to a remote computer and may be executed.
The embodiment of the present disclosure is not limited to the above example and various changes can be made without departing from the spirit and scope of the present disclosure.
The present disclosure contains subject matter related to that disclosed in Japanese Priority Patent Application JP 2011-111644 filed in the Japan Patent Office on May 18, 2011, the entire content of which is hereby incorporated by reference.
| Number | Date | Country | Kind |
|---|---|---|---|
| 2011-111644 | May 2011 | JP | national |
