Buy intent estimation and its applications for social media data

Information

  • Patent Grant
  • 11004096
  • Patent Number
    11,004,096
  • Date Filed
    Wednesday, November 25, 2015
    9 years ago
  • Date Issued
    Tuesday, May 11, 2021
    3 years ago
  • Inventors
  • Original Assignees
  • Examiners
    • Yesildag; Mehmet
    Agents
    • Hayes; Jennifer
    • Nixon Peabody LLP
Abstract
A system and method that enables usage of data embedded inside social media data to help estimate buy intent (BI), to allow companies or organizations to utilize this information to track conversations about their brand, to engage with their customers/users, to conduct advisement and investment efficiency analysis, to manage and reduce potential risk, and identify the factors that may affect company sales and revenues.
Description
FIELD OF THE INVENTION

The subject matter of the present invention relates to systems and methods for making statistical inferences based upon large quantities of largely unstructured data.


BACKGROUND OF THE INVENTION

In social media, each day, billions of messages including texts, pictures, videos, blogs, surveys, discussions and feed-backs etc. are published on a variety of networks. A tremendous quantity of information is embedded inside social media data.


It can be understood that companies and other organizations might benefit from utilizing social media data to discover information in conversations about their brand, to engage with their customers, to conduct advisement and investment efficiency analysis, to manage and reduce potential risk and identify the factors that affect company sale and revenues. Thus, it can be understood that it would be beneficial to have systems for estimating and inferring end user buy intent from massive social media data.


SUMMARY OF THE INVENTION

The present invention relates to systems and methods for making inferences based upon social media data. More particularly, the present invention relates to systems and methods that enable usage of data embedded inside social media data to help estimate buy intent (BI), to allow companies and/or organizations to utilize this information to track conversations about their brand, to engage with their customers/users, to conduct advisement and investment efficiency analysis, to manage and reduce potential risk, and identify the factors that may affect company sales and revenues.





BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings illustrate one or more exemplary embodiments and, together with the detailed description, serve to explain the principles and exemplary implementations of the present inventions. One of skill in the art will understand that the drawings are provided for purposes of example only.


In the Drawings:



FIG. 1 is a flow chart diagram of a social media data collection method, in accordance with some embodiments of the present invention;



FIG. 2 is a flow chart diagram of a Buy Intent Detection method; and



FIG. 3 is part of a flow chart diagram of a Message Processing method.





DETAILED DESCRIPTION

Various exemplary embodiments of the present inventions are described herein in the context of systems and methods for making statistical inferences based upon large quantities of largely unstructured data.


Those of ordinary skill in the art will understand that the following detailed description is illustrative only and is not intended to be limiting. Other embodiments will readily suggest themselves to such skilled persons having the benefit of this disclosure, in light of what is known in the relevant arts.


In the interest of clarity, not all of the routine features of the exemplary implementations are shown and described. It will be appreciated that in the development of any such actual implementation, numerous implementation-specific decisions must be made in order to achieve the specific goals of the developer.


Throughout the present disclosure, relevant terms are to be understood consistently with their typical meanings established in the relevant art.


Generally speaking, a system and methods are herein disclosed that enable usage of data embedded inside social media data, to allow companies or organizations to utilize this information to track conversations about their brand, to engage with their customers/users, to conduct advisement and investment efficiency analysis, to manage and reduce potential risk, and identify the factors that affect company sales and revenues. The following describes ways of using and analyzing collected social media data to determine the “buy intent” of an individual.


As will be described in further detail below, Buy intent (BI) may be defined as a real number ranging from 0.0 to 1.0 representing how likely one social media user will purchase or already bought any item from given set of products from a brand. BI is estimated based on a collection of relevance social media messages from a single user over a given period time.


As used herein, the acronym POS refers to part-of-speech tagging also called grammatical tagging or word-category disambiguation. It is the process of marking up a word in a text (corpus) as corresponding to a particular part of speech, based on both its definition, as well as its context—i.e. relationship with adjacent and related words in a phrase, sentence, or paragraph.


As shown in FIG. 1, a “Social Media Data Collection System” 100 is a server-side system and process which is deployed across many servers. The major functionalities for this system include fetching/collecting data from social media networks, for example, Face Book, Twitter, Renren, Sina Weibo, Wechat, Linkedin and many other blogs and many other web sites. In practice, the number of servers used for data collection is based on the system configuration. If there are more companies, more customer accounts, or more keywords to search on web sites, the system may dynamically deploy more servers. After data is collected, it is persisted and maintained in the database first, and then pushed to a container of a distributed network cache for further processing. The system may include many data process systems, many or all of which may consume data from a specific container in a distributed network cache, and push the analyzed results to another container.


According to some embodiments, methods are provided for substantially automated extracting of buy intent (BI) indications of social media users, using enabling computing devices to process social media data to estimate buy intent; to enable manual grading of real social data for BI estimation training; to enable combining of exact match analysis, vector space modeling, POS tagging and message replacement, and using Bayesian classification algorithms for BI estimation; utilizing BI scores to identify potential buyers; utilizing BI scores to Identify positive and negative influences; utilizing BI scores to estimate advertising efficiency; utilizing BI scores to estimate parameters for targeted fixed effects like gender, age, education, income, region, search history and purchase patterns; utilizing BI score to identify potential buyers common properties; and utilizing BI scores to trace stimulation factors for potential buyer status changes.


As can be seen in FIG. 1, in step 105 a clustering of servers is initiated to handle the organization of available data into manageable and linked groups by Social Media Data Collection System 100.


In step 110 a pre-defined configuration is loaded into the cluster that has been started to enable the clustering and capability establishment of the server cluster to perform the necessary functions for data collection and organization. The configuration also enables the cluster to assess its own capacity to handle the data volume and dynamically set the cluster size


In step 115 servers can be started in the cluster, to enable the configured cluster data to be processed and/or served to system and/or system users.


In step 120 a decision is to be taken whether there are enough servers to process and/or serve the users. If not, at step 125 additional server(s) may be added.


If there are enough servers at step 120, data is fetched in step 130, from multiple social media sources, in multiple data formats.


In step 135 raw data is acquired from the collected data, to be further processed.


In step 140 an index is created from the raw data, for enabling rapid categorization, sorting, fileting and searching of social media data.


In step 145 the raw data is processed by an algorithm to detect the specific user(s), and to correlate detected user(s) to the user profiles in the Social Media Data Collection System 100.


In step 150 the indexed and/or user correlation data are further processed to determine whether the collected data is to be persisted/maintained in the system 100, or is to be distributed to system users.


In step 160 the processed persistent and/or distributed data is fed to a further BI processing engine or element, for BI specific analyzing, to detect user Buy Intent.


In the embodiment illustrated in FIG. 2, Buy Intention Detection is one of the parts of the Buy Intent analysis process. The system may include many independent processes that all pull messages from one container in a distributed network repository. The flow chart in FIG. 2 describes the major elements and their interaction in the system.


In preferred practice, the collected data is supplied to a computational-capable server having at least a processor and at least a storage capability, to teach the processor to analyze the available data. Once taught, additional data can be provided to the server to generate a rating of the buy intent of the customers. For example, there are many millions of messages collected in the system per hour or even second, and the number is continually growing daily as more customers, accounts and different search criteria and interests are entered. The system's servers may be deployed in a cluster with virtually unlimited computing power. This process is dynamic in that more servers can be added automatically if needed.


As can be seen in FIG. 2, a work flow example of a “Buy Intent Detection System” is described.


In step 205, the BI process is started by the Social Media Data Collection System 100.


In step 210 data is fetched from multiple social media sources, and in step 215, the fetched social media data is collected together in the system data storage facilities.


In step 220 the data is normalized, to convert all received data into a unified format, by system data converter element.


In step 225 the data is processed by a dictionary, optionally with multiple languages, to further normalize data from multiple languages.


In step 230 the data is processed using a POS tag Analysis Engine, to identify critical POS sale data.


In step 235 search indexes are generated, to help rapidly search and sort collected data.


In step 245 data history is used to help generate more accurate search indexes. In step 240 learned data is updated, from the search indexes and/or historical data.


In step 250 the system determines whether sufficient learned data exists to generate accurate search indexes.


If no learned data exists, then in step 260 vector space modeling analysis may be executed, to further process collected data, to complement the accuracy of the learned data. Additionally or alternatively, in step 255 message similarity analysis is executed, to further process collected data, to complement the accuracy of the learned data.


If sufficient learned data exists, then in step 265 BI related data is fetched or created into a file, for example, including data source, time, brand, product, BI score, user information, influence weight, etc.


In step 270 a decision is taken by the system as to whether the BI information acquired is accurate enough.


If not, in step 275, Bayesian analysis is executed on the processed BI data.


Alternatively or additionally, if the data is accurate enough, in step 280, if the data is determined to be persistent, the system BI statistics are updated, to include the latest BI definitions, classifications, etc.


In step 295 the processed BI data as determined by the above steps is distributed to the system's servers, for distributions to system elements or components.


In step 296, the distributed BI data is ready for usage by system users.


The flow chart in FIG. 3 illustrates elements in the system's message system, to convey a part of the functionality being described herein. The system may have many servers on a cloud that collect data from social media. The collected data is put into a virtual processor location, referred to hereinafter as a “container” of a network distributed cache. Many processes can concurrently access the same container at any time. Each process can pull one message a time and process it, and may push the modified data into another container. A user can create any number of containers at run time. The system also enables many processes to work similarly, whereby the number processes run in the system is dependent on the system configuration. For example, a user may configure from 1 to hundreds, thousands or more processes at will. Of course, more processes may require more computing power.



FIG. 3 is a 2-part flow chart diagram of a generic Message Processing Method, in accordance with some embodiments. As can be seen, in step 305 the container manager process is started by Social Media Data Collection System 100.


In step 310, the collected social media data is sent to multiple data collectors 1 to n.


In step 315 the data collectors data is consolidated, for example, collected from different sources into system data collectors.


In step 320 the data is normalized, for example, to aggregate different formats and types of data.


As can be seen, in step 325 multiple processors are used to pull BI related data from a container(s) and to further process one or more data elements, and then push the resultant processed data elements to a further data container.


In step 330 data enrichment is executed, optionally including processing the pre-processed data for BI related information such as sentiment, mood, intention, spam etc.


In step 335, container 2 may be further processed by Social Media Data Collection System 100.


In step 340 a data analysis engine processes container data for further BI related metrics, such as influence, topic, trend, popular keys, buy intention, targeting analysis etc.


In step 345 container 3 may be further processed by Social Media Data Collection System 100.


In step 350 a customer role engine dispatches the message to different queues based on system requirements, such as customer rules, conditional processing etc.


In step 355 container 4 may be processed by Social Media Data Collection System 100.


In step 360 a report engine processes the data to generate BI related reports.


In step 365 container x is processed by Social Media Data Collection System 100.


In step 370 a data API for paid customers is run, to manipulate and fetch data for advanced features or functions as may be used by paid users.


In step 375 container n is processed by Social Media Data Collection System 100.


In step 380 thread updates are monitored to determine BI related modifications in data threads.


The system described herein may have many servers on a cloud that collect data from social media. The collected data is put into a container of a network distributed cache. Many processes can concurrently access same container at any time. Each process can pull one message a time and process it, and may push the modified data into another container. A user can create any number of containers at run time. The system also enables many processes to work similarly, whereby the number processes run in the system is dependent on the system configuration. For example, a user may configure from 1 to hundreds, thousands or more processes at will. Of course, more processes may require more computing power.


It should be appreciated that embodiments of the present invention may include a combination of one or more of the following elements:

    • Combine user social media data from multiple source (FB, Twitter, Linkedin, Renren, Sina weibo, Tencent wechat etc.) to estimate buy intent.
    • Manually grading social data for BI.
    • Feeding data into the system for AI and Bayesian learning for BI.
    • Estimating BI using a combination of first direct matching, Vector Space analysis, POS tag analysis and message replacement, and then Bayesian statistics.
    • Utilizing BI scores to identify potential buyers.
    • Utilizing BI scores to Identify positive and negative influencers.
    • Utilizing BI scores to estimate advertising efficiency.
    • Utilizing BI score to estimate parameters for targeted fixed effects like gender, age, education, income, region, search history and purchase patterns.
    • Utilizing BI scores to identify potential buyers' common properties.
    • Utilizing BI scores to trace stimulate factors for potential buyer status change.


In one example, each message can be classified into five categories as follows:









TABLE 1







Message buy intent category list









Category Name
Description
Score





No buy intent
Message contains no buy intent
0.00


Weak intent
Message contains weak buy intent
0.25


Medium intent
Message contains medium buy intent
0.50


Strong intent
Message contains strong buy intent
0.75


Bought
Owner already purchased item(s) from brand
1.00









Let Vki be the actual values and Nki be actual occurrence of Vki for individual K. The buy intent for individual k can then be calculated as:







N
k

=




i
=
1

5



N
ki









BI
k

=




i
=
1

5




N
ki

×


V
ki

/

N
k








The BIk and Nk are two independent variables though the Nk has significant effects on BIk estimation accuracy. We have found that the accuracy of BIk estimation becomes relatively stable when the number of messages used for each individual exceeds 30. Therefore, the analysis only chooses the individuals who have 30 or more relevant messages to participate in the data analysis, thereby to increase the estimation accuracy. A key technology relates to the classification of each relevant message into specified categories based on learned data.


BI Estimation


The BI estimation process can include sample dataset manual grading, raw data preparation, classification engine dynamic data training, BI calculation and parameters estimation etc.


Dataset manual grading: A significant process is classification engine training. It requires grading a set of messages into categories, for example the five categories described above, and feeding them into classification engine learn module. Although there is no lower limits of number graded messages, the number should exceed ten thousands to satisfy minimum estimation accuracy. Generally speaking, the more graded messages for learning will resulting in more accurate estimation. The estimation accuracy obeys logistic distribution with the following properties:


Probability function:







P


(
x
)


=


e


-

(

x
-
m

)


/
b




b


[

1
+

r


-

(

x
-
m

)


/
b



]


2






Distribution function:







D


(
x
)


=

1

1
+

e


-

(

x
-
m

)


/
b








The accuracy improvement becomes less and less significant if number of graded messages reaches more than hundred thousand. The actual regression of actual accuracy improvement and number graded messages for training dependents on actual data and quality of grading.


Raw data preparation: Graded data must be normalized before either fed into classification engine's learn module or used for BI estimation. This process transfers message text to lowercase and eliminates irregular spacing. It first changes word case and adds spaces before and after each punctuation character and then parses whole message into words and punctuation array using space as delimiter. The misspelled words will be replaced based on Bayesian probability. Then three messages will be reconstructed. The first message is constructed by inserting single space between consecutive words and appending punctuation character without using any space. The second message is constructed similarly as first one except original stop words being eliminated from word array. The third message is constructed similarly as first one except all the words in the array are transferred to corresponding root words. For example, the word “sits” and “sat” are transferred to sit, the word “cars” and “happier” are transferred car and happy respectively. Three MD5 hashes are calculated and persistent along with classification category information.


POS tags input for classification engine: In order to detect POS of social media message, huge amount of graded n-gram data has been incorporated into classification engine (CE). These n-grams are generated based on the largest publicly-available, genre-balanced corpus of English—the 450 million word Corpus of Contemporary American English (COCA), 1.8 billion words data from GloWnE and 1.9 billion words from 4.4 million Wikipedia articles. The data is consisted of three pieces of information: 1, 2, 3, 4, 5-word sequences, frequency counts and corresponding individual POS tags for the word sequences. The information stored in CE memory efficiently. Original phrase will be used as identifier If actual phrase char length is smaller than 16. The MD5 hash will be used as identifier otherwise.









TABLE 2







Partial Penn Treebank POS tag list









Category
Examples
Tags





Adjective
happy, bad
JJ


Adjective, ordinal number
72nd, last
JJ


Adjective, comparative
happier, worse
JJR


Adjective, superlative
happiest, worst
JJS


Adjective, superlative, semantically
chief, top
JJ


Adjective, cardinal number
3, fifteen
CD


Adjective, cardinal number, one
one
CD


Adverb
often, particularly
RB


Adverb, negative
not, n't
RB


Adverb, comparative
faster
RBR


Adverb, superlative
fastest
RBS


Adverb, particle
up, off, out
RP


Adverb, question
when, how, why
WRB


Adverb, degree & question
how, however
WRB


Adverb, degree
very, so, too
RB


Adverb, degree, postposed
enough, indeed
RB


Adverb, nominal
here, there
RB


Conjunction, coordination
and, or
CC


Conjunction, subordinating
although, when
IN


Conjunction, complementizer
that
IN


Determiner
this, each, another
DT


Determiner, pronoun
any, some
DT


Determiner, pronoun, plural
these, those
DT


Determiner, prequalifier
quite
PDT


Determiner, prequantifier
all, half
PDT


Determiner, pronoun or double conj.
both
DT(CC)


Determiner, pronoun or double conj.
either, neither
DT(CC)


Determiner, article
the, a, an
DT


Determiner, postdeterminer
many, same
JJ


Determiner, possessive
their, your
PRP$


Determiner, possessive, second
mine, yours
PRP


Determiner, question
which, whatever
WDT


Determiner, possessive & question
whose
WP$


Noun
aircraft, data
NN


Noun, plural
women, books
NNS


Noun, proper, singular
London, Michael
NNP


Noun, adverbial
tomorrow, home
NN


Noun, adverbial, plural
Sundays, weekdays
NNS


Pronoun, nominal
none, everything, one
NN


Pronoun, personal, subject
you, we
PRP


Pronoun, personal, subject, 3SG
she, he, it
PRP


Pronoun, personal, object
you, them, me
PRP


Pronoun, reflexive
herself, myself
PRP


Pronoun, question, subject/object
who, whoever
WP


Pronoun, existential
there
EX









POS tags selection: The Classification engine adapts Penn Treebank POS tag system. Every word in social media message is associated with POS tag. All possible POS tags are listed in Table 2. The algorithm for POS tag selection is based on probability learned in previous step. The message is normalized and parsed into word and punctuation array. Slide window can have maximum five words sequence and any phrase stops whenever it encounters a punctuation. The original phrase or its MD5 hash (if phrase is longer than 16 bytes) is constructed to retrieve learned POS tag information. The corresponding tags in learned data will be used for current message if match found. Relative heavier weight is given to longer n-gram if there is conflict between different sizes of n-gram. The selection always starts in current word and tries to use longer n-gram. After exhausting sequences and making the POS choice for each word, work window is slide to right and whole process is repeated till all words in message are exhausted.


Princeton WordNet for classification and similarity calculation: The Classification engine introduces Princeton WordNet for classification index computing. WordNet® is a large lexical database of English and it groups nouns, verbs, adjectives and adverbs into 117 000 cognitive synonyms (synsets). Synsets are interlinked by means of conceptual-semantic and lexical relations. CE also incorporates more than 124 thousands manual graded phrases (two or more words) into WordNet and expands grouping to all kinds of POS types. The new generated “Synsets” are utilized intensively to compute similarity index between two messages and detect whether given message is similar to learned data. Similarity index between two messages are calculated using following steps:

    • Input message normalization
    • Input message parsing
    • Input message POS selection
    • Using new formed POS array to retrieve all corresponding learned messages
    • Computing word weight, start from index value 1.0 and determine steps needed to exchange words in current message with words in synsets to make current message similar to learned data. The fewer traverse steps will result in higher match index. The match index will be one if identical message found in learned data set.
    • This phenomenon may be expressed more clearly in genetic terminology. One may view each message as chromosome and each word in the message as locus on the chromosome. Number of loci on a chromosome is unlimited. The effects from different loci are additive. Each locus may have two or more alleles which may interact among them and show dominance effect. Multiple alleles correspond multiple words in single synonyms group in synsets.
    • One may put different weights on different POS types and on steps needed to change each word to match learned data.






I
=




i
=
0

n




w
i






j
=
0


m
ij




(

1
-

v
ij


)










    • Where n is total number words in message, wi is weight on i'th word based on POS, mij is the steps needed to transform original word to one learned word in synsets so that new formed message match one of the learned messages. mij can be 0.0 that means current word does not needs transformation. The product part in above formula will become 1.0 if no step needed for transformation. Vij is the contribution cost for transformation for i'th word at step j.





Vector space model and messages similarity calculation: The vector space model is widely used for related documents retrieval and messages similarity calculation mainly because of its conceptual simplicity and the appeal of the underlying metaphor of using spatial proximity for semantic proximity. Vector space model treats message as a point in an n-dimensional spaces where n is the number of common words in the two messages or message and a category. The coordinators of given message and group are calculated based word frequency occurred in message and group of messages. The similarity coefficient is usually expressed as vectors normalized correlation coefficient as follows:







cos


(

g
,
m

)


=





i
=
1

n




g
i



m
i









i
=
1

n



g
i
2









i
=
1

n



m
i
2









Where gi is i'th word frequency for one of learned categories and mi is the i'th word frequency for current message. The advantage of vector space model is that it uses little computer memory and computing algorithm is simple and direct. The disadvantage is that it does not use other information like word order, word combinations, word meaning and AI technology etc.


Bayesian classification and Bi estimation: When conducting Bayesian classification, BI classification information is derived from graded data and current message. To estimate the probability of a current message belonging to a certain BI category, the following can be used:







P


(


C
i

|
W

)


=


P


(

W
|

C
i


)



P


(
W
)








Where Ci represents different BI subclass. Since only relative values are of interest, P(W) can be ignored.








P


(


C
i

|
W

)



P


(


C
j

|
W

)



=



P


(

W
|

C
i


)



P


(

W
|

C
j


)



=


P


(



W
1



W
2





W
n


|

C
i


)



P


(



W
1



W
2





W
n


|

C
j


)








From the Chain rule, we have







P


(


W
1



W
2





W
n


)


=


P


(

W
1

)




P


(


W
2

|

W
1


)




P
(


W
3

|


W
1




W
2













P


(


W
n





i
=
1


n
-
1




W
i


)











We use simplest phrases weight and calculate each phrase probability as follows








2

m
-
1







i
=
1


n
-
m




P


(


W
i



W

i
+
1





W

i
+
2













i
+
m



)




=




i
=
1


n
-
m





C


w
i



w

i
+
1










w
m



/

S
m







Bi classification is determined based on calculated probability of current message association with each BI category.


BI Estimation


After Graded BI category data has been fed into a CE learning module, message are normalized and parsed into word and punctuation array. The POS tags are selected based on the probability model described above. Exact message information, different length word phrases are constructed and occurrence frequencies are accumulated and recorded. Vector space model and Bayesian basic statistics also established. At run time, BI value for each message is calculated as follows:

    • Input message normalization
    • Input message parsing
    • Incorrect words replacement
    • Exact match detection. This process basically detects whether we have exact message learned. If found, the corresponding BI category will be returned.
    • Vector space modeling to detect smallest deviation from any category. This is done with given threshold accuracy value. Only higher accuracy (close 1.0) be chosen as return value.
    • POS type and associated words are used to detect whether we have similar messages learned in each category. This process demands higher computing resources. Many messages are retrieved from repository based on words within current message and transverse to detect whether current message can be made similar with any of learned messages.
    • Finally Bayesian analysis is conducted to choose the BI category based highest association probability. This is catch all process and it always yields classification result.


      BI Applications


After calculating the BI score for each social media message, each message can be used as an independent variable and fitted into general mixed model equations to estimate the best linear unbiased estimators for many major factors, like author age, gender, region, income, education etc. The correlation coefficients between sentiment score and BI score can be estimated. The BLUE of advertisement efficiency can be computed by introducing targeted criteria and other fixed independent variables into model. By calculating weighted average of BI score for individual user, we can further identify potential buyers at early stage and offer guideline for company production and other activity planning. Since every user in social media network is interlinked, BI score can also be used to identify the most positive and negative influencers to a given brand. User's BI score can also be used as selection filter to generate subset of social messages which then be fed into further topic analysis model to determine common properties among potential buyers. Also, filtered data is used identify the real relationship among potential buyers and trace circumstances that stimulate user status change. Following is the basic mixed model formula for the best linear unbiased estimation (BLUE).

Y=X′β+Z′μ+e

Where:

  • Y is observation dependent matrix with n rows and m columns containing BI as one column
  • X is relation matrix (n×p) that links fixed β to observations
  • β is fixed parameter vector with dimension p×1
  • Z is relation matrix(n×q) that link random μ to observations
  • μ is random parameter vector with dimension q×1
  • e is random error effect with dimension n×1
  • E(e)=0
  • var(e)=R, n×n systematic error variance matrix
  • var(μ)=G, q×q systematic variance-covariance matrix
  • con(μ, e′)=0
  • Mixed model equations then can be written as:








(





X




R

-
1



X





X




R

-
1



Z







Z




R

-
1



X






Z




R

-
1



Z

+

G

-
1






)



(



β




μ



)


=

(





X




R

-
1



Y







Z




R

-
1



Y




)





  • Solution to generic model above is then:








(



β




μ



)

=



(





X




R

-
1



X





X




R

-
1



Z







Z




R

-
1



X






Z




R

-
1



Z

+

G

-
1






)


-
1







(





X




R

-
1



Y







Z




R

-
1



Y




)







At this juncture, it should be clear that, in other embodiments, methods are provided for enabling computing devices to use social media data to estimate to enable manual grading of real social data for BI estimation training; to enable combining of exact match analysis, vector space modeling, POS tagging and message replacement, and Bayesian classification algorithms for BI estimation; utilizing BI scores to identify potential buyers; utilizing BI scores to Identify positive and negative influences; utilizing BI scores to estimate advertising efficiency; utilizing BI scores to estimate parameters for targeted fixed effects like gender, age, education, income, region, search history and purchase patterns; utilizing BI score to identify potential buyers common properties; and utilizing BI scores to trace stimulation factors for potential buyer status changes.


Further at this juncture, it should be clear that the capability of the system to provide automated identification, analysis and use of available social media data can enable companies to enhance revenue generation and business decision making.


As mentioned above, the system may be enabled by data processing algorithms that integrate Vector space models and Bayesian basic statistics. Although the underlying mathematics theory being used herein is partially similar in Vector space and Bayesian statistics analysis, the way in which the system utilize and apply these theories are unique, particularly in terms of the parameters derivation, actual data manipulation and iteration criteria. The system described herein incorporates AI and generic algorithms into existing vector space and Bayesian analysis, thereby changing both the process and the results.


According to some embodiments, the buy intention analysis is substantially executed using a computer code (for example, a computer program which is written using C#) running on a server (for example, window server 2008) a connected to a data cloud (for example, Amazon servers cloud). There can be multiple physical servers running on the cloud on a cluster system behind a load balancer. The load balancer is configured to receive huge numbers (example, Millions per second) of social media data items that are downloaded every minute, and to distribute the data to one of the servers in the cluster system. The BI system on the server analyzes the data and derives a buy intent value between 0.0 and 1.0. The higher the number means the stronger the buy intent, wherein the maximum value is 1.0.


In summary, the following aspects are noteworthy:

    • The system collects social media information from multiple social media channels like Face book, twitter, Linkedin, Renren, Tecent, Sina Weibo etc.
    • The system manually grades partial data initially for BI, and feeds the data into AI and Bayesian learning systems.
    • The system uses artificial intelligence to link multiple messages published on multiple social media sites to the same user.
    • The system correlates, executes regression, indexes and persists or maintains all this data for fast retrieval.
    • The BI system is trained using AI and statistical methods uniquely for BI.
    • The system uses several innovative mechanisms to estimate BI (see details later). Both the concept and methodology for doing so are designed and configured specifically for social media data.
    • The system runs algorithms to use BI dynamically in advertisements.


Further, embodiments of the BI system described herein may be used to calculate a numeric value (between 0.0 to 1.0), to represent whether the user is likely to buy one or more products from a brand.


Organizations may use this data to implement, for example:

    • Business activity planning
    • Material purchasing
    • Target advertisement and increasing sales
    • Product improvement
    • Customer relation management
    • Companies can use this data as an independent variable to identify whether age, gender, incoming, region, ethic group and other social groups has a preference for their product
    • Risk management


Exemplary embodiments have been described with reference to specific configurations. The foregoing description of specific embodiments and examples has been presented for the purpose of illustration and description only, and although the invention has been illustrated by certain of the preceding examples, it is not to be construed as being limited thereby.

Claims
  • 1. A method implemented on a processing device, the method using information embedded in social media data messages to identify potential buyers and determine the potential buyers' buy intent (BI) in terms of BI scores and thereby provide companies with market information to influence buying decisions of individuals, improve advertisement efficiency, reduce potential risk and grow sales and revenue, the method comprising: establishing a social media data messages collection system, wherein the social media data messages collection system comprises a plurality of servers and storage containers located in a cloud, wherein the plurality of servers is configured with processing capability, wherein a number of servers within the plurality of servers is dynamically adjustable based on social media data messages collected and processed in the social media data messages collection system;wherein the social media data messages are based on a number of companies, brands, customers that are catered to, and a number of key-words used;fetching and collecting, by the social media data messages collection system, social media data messages from social media network sites and websites;saving the collected social media data messages in the storage containers located in the cloud;normalizing the social media data messages collected in the storage containers by a data convertor element located within the plurality of servers, wherein normalizing the collected social media data messages comprises: processing the social media data messages by a multi-language dictionary located within the plurality of servers, wherein the multi-language dictionary comprises multiple languages to convert social media data messages from foreign languages;converting the social media data messages processed by the multi-language dictionary using a data converter element within the plurality of servers, wherein the data converter element is configured to convert the social media data messages collected in the storage containers into a unified standard word format; andprocessing and tagging the social media data messages converted by the data converter element using a part of speech (POS) tag analysis engine, wherein the POS tag analysis engine is configured to generate accurate search indexes for improved search and collection of relevant social media data messages;training the social media data collection and processing system using the collected social media data messages for artificial intelligence (AI) based BI estimation, wherein the training comprises supplying the collected social media data messages to a first server within the plurality of servers, wherein the first server comprises at least one processor configured to analyze the social media data messages to generate a BI estimation; andproviding the normalized and POS tagged social media data messages to the at least one processor of the first server to generate BI estimation in the form of BI scores, wherein generating BI scores comprises: processing the normalized and POS tagged social media data messages using vector space modelling, exact match analysis, message replacement and Bayesian classification algorithms to determine a first group of social media data messages to generate the BI score estimation for a potential buyer, wherein the first group of social media data messages comprises a number of social media data messages relating to a brand, a product or a service associated with the potential buyer that exceeds a minimum threshold value, wherein the number of social media data messages that exceeds a minimum threshold value are associated with the potential buyer, and wherein the BI score estimation of the potential buyer is converted into a BI score using the at least one processor.
  • 2. The method of claim 1, further comprising utilizing the BI score to identify potential buyers of goods and services.
  • 3. The method of claim 1, further comprising utilizing the BI score to identify positive and negative influencers of potential buyers.
  • 4. The method of claim 1, further comprising utilizing the BI score to estimate advertising efficiency.
  • 5. The method of claim 1, further comprising utilizing the BI score to estimate parameters for targeted advertising using fixed effects like gender, age, education, income, region, search history and purchase patterns.
  • 6. The method of claim 1, further comprising utilizing the BI score to identify parameters that impact a potential buyer's purchase decision.
  • 7. The method of claim 1, further comprising utilizing the BI score to trace factors for potential buyer status changes.
  • 8. The method of claim 1, wherein a BI score for an individual k is calculated from Nk social media data messages as:
  • 9. The method of claim 8, wherein NK has the preset value that exceeds 30, and i the number of the group of learned categories equals 5.
  • 10. A system for determining buy intent (BI) in terms of BI Scores by using social media data messages, the system comprising: a plurality of distributed data collection modules configured to collect the social media data messages in a plurality of languages from web based social media sites;a multi-language dictionary server configured to translate the social media data messages in the plurality of languages;a part of speech (POS) tag analysis engine configured to POS tag the social media data messages and establish a minimum threshold value of social media data messages using vector space modelling, exact match analysis, message replacement and Bayesian classification algorithms to determine a first group of social media data messages to generate a BI score estimation for a potential buyer;a classification engine configured to classify the first group of social media data messages into BI categories and classify each BI category into a word index, thereby grouping the first group of social media data messages into a learned set of categories; andat least one cloud-based distributed processor dynamically configured to construct a vector space model based upon the learned set of categories, to generate a BI score there from, wherein constructing the vector space model comprises:converting common words in a graded set of social media data messages into the learned set of categories, anddetermining a similarity coefficient based on word frequency.
  • 11. The system of claim 10, wherein the similarity coefficient is expressed as:
  • 12. The system of claim 10, wherein the BI score for an individual k is calculated by the at least one cloud-based distributed processor as:
  • 13. The system of claim 12, wherein the BI score for the individual k is calculated as a numeric value to represent whether a user is likely to buy one or more products from a brand.
  • 14. The system of claim 9, wherein the BI score for the individual k is calculated as a numeric value, between 0.0 to 1.0, expressed as a set of five learned categories representing the BI score of the individual.
  • 15. A method implemented on a processing device, the method using social media data from multiple online sources to estimate buy intent (BI) in terms of BI scores, using cloud-based processing, the method comprising: collecting social media data located in the multiple online sources using a plurality of data collectors, wherein the multiple online sources are located in a cloud-based storage;consolidating the collected social media data in containers in the cloud-based storage;normalizing the consolidated social media data to identify key phrases, using POS tag analysis and language dictionaries to process variations in usage and different languages used using the cloud-based processing;identifying and generating accurate search indexes using POS tagging combined with available historic data and the identified key phrases, wherein the generated search indexes comprises graded real social data for categorizing the social media data into learned set of categories for BI assessment and AI learning; andusing the generated search indexes with exact match analysis, vector space modelling, message replacement and Bayesian classification algorithms to generate a BI score for each individual from the learned set of categories;wherein the BI score is based on: sentiment, data source information, time, brand, product user info, and influencer weightage;wherein the BI score for an individual k is calculated as:
  • 16. The method of claim 10, wherein the cloud-based storage is located in distributed containers in the cloud.
  • 17. The method of claim 10, further comprising utilizing the BI scores to identify positive and negative influencers, estimate advertising efficiency, estimate parameters for targeted advertising using fixed effects like gender, age, education, income, region, search history and purchase patterns, identify potential buyer's common properties and find factors impacting for potential buyer status change.
  • 18. The method of claim 10, further comprising utilizing the BI scores to estimate advertising efficiency.
  • 19. The method of claim 10, further comprising utilizing the BI score to estimate parameters for targeted advertising using fixed effects like gender, age, education, income, region, search history and purchase patterns.
  • 20. The method of claim 10, further comprising utilizing the BI score to identify potential buyer's common properties.
  • 21. The method of claim 10, further comprising utilizing the BI score to trace stimulate-factors for potential buyer status changes.
  • 22. The method of claim 10, wherein the BI score for an individual k is estimated as a numeric value (between 0.0 to 1.0) representing a user's likelihood of buying one or more products or services from a brand.
US Referenced Citations (77)
Number Name Date Kind
6094829 Li Apr 2000 A
6601234 Bowman-Amuah Jul 2003 B1
7308418 Malek et al. Dec 2007 B2
7761848 Chaffin Jul 2010 B1
7930302 Bandaru et al. Apr 2011 B2
8225271 Eldridge et al. Jul 2012 B2
8266148 Guha et al. Sep 2012 B2
8584082 Baird et al. Nov 2013 B2
8635281 Kelly Jan 2014 B2
8645912 Welchman et al. Feb 2014 B2
8683433 Nayak et al. Mar 2014 B2
8688726 Mahajan et al. Apr 2014 B2
8694520 Krishnakumar et al. Apr 2014 B1
8707259 Trofin et al. Apr 2014 B2
8732669 Valdiviezo et al. May 2014 B2
8819659 Ramer et al. Aug 2014 B2
8832644 Hirsch Sep 2014 B2
8924914 Boeckenhauer et al. Dec 2014 B2
8978006 Hirsch Mar 2015 B2
8996371 Hurvitz et al. Mar 2015 B2
9002892 Benyamin et al. Apr 2015 B2
9116983 Ameri-Yahia et al. Aug 2015 B2
9223831 Baker et al. Dec 2015 B2
9448776 Sankaran Sep 2016 B1
9672032 Zhou Jun 2017 B2
9696967 Balasubramanian Jul 2017 B2
10073794 Thomas et al. Sep 2018 B2
10095686 Zhang Oct 2018 B2
20010027472 Feng Oct 2001 A1
20060053156 Kaushansky et al. Mar 2006 A1
20070050446 Moore Mar 2007 A1
20070174255 Sravanapudi Jul 2007 A1
20080249764 Huang et al. Oct 2008 A1
20090319342 Shilman et al. Dec 2009 A1
20100114654 Lukose et al. May 2010 A1
20100131563 Yin May 2010 A1
20100169304 Hendrickson et al. Jul 2010 A1
20100235313 Rea et al. Sep 2010 A1
20110113447 Jung May 2011 A1
20110191372 Kaushansky et al. Aug 2011 A1
20110213655 Henkin Sep 2011 A1
20120036007 Robertson et al. Feb 2012 A1
20120116868 Chin et al. May 2012 A1
20120158489 Benyamin et al. Jun 2012 A1
20120203584 Mishor Aug 2012 A1
20120215625 Ramer et al. Aug 2012 A1
20120246033 Keiser Sep 2012 A1
20120246054 Sastri et al. Sep 2012 A1
20120290340 Ramanujam et al. Nov 2012 A1
20120290938 Subbarao et al. Nov 2012 A1
20120296845 Andrews et al. Nov 2012 A1
20130080264 Umeda Mar 2013 A1
20130103385 Ghosh et al. Apr 2013 A1
20130124653 Vick May 2013 A1
20130132851 Konopniki et al. May 2013 A1
20130166379 Ehindero et al. Jun 2013 A1
20130238356 Torrii Sep 2013 A1
20130246430 Szucs et al. Sep 2013 A1
20130263019 Castellanos et al. Oct 2013 A1
20130311323 Stoll et al. Nov 2013 A1
20130339126 Cui et al. Dec 2013 A1
20140025749 Glowacki Jan 2014 A1
20140025758 Glowacki Jan 2014 A1
20140056526 Scipioni Feb 2014 A1
20140136323 Zhang et al. May 2014 A1
20140188897 Baker et al. Jul 2014 A1
20140254454 Calcev Sep 2014 A1
20140337320 Hernandez et al. Nov 2014 A1
20150106304 Gupta Apr 2015 A1
20150112753 Suvarna Apr 2015 A1
20150186790 Ehlen Jul 2015 A1
20160087933 Johnson Mar 2016 A1
20170109141 Thomas et al. Apr 2017 A1
20170243125 Thomas et al. Aug 2017 A1
20170300564 Feng et al. Oct 2017 A1
20170315996 Fothergill Nov 2017 A1
20180165706 Feng et al. Jun 2018 A1
Foreign Referenced Citations (7)
Number Date Country
2698955 Feb 2014 EP
2010008685 Jan 2010 WO
2013101812 Jul 2013 WO
2013158790 Oct 2013 WO
2013158840 Oct 2013 WO
2013177230 Nov 2013 WO
2014143018 Sep 2014 WO
Non-Patent Literature Citations (10)
Entry
S. Buckley et al., “Social media and customer behavior analytics for personalized customer engagements,” in IBM Journal of Research and Development, vol. 58, No. 5/6, pp. 7:1-7:12, Sep.-Nov. 2014, doi: 10.1147/JRD.2014.2344515. (Year: 2014).
Christl et al., Using Mobile Technology for Inter-Organisational Collaboration and End-Customer Integration, ACM, 2013, pp. 1-8.
Rosa et al., Smart Composition of Reusable Software Components in Mobile Application Product Lines, ACM, 2011, pp. 45-49.
Sung et al., Mobile Application Development Classes for the Mobile Era, ACM, 2014, pp. 141-146.
Bharat et al., Building Distributed, Multi-User Applications by Direct Manipulation, ACM, 1994, pp. 71-81.
Koziokas et al., Usability Testing of Mobile Applications: Web vs. Hybrid Apps, ACM, 2017, pp. 1-2.
Meskens et al., Plug-and-Design: Embracing Mobile Devices as Part of the Design Environment, ACM, 2009, pp. 149-153.
Yang et al., Personalized Mobile Application Discovery, ACM, 2014, pp. 49-54.
Zhan et al., MCKit: A Mobile App for Conferences, ACM, 2014, pp. 59-64.
Kirk, M., Thoughtful Machine Learning: A Test-Driven Approach, O'Reilly Media, Inc., Ch. 4, Naïve Bayesian Classification, 2014.
Related Publications (1)
Number Date Country
20170148038 A1 May 2017 US