The present invention relates generally to social media and social networks and, more particularly, to system and method for determining the effectiveness and engagement of a crowd network based on the message-passing between the nodes of a network. This measure is called as network entropy.
In the recent past it is increasingly being observed that social media plays an important role in putting a dormant crowd into action. The most recent examples of social media fueled crowd activism are the Egyptian revolution, the Spanish May 15th movement, and the most recent Occupy Wall Street protests that spread out across many cities in the United States. The common underlying theme in all these examples of activism is the intelligent use of social media such as Facebook, Twitter, etc. to bring a crowd into action. Beneath any crowd is a network of people who are engaged towards a common goal for a crowd to perform a purposeful action. Therefore, it is desirable to be able to understand why and how the engagement of a crowd changes from time to time. For example, some movements start off great but die out soon, some movements start out small but pick up momentum and turn into a full-fledged revolutions.
Crowds can be represented as complex networks. Most real complex networks, such as telecommunication networks, are not homogeneously linked by similar type of edges. Most real complex networks display certain characteristic properties like small world phenomenon and scale free distributions. Today's hyper-connected networks of people, information, and devices pose an entirely different challenge—in order to extract the value of today's crowd networks, it is not enough to understand simply the connections of the network. It is also important to understand the activity on those connections, representing the engagement of the crowd, and how it grows or shrinks over time and under what conditions. Only then the true value of the crowd network can be extracted.
Complex network properties like clustering, shortest path lengths, etc. are traditional measures for examining properties of a graph network. In the case of a social network, the notion of ‘engagement’ or ‘activity’ between the people/nodes in the network is an important aspect. Engagement has certain properties like working towards a common goal or working in parallel on different tasks to achieve a common higher task, etc. Up until now, however, the relationship between engagement and network structures has not been well-defined. This invention provides a method for computing the value of a social network based on network entropy and for extracting the network structures that contribute to that value.
The present invention proposes that the value of a crowd network comes from network entropy and a very well engaged crowd has more value than a poorly engaged crowd. The engagement (and the resultant value generated from the engagement) of a crowd network can be measured in terms of the entropy of the network. The entropy is calculated as a function of the probability distribution of incoming and outgoing messages, which represent the entropy or uncertainty in the activity over the network. In this way, the activity occurring over a network in terms of message exchange is translated as a measure of the value of the network. An evolutionary algorithm is also presented to optimize the entropy of a network by successively changing the network topology. Results indicate that the value of a crowd network very closely relate to its small world-ness, sparsity, and connectedness.
In one novel aspect, a method for computing the value of a crowd network by a server computer is proposed. First, the server collects statistics information representing communication activities among a total of n communication nodes in the crowd network. Next, the server constructs an adjacency matrix A, an incoming message probability matrix X, and an outgoing message probability matrix Y of the crowd network. Next, the server computes an inbound entropy Hin of the crowd network using matrices A and X, and an outbound entropy Hout of the crowd network using matrices A and Y. Finally, the server calculates the value of the crowd network using the inbound entropy Hin, the outbound entropy Hout, and a pre-defined weighting parameter α, and outputs the value indicating an engagement level of the crowd network.
In one embodiment, the crowd is formed by users registered in a social network website, such as Facebook, LinkedIn, or Twitter. If users i and j are connected (e.g., friends in Facebook, connected in LinkedIn, or follower in Twitter), then there is an edge/link in the graph and the corresponding element aij in the adjacency matrix is 1. Also, the incoming and outgoing message probabilities can be determined by how much interaction has occurred between these two users. In another embodiment, a crowd network consists of all the email users in a public email server such as Gmail, Yahoo Mail, or Hotmail. In this case, the incoming and outgoing message probabilities between two email users can be simply calculated from the number of emails received and sent among the email users. In yet another embodiment, all mobile phone users from a service provider such as Verizon, AT&T, or Sprint forms a crowd network. The message probability matrices can be determined by the usage of phone calls and/or texting among the mobile phone users. These are examples of explicit message-passing in networks. The same can be imagined for implicit message-passing. For example, passing of social cues such as buying a product.
In another novel aspect, the relationship between the engagement/entropy of a crowd and its corresponding network structural parameters is identified. In one embodiment, an entropy maximization algorithm is proposed. The main objective of the algorithm is to find a network topology that maximizes the entropy of the network (e.g., the value of the crowd) in terms of the incoming and outgoing probability distributions of the crowd network. From this process, it can be seen that crowds which are better engaged do seem to be small worlds with high clustering and low average shortest path length. They also generally seem to be densely connected into a single giant connected component. Understanding more deeply the aspects of crowd network structures that contribute to the overall collective value will enable new use and applications of crowds and effective leverage of crowd behaviors for future problem solving.
In the above-mentioned examples, the network structure and communication activity are autonomous. For example, whoever participated in a social network website is not planned. Also the communication activity among all the users is not controlled as well. However, in some other cases, a crowd network could be a controlled network, i.e. the nodes and communication among the nodes are managed and can be adjusted. For example, a managed organization where people's communication activity pattern is ordered to follow certain structure such as management structure. Power grid is another controlled connected network where the structure can be adjusted for better performance. In these applications, a higher value of the crowd is likely to imply a better efficiency and effectiveness. Therefore, by adjusting the network to reach its maximum entropy (e.g., value), the optimal operational efficiency can be achieved. It is another innovative aspect of the present invention that a heuristic method is proposed to finding the network topology and communication activity pattern that produce the maximum value of the crowd.
Other embodiments and advantages are described in the detailed description below. This summary does not purport to define the invention. The invention is defined by the claims.
The accompanying drawings, where like numerals indicate like components, illustrate embodiments of the invention.
Reference will now be made in detail to some embodiments of the invention, examples of which are illustrated in the accompanying drawings.
A crowd network is very different from typical static and structured communication networks. Although underlying the crowd network is a social network of connected individuals via communication links, the phenomenon of crowd happens only in the presence of certain characteristic conditions. One of the central conditions is the messaging activity that happens between the individuals which form the crowd. The so-called value of a crowd network comes from the ability of the individuals to interact with each other and communicate with each other by sending and receiving messages. It is because of this activity on the communication links of the crowd network that creates the power and capability to achieve valuable actions ad results.
A crowd network has structural and behavioral complexities not present in a more static communication network. As such, crowd networks have significantly different dynamics and models of information and opinion spreading than a more predictable device communication network. Key differences are the social clustering that happens in a crowd network—effectively creating clusters of large strong connected components, as well as a typically small amount of weak links that connect the entire network into one. Furthermore, the activity of communications across the connected links as well as the triggers for growth or decline of that activity creates emergent network effects whose drivers today are largely unknown and undefined. It has been shown that the value of the crowd response in real-life crowd networks hinged on communication activity throughout the network, i.e., engagement and growth of that engagement.
In one novel aspect, the value of a crowd network can be defined in terms of the overall engagement of the individuals within the crowd. It is proposed that a very well engaged crowd has more value than a crowd that is not engaged. If the activity among the nodes is high, then it means that the crowd is better engaged. On the other hand, if the activity among the nodes is low, then it means that the crowd is less engaged. The activity between any two nodes can be measured as the number of messages exchanged between the two nodes. Therefore, engagement is hypothesized as messages being exchanged over the crowd network, and the value of the crowd network is calculated from the network entropy of incoming and outgoing message probability distributions.
As illustrated in
Let a crowd network C be represented by a graph G=(V, E), where V={v1, v2, . . . , vn} and E={e1, e2, . . . , em}, where n is the number of nodes and m<=n(n−1)/2 is the number of links. The graph G is represented by an adjacency matrix A={aij|i, j<=n} in which each element aij=[0, 1] denotes the existence of a communication link between nodes i and j. If no link exists, then aij=0. Otherwise, aij=1. In general, a graph representing a network can be directional (where the link has direction) or non-directional (where the links has no direction). Furthermore, a graph can have weighted links with a numerical value associated to each link or un-weighted links having uniformly equal weight. More specifically, the direction of a link corresponds to the direction of receiving (incoming) or sending (outgoing) messages between two nodes, and the weight of each link corresponds to the probability of receiving or sending messages between the two nodes.
The probability distribution of such weighted directional graph can also be represented by two matrices X and Y. The incoming messaging probability distribution matrix X={xij|i,j<=n}, where each element in the matrix is the probability of node i receiving messages from node j. The outgoing messaging probability distribution matrix Y={yij|i,j<=n}, where each element in the matrix is the probability of node i sending messages to node j. The size of the matrices n is equal to the number of nodes. In the example of
Once the adjacency Matrix A, the incoming message probability matrix X, and the outgoing message probability matrix Y are constructed for a given crowd network, its network entropy and value can then be computed. The entropy of a network is the measure of the uncertainty in a network. It measures the network's heterogeneity in terms of the diversity in the incoming and outgoing message distributions. The value of a node is calculated in terms of the incoming and outgoing message entropy, which is the entropy of the incoming probability and outgoing probability distributions associated with the particular node. Note that the entropy here does not try to measure the actual information content of the messages being exchanged, but only the randomness in the incoming and outgoing message probability distributions. The cumulative incoming and outgoing message entropies of a network are calculated as the summation of all the individual incoming and outgoing node entropies. Thus,
Finally, the total value of a crowd network C is calculated as a weighted measure of the incoming and outgoing entropies of the network. Thus, the value is represented as a function of weighing variable α. The variable allows for weighing in the incoming and the outgoing network entropies.
V(α)=αHin+(1−α)Hout (2)
The data source servers are used for the server computer to retrieve statistics information representing communication activity of a particular crowd network. In a first example, data source server 304 is a public email server such as Gmail, Yahoo Mail, or Hotmail, which contains all the email user accounts and email repository among all the email users that form the crowd. In this example, data retrieved from server 304 include the email user accounts and the number of emails exchanged among the email users. Based on such statistics, the incoming message probability (e.g., likelihood of receiving an email) between two users and the outgoing message probability (e.g., likelihood of sending an email) between two users can be constructed.
In a second example, data source server 305 contains all the phone call history of customers of a mobile service provider. In this example, all mobile phone users from a service provider such as Verizon, AT&T, or Sprint form a crowd network. The adjacency and message probability matrices can be determined by the usage of phone calls and/or texting among the mobile phone users. For example, the outgoing message probability (e.g., making a phone call) from user A to B equals the number of phone calls made from user A to B divided by the total number of phone calls made from user A.
In a third example, data source server 306 is a social network website, in which all the registered users and communications (messages and postings etc.) are stored on the server 306. The crowd is formed by users registered in the social network website, such as Facebook, LinkedIn, or Twitter. If user i and user j are connected (e.g., friends in Facebook, connected in LinkedIn, or follower in Twitter), then there is an edge/link in the graph and the corresponding element aij in the adjacency matrix is 1. Also, the incoming and outgoing messaging probabilities can be determined based on how much interaction has occurred between these two users.
In a fourth example, data source server 307 represents other networks in which messages are implicitly communicated with each member of the network. There exist some other types of networks, e.g., influence networks or citation networks, in which messages are implicitly sent and received. For example, one member of the network is influenced by another member of the network to buy a pair of NIKE shoes without being explicitly told to do so. These types of networks are also applicable as long as the statistics of such implicitly communicated messages can be collected by data source server 307.
From the above examples, it can be seen that as the value of a crowd can be quantified under Equation (2), the effectiveness or engagement of different social networks (e.g., Facebook vs. LinkedIn) can be compared. These comparisons could have multiple usages. For example, the result of comparing social network websites can be utilized as one of the criteria in selecting a website as an advertisement vendor.
Crowds can be represented as complex networks with different topologies and internal structural properties like small worlds, community organization, component structure, sparseness, etc. Therefore, crowds can also be analyzed for their complex network properties like the clustering in a network, shortest path lengths, size of the giant component etc. One way to analyze the relationship between the network topologies/structural properties with the value of the network is via the application of an entropy maximization algorithm. The main objective of the algorithm is to find a network topology that maximizes the entropy of the crowd network (value of the crowd) in terms of the incoming and outgoing probability distributions of the crowd network.
Initialization is performed at block 601 to set the network size n, the message probability distribution, and the maximum number of links m. In addition, working variables including Iterator i and initial value of crowd VC are set to be 0. For initial iteration i=0, adjacency matrix Ai with size n×n is randomly generated. At block 602, checking is performed to make sure that the matrix has less than m links. If the number of links exceeds m, a new iteration starts at block 607. At block 603, the value of crowd represented by matrix Ai is calculated using network entropy. If the newly calculated value Vci is greater than the existing value Vc, a better topology is found and Vc is set to the newly found value Vci at block 606 and a new iteration starts at block 607. Otherwise, if Vci is not greater than Vc, a checking at block 605 is made to see if Vc has ever increased in the last n2 iterations. If not, the process ends. Otherwise, a new iteration starts at block 607, which randomly flips each element aij with probability (p) in the adjacency matrix Ai.
a) shows that as the cumulative entropy increases so do the number of active links (m) in a network. The total number of active links represents number of connections between nodes in the network. This means that the value of a crowd increases with the number of people communicating with each other, which seem to be self-evident.
From the above analysis, first, it is observed that the entropy optimization process leads to networks which are small worlds with increasing clustering and decreasing shortest path length. Small worlds are characterized by low average shortest path length and high clustering. This phenomenon is observed when the average distance between two nodes grows logarithmically with the size of network. Thus, the value of a crowd in terms of network engagement is inherently associated with its small world-ness. Second, as the network gets denser in terms of the number of links the engagement increases. This means that the average number of nodes an individual node connects to increases with the entropy. This may seem self-evident as the number of links increase through the optimization process. The value of crowd then is a function of the average number of nodes a node connects to (average degree). Third, the connectedness of a network is usually measured in terms of the number of components. As it is seen from
In one or more exemplary embodiments, the functions described above may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored on or transmitted over as one or more instructions or code on a computer-readable (processor-readable) medium. Computer-readable media include both computer storage media and communication media including any medium that facilitates transfer of a computer program from one place to another. A storage media may be any available media that can be accessed by a computer. By way of example, and not limitation, such computer-readable media can comprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that both can be used to carry or store desired program code in the form of instructions or data structures, and can be accessed by a computer. In addition, any connection is properly termed a computer-readable medium. For example, if the software is transmitted from a website, server, or other remote source using a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, then the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of medium. Disk and disc, as used herein, include compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk, and blue-ray disc where disks usually reproduce data magnetically, while discs reproduce data optically with lasers. Combinations of the above should also be included within the scope of computer-readable media.
Although the present invention has been described in connection with certain specific embodiments for instructional purposes, the present invention is not limited thereto. For example, although a crowd is referred to as a social network of people communicating with each other in most examples, the crowd can be extended to other connected networks such as machine-to-machine networks. In one specific example of a power grid, power supplies are nodes and power lines are links. Accordingly, various modifications, adaptations, and combinations of various features of the described embodiments can be practiced without departing from the scope of the invention as set forth in the claims.
Number | Name | Date | Kind |
---|---|---|---|
5608620 | Lundgren | Mar 1997 | A |
6466928 | Blasko et al. | Oct 2002 | B1 |
6473084 | Phillips et al. | Oct 2002 | B1 |
6606615 | Jennings et al. | Aug 2003 | B1 |
6792399 | Phillips et al. | Sep 2004 | B1 |
6795793 | Shayegan et al. | Sep 2004 | B2 |
6912511 | Eliezer et al. | Jun 2005 | B1 |
7130777 | Garg et al. | Oct 2006 | B2 |
7155510 | Kaplan | Dec 2006 | B1 |
7236953 | Cooper et al. | Jun 2007 | B1 |
7249077 | Williams | Jul 2007 | B2 |
7406459 | Chen et al. | Jul 2008 | B2 |
7415436 | Evelyn et al. | Aug 2008 | B1 |
7519562 | Vander Mey et al. | Apr 2009 | B1 |
7519564 | Horvitz | Apr 2009 | B2 |
7542881 | Billiotte et al. | Jun 2009 | B1 |
7546285 | Baker, Jr. | Jun 2009 | B1 |
7552076 | Uenohara et al. | Jun 2009 | B1 |
7555454 | Cooper et al. | Jun 2009 | B2 |
7565313 | Waelbroeck et al. | Jul 2009 | B2 |
7566270 | Amaitis et al. | Jul 2009 | B2 |
7590554 | Chen et al. | Sep 2009 | B2 |
7668773 | Pruitt | Feb 2010 | B1 |
7672904 | Powell | Mar 2010 | B2 |
7707062 | Abramowicz | Apr 2010 | B2 |
7716329 | Lee et al. | May 2010 | B2 |
7742968 | Guler et al. | Jun 2010 | B2 |
7742972 | Lange et al. | Jun 2010 | B2 |
7773538 | Wang et al. | Aug 2010 | B2 |
7865418 | Uenohara et al. | Jan 2011 | B2 |
7880741 | Proebsting et al. | Feb 2011 | B2 |
7974904 | Frischer | Jul 2011 | B2 |
7996296 | Lange | Aug 2011 | B2 |
8229824 | Berg et al. | Jul 2012 | B2 |
8275695 | Lange et al. | Sep 2012 | B2 |
8341065 | Berg et al. | Dec 2012 | B2 |
8370249 | Lange et al. | Feb 2013 | B2 |
8380595 | Fogarty | Feb 2013 | B1 |
8396777 | Fine et al. | Mar 2013 | B1 |
8583470 | Fine et al. | Nov 2013 | B1 |
8606618 | Dabke | Dec 2013 | B2 |
8612331 | Hanson et al. | Dec 2013 | B2 |
8788386 | Fogarty | Jul 2014 | B1 |
20020038321 | Keeley | Mar 2002 | A1 |
20020073009 | Hogg et al. | Jun 2002 | A1 |
20020095305 | Gakidis et al. | Jul 2002 | A1 |
20020107722 | Laurin et al. | Aug 2002 | A1 |
20030074166 | Jackson et al. | Apr 2003 | A1 |
20030078829 | Chen et al. | Apr 2003 | A1 |
20040002891 | Chen et al. | Jan 2004 | A1 |
20040153456 | Charnock et al. | Aug 2004 | A1 |
20040162751 | Tsyganskiy et al. | Aug 2004 | A1 |
20040176994 | Fine et al. | Sep 2004 | A1 |
20050021441 | Flake et al. | Jan 2005 | A1 |
20050108120 | Malka et al. | May 2005 | A1 |
20050114161 | Garg et al. | May 2005 | A1 |
20050228739 | Leibowitz | Oct 2005 | A1 |
20050256800 | Hogg et al. | Nov 2005 | A1 |
20060117303 | Gizinski | Jun 2006 | A1 |
20060122927 | Huberman et al. | Jun 2006 | A1 |
20060205483 | Meyer et al. | Sep 2006 | A1 |
20060206365 | Boardman et al. | Sep 2006 | A1 |
20060224442 | Round et al. | Oct 2006 | A1 |
20070078699 | Scott et al. | Apr 2007 | A1 |
20070078756 | Hogg et al. | Apr 2007 | A1 |
20070136105 | Huberman et al. | Jun 2007 | A1 |
20070136429 | Fine et al. | Jun 2007 | A1 |
20070198387 | Uenohara et al. | Aug 2007 | A1 |
20070250429 | Walser et al. | Oct 2007 | A1 |
20070256093 | Hiler | Nov 2007 | A1 |
20070288579 | Schunemann | Dec 2007 | A1 |
20080120166 | Fernandez et al. | May 2008 | A1 |
20080140786 | Tran | Jun 2008 | A1 |
20080222191 | Yoshida et al. | Sep 2008 | A1 |
20080281610 | Yoshida et al. | Nov 2008 | A1 |
20080288209 | Hunt et al. | Nov 2008 | A1 |
20080288326 | Abramowicz | Nov 2008 | A1 |
20080319825 | LaComb et al. | Dec 2008 | A1 |
20090024549 | Johnson | Jan 2009 | A1 |
20090073174 | Berg et al. | Mar 2009 | A1 |
20090076939 | Berg et al. | Mar 2009 | A1 |
20090076974 | Berg et al. | Mar 2009 | A1 |
20090094039 | MacDonald et al. | Apr 2009 | A1 |
20090182624 | Koen et al. | Jul 2009 | A1 |
20090198565 | Pluschkell et al. | Aug 2009 | A1 |
20090254386 | Wang et al. | Oct 2009 | A1 |
20090254475 | Pennock et al. | Oct 2009 | A1 |
20090313350 | Hundscheidt et al. | Dec 2009 | A1 |
20090327054 | Yao et al. | Dec 2009 | A1 |
20100058249 | Sarkar et al. | Mar 2010 | A1 |
20100179930 | Teller et al. | Jul 2010 | A1 |
20110137848 | Stephens | Jun 2011 | A1 |
20120089617 | Frey | Apr 2012 | A1 |
20120130860 | Suzuki et al. | May 2012 | A1 |
20120150585 | Dabke | Jun 2012 | A1 |
20120239608 | Rinearson | Sep 2012 | A1 |
20130262193 | Sundaresan | Oct 2013 | A1 |
20130311455 | Kritt et al. | Nov 2013 | A1 |
20130332468 | Hardas et al. | Dec 2013 | A1 |
20130346501 | Hardas et al. | Dec 2013 | A1 |
20140025690 | Tareen et al. | Jan 2014 | A1 |
20140108426 | Goldberg et al. | Apr 2014 | A1 |
Number | Date | Country |
---|---|---|
2421812 | Jul 2006 | GB |
0186532 | Nov 2001 | WO |
Entry |
---|
Almeida, Daniell, The Relevancy of Group Expertisefor the Accuracy of a Prediction Market, Erasmus School of Economics, May 2010. |
Bao, Yong et al., Comparing Density Forecast Models, Journal of Forecasting, vol. 26, 2007. |
CFO and Crowdcast Launch Financial Prediction Market, PRWeb, Jul. 16, 2009. |
Chen, Kay-Yut et al., Eliminating Public Knowledge Biases in Information Aggregation Mechanisms, HP Laboratories, Mar. 21, 2003. |
Chen, Kay-Yu et al., Forecasting Uncertain Events with Small Groups, HP Laboratories, Jun. 25, 2001. |
Chen, Kay-Yut et al., Predicting the Future, Information Systems Frontier, vol. 5, No. 1, 2003. |
Dubil, Robert, Economic Derivatives Markets-New Opportunities for Individual Investors: A Research Agenda, Financial Services Review, vol. 6, No. 2, pp. 89-104, Summer 2007. |
Harnessing Collective Intelligence to Improve Strategic Planning, Crowdcast, White Paper, 2010. |
Ledyard, John et al., An Experimental Test of Combinatorial Information Markets, Dec. 2007. |
Crowdcast.com web pages, Crowdcast, Jul. 2009. |
Marzban, Caren, The ROC Curve and the Area under it as Performance Measures Weather and Forecasting, vol. 19, 2004. |
Myung, In Jae et al., Maximum Entropy Aggregation of Expert Predictions, Management Science, vol. 42, No. 10, Oct. 1996. |
Pennock, David M. et al., The Power of Play: Efficiency and Forecast Accuracy in Web Market Games, NEC Research Institute Report, 2000-168, Feb. 17, 2001. |
Plott, Charles R., et al., Information Aggregation Mechanisms: Concept, Design, and Implementation for a Sales Forecasting System, California Institute of Technology, Mar. 2002. |
Schrieber, Jared M., The Application of Prediction Markets to Business, MIT, Jun. 2004. |
Ullah, Aman, Entropy, divergence and distance measures with econometric applications, Journal of Statistical Planning and Inference, vol. 48, 1996. |
Vaughan, J.W., Learning from collective preferences, behavior, and beliefs. (Order No. 3381885, University of Pennsylvania). ProQuest Dissertations and Thesis, 152, 2009. |
Wolfers, Justin et al., Interpreting Prediction Market Prices as Probabilities, Jan. 8, 2007. |
Wolfers, J., et al., Prediction markets. The Journal of Economic Perspectives, 18(2), 107-126, 2004. |
Number | Date | Country | |
---|---|---|---|
20140019557 A1 | Jan 2014 | US |