This instant specification relates to clustering data sets.
One of the largest areas of retail loss is in the fraudulent use of bank and credit cards in online transactions. Some current fraud detection systems attempt to identify fraudulent transactions by using predictive models that identify a transaction as fraudulent based on predictive variables such as an average spending amount for a particular purchaser in a transaction. For example, if a purchaser rarely makes purchases of above $100, then a transaction associated with the purchaser for $800 may be indicative of fraud. The average, or typical, spending amount for the individual can be encoded in the predictive variables used by the fraud detection system.
In general, this document describes a probabilistic method for computing indirect relationships between first data based on direct relationships between the first data and second data. For example, merchants can be clustered based on transactions with purchasers. Profiles can then be derived and associated with merchant clusters for use in detecting fraudulent transactions.
In a first general aspect, a computer-implemented method is described. The method includes accessing a data structure that includes information about purchasers, merchants, and financial transactions between the purchasers and the merchants and generating purchaser clusters. Generating purchaser clusters includes clustering the purchasers based on which purchasers make purchases from the same or similar merchants. Each purchaser cluster adopts associations between purchasers belonging to the purchase cluster and merchants from which these purchasers have made purchases.
The method also includes generating merchant clusters, where generating the merchant clusters includes clustering merchants based on which merchants are associated with the same or similar purchase clusters and outputting profile information that characterizes typical purchases associated with one or more of the merchant clusters for use in detecting fraudulent transactions.
In a second general aspect, a system is described. The system includes a data structure that, in turn, includes information about purchasers, merchants, and financial transactions between the purchasers and the merchants. The system also includes a purchaser clusterer to generate purchaser clusters including clustering the purchasers based on which purchasers make purchases from the same or similar merchants. Each purchaser cluster adopts associations between purchasers belonging to the purchase cluster and merchants from which these purchasers have made purchases. The system also includes a merchant clusterer to generate merchant clusters comprising clustering merchants based on which merchants are associated with the same or similar purchase clusters and an interface to output profile information that characterizes typical purchases associated with one or more of the merchant clusters for use in detecting fraudulent transactions.
The systems and techniques described here may provide one or more of the following advantages. First, merchants may be clustered based on how purchasers relate to merchants regardless of whether the system has any information about how the merchants related to each other. Additionally, the soft clustering of merchants patronized by a cardholder may enable cardholder spending to be characterized in a way that is both descriptive and statistically significant. By producing a time average in each merchant category, a model can create a detailed pattern of cardholder spending. Changes in this detailed pattern of spending can signal fraud.
The details of one or more embodiments are set forth in the accompanying drawings and the description below. Other features and advantages will be apparent from the description and drawings, and from the claims.
Like reference symbols in the various drawings indicate like elements.
This document describes systems and techniques for generating profile information associated with clusters of merchants, where the profile information can be used to detect possible fraudulent transactions based on deviations from, for example, spending averages associated with the clusters of merchants. For example, if a merchant belongs to a particular merchant clusters that has norm spending average of about $40.00 per transaction, a transaction with the merchant for $450.00 may indicate the transaction is fraudulent. Furthermore, spending associated with a particular merchant cluster relative to total spending can be monitored. For example, if spending in a particular merchant cluster suddenly becomes more prominent in comparison with total spending, this may be an indication of fraud.
In some implementations, a clustering system may generate merchant clusters by first grouping purchasers based on whether the purchasers have a similar frequency of transactions with a similar set of merchants. The clustering system may then use the groups of purchasers, or purchaser clusters, as a data source to create merchant clusters. For example, the clustering system can determine—for each purchase cluster—a probability that a transaction (e.g., between a merchant and purchaser) is associated with that purchaser cluster. The clustering system may then cluster merchants associated with the analyzed transactions based on whether the merchants' transactions have a similar distribution of probabilities.
In a simple illustrious example, a first merchant may have first and second transactions with probabilities 0.3 and 0.7, respectively, that the transactions are associated with a first purchase cluster. A second merchant may have third and fourth transactions with probabilities of 0.25 and 0.6, respectively. The clustering system may cluster the first and second merchant into a merchant cluster based on the similar distribution of probabilities that their transactions are associated with the first purchase cluster. If, on the other hand, the second merchant had a probability distribution of 0.9 and 0.45, the clustering system may have grouped the merchants in separate merchant clusters because of the dissimilarity in probability distribution.
In more complicated examples, the merchants may be associated with many transactions, which are in turn, associated with a multitude of purchaser clusters. Additionally, the clustering system can include similarity threshold(s) that guide how the clustering system determines how similar the probability distributions should be before merchants are associated with a particular cluster (or multiple clusters), which is explained in more detail below.
Numerically labeled arrows of
The information 110 can include purchaser identifiers (e.g., an identifier associated with an account involved in a transaction), merchant identifiers involved in transactions, spending amounts of the transactions, time/date stamps associated with the transactions, etc. Merchant identifiers and purchaser identifiers are also referred to herein as “merchants” and “purchasers” for simplicity of explanation.
The clustering system 102 can include a clusterer 112 that groups, or clusters, purchasers based on, for example, whether they made purchases from the same set of merchants with a similar frequency. The clusterer 112 also can cluster merchants. For example, the cluster 112 can group merchants based on probabilities that transactions associated with the merchants are associated with substantially similar purchaser clusters. This will be explained in greater detail in association with the following figures.
The clustering system 102 may include a profile generator 114. The profile generator 114 can derive profile variables associated with the merchant clusters for inclusion in merchant cluster profiles that describe typical activity associated with merchants that belong to particular merchant clusters. The merchant cluster profiles 116 may be transmitted by the clustering system 102 to a model database 118 as indicated by an arrow labeled “2.”
For example, a merchant cluster profile 116 can include variables associated with particular merchant clusters, where the variables indicate a typical amount of money spent per transaction, per time period, a typical number of transactions per time, etc.. In some implementations, the model database 118 can store other types of variables used to predict fraud such as variables associated with particular merchants, variables associated with particular purchasers, variables associated with particular purchaser clusters, etc.
The fraud detection system 104 can access the information stored in the model database 118 as indicated by an arrow labeled “3.” The fraud detection system 104 can train models using the information stored in the database 118, where the models are used to detect fraudulent transactions. For example, the models can be implemented using a neural network that applies optimization theory and statistical estimation to the variables in order to identify transactions that deviate from a norm associated with the particular kind, or type, of transaction analyzed by the fraud detection system 104.
The fraud detection system 104 can include model logic 120, which applies the model (e.g., trained neural network) to a transaction stream 122 that is received at the fraud detection system 104 as indicated by an arrow labeled “4.” In some implementations, the transaction stream 122 can include posts of completed transactions transmitted from merchants 124 involved in the transactions. In other implementations, the transaction stream 122 can include completed transactions associated with a financial institution that transferred payment as part of the transaction (e.g., credit card companies 128 and/or banks 128).
In yet other implementations, the transaction stream can include currently pending transactions. For example, before a credit card company 126 approves a payment to a particular merchant, the credit card company 126 may transmit the transaction to the fraud detection system 104. If the fraud detection system 104 determines that the transaction is likely fraudulent, the credit card company 126 can refuse to process payment for the transaction. If, on the other hand, the fraud detection system 104 determines that the transaction is likely valid, the fraud detection system 104 can transmit a message indicating that the credit card company 126 should process payment for that transaction.
The fraud detection system 104 can use the model logic 122 to score transactions, where the score may indicate a likelihood that the transaction is fraudulent (or valid). The fraud detection system 104 can transmit the scored transaction stream 130 to the fraud alert system 106 as indicated by and arrow labeled “5.”
In some implementations, the fraud alert system 106 can transmit alerts to one or more parties associated with a fraudulent transaction as indicated by an arrow labeled “6.” For example, the fraud alert system 106 may prompt an operator to call a bank cardholder associated with a transaction that is likely fraudulent. In another example, a fraud alert system can transmit a message to a merchant or credit card company indicating that a pending transaction is fraudulent and that the party should cancel or decline the transaction.
In another implementation, the fraud alert system can transmit information that indicates that a particular transaction is likely not fraudulent. For example, if a party to the transaction submits the transaction to the fraud detection system to determine whether to approve a payment or complete the transaction, the fraud alert system can transmit information back to the transmitting party indicating that the transaction should be processed because it is likely not fraudulent.
In yet other implementations, the scored transaction stream 130 can be forwarded to the transaction database 108 for use in updating the merchant cluster profiles or other variables associated with fraud, and consequently, the model used to identify fraudulent transactions.
Components of the system 100, such as the databases 108 and 118, the clustering system 102, the fraud detection system 104, and the fraud alert system are depicted in
In some implementations, the clustering system 200 can co-cluster categorical data as opposed to clustering continuous multivariate data; however, the same rational may apply to co-clustering as is applied to continuous clustering. Probabilistic, or “soft,” co-clustering may permit each entity (or observation) to have a probability of membership in each cluster. This may be appropriate when the clustering is an approximate model of a population so that some entities might belong to more than one cluster.
Before describing the elements of
Referring to
A bipartite graph can include two groups of entities—subjects and objects—in a graph, where every edge (also referred to as a “verb”) begins on a subject and ends on an object. If the subjects represent people, objects represent goods, and a relationship between them is “person purchases object.” The clustering system 200 can represent a purchasing history of a group of people by weighting on the edges to represent frequency of purchase. Similarly, if the subjects represent documents, the objects represent words, and the verb is “contains,” then the edges of the graph can represent a frequency of occurrence of a word within a document. For the next several paragraphs, the terms subject, verb, and object are used to describe the elements of a graph used in clustering.
In some implementations, mathematically clustering subjects based on such probability vectors (e.g., probabilities in a row of a table like table 420) identifies similarities between subjects based on their relationships with objects. For example, the clustering system 200 may identify that subjects 1 and 2 have similar probability vectors, whereas subject 3 has a different probability vector than either subject 1 or subject 2.
If subject 1 and subject 2 are combined into a single cluster (or super vertex) and subject 3 is placed in its own cluster, then objects 1 and 2 can be identified as related based on their connection to the subject-1-subject-2 cluster; however, objects 3 and 4 seem only related to one another by their relationship to subject 3. Co-clustering can include a technique for computing these indirect relationships among subjects and indirect relationships among objects.
In some implementations, soft co-clustering of subject and objects is accomplished in two phases using two different generative models. Phase I can use the frequency of objects associating with a given subject (e.g. the row data in Table 340 of
In some implementations, soft co-clustering is implemented using a generative model to create weights in the SVOF graph. The weights m on edges emanating from a subject “i” to all objects include integers chosen from a multinomial distribution with given probability p (where p is bolded to indicate it is a vector of values). The probability p, in turn, may be chosen according to a Dirichlet distribution that uses an intensity x. The intensity x may be chosen from a finite set of possible intensity vectors X according to a discrete distribution. A finite choice of C possible intensity vectors X can correspond to a membership of a subject in any of C subject clusters.
Latent variables in the DMM model 450 include an intensity matrix X and a probability vector {right arrow over (w)} according to some implementations. Rows of the intensity matrix X can correspond to subject clusters and columns can correspond to objects. The subject clusters may be randomly chosen according to a discrete distribution with a probability vector {right arrow over (w)}.
In some implementations, the clustering on subjects provided by the DMM model 450 is soft in the sense that a membership of a subject “i” in a subject cluster “c” is a probability. For example, for a given subject “i” the probability that it came from cluster “c” is dependent on the weights/frequencies m on the outgoing edges of subject “i,” where the weights/frequencies can be alternatively expressed using values in the subject's row in a table like the table 320 of
Given a fit DMM model as described in table 4C, the probability given in the above equation can be exactly computable. In fact there is a probability vector describing the membership of subject “i” in each of the subject clusters, according to some implementations. This probability vector describing the membership may be used in the “soft,” or probabilistic, co-clustering of subjects.
Although the example phase I DMM model alone does not cluster objects, it can provide a kind of data source for clustering them. For example, a probability that a single object “j” was chosen from a subject cluster “c,” is given by p(component=c|{right arrow over (e)}j) where {right arrow over (e)}j is zero in all coordinates except the j-th coordinate where it is 1. So, the DMM model can give a probability vector that an object was chosen from each subject cluster. The example phase II generative model clusters objects may be based on this subject cluster probability vector p.
In one implementation, the example phase II model is a two stage Dirichlet Mixture (DM) Model that chooses probability vectors p based on a distinct intensity vector X[k,.], which is a row from an intensity matrix X. This row choice is made according to a discrete object cluster probability vector w.
For each object “i,” the example Phase II DM model 510 provides a probability that object “i” belongs to an object cluster “c.”
Object “i” can be completely characterized by probability vector {right arrow over (p)}i just as subject “i” can be characterized by the frequency vector {right arrow over (m)}i in the example phase I DMM 450. This demonstrates that for any object “i,” the phase II DM model 510 can provide a soft clustering.
Referring to
As previously described, the purchaser clusterer 206 can include a three-stage DMM model 210 to cluster purchasers. For example, the DMM model 210 can include a multinomial model 212, a Dirichlet model 214, and a discrete model 216, where the output of one model may be used to parameterize a second model. Similarly and as previously described, the merchant clusterer 208 can include a DM model 218 used to cluster the merchants. The DM model 218 can include a Dirichlet model 220 and a discrete model 222 such as the models described in
The clusterer 204 also can include a maximum likelihood estimator 224 to estimate parameters of a DMM model such as the DMM model described in
In some implementations the maximum likelihood estimator 224 can estimate parameters of the DMM model using a cross the entropy (CE) method. In the following general description, the CE method is implemented as a Monte Carlo technique. For example, the CE method can place a prior distribution on all parameters to be estimated. One choice for a vector parameter is {right arrow over (x)}˜N({right arrow over (μ)},σI)., a multivariate normal distribution with a diagonal covariance matrix. The mean and the standard deviation of this distribution are variable but bounded. The chosen parameter vectors may dictate a negative log likelihood contribution, θ({right arrow over (m)}j;{right arrow over (x)}i), for each simulated parameter {right arrow over (x)}i, and each data record {right arrow over (m)}j.
In one implementation, the maximum likelihood estimator (MLE) 224 can implement a CE maximum likelihood estimation algorithm as follows. First, for each parameter, the MLE can select several xi˜N(μi,σi). Second, for all parameter guesses {right arrow over (x)}i, the MLE can choose q exemplars that have the smallest negative log likelihoods
These exemplars may be referred to as the elite set of parameter guesses.
Third, the MLE can compute the mean and the standard deviations for the elite set. On convergence, the MLE can end the algorithm. Otherwise the MLE can return to the second step. In this way, the MLE can fit the phase I DMM model and the phase II DM model. The clusterer 204 can then output information 226 for each merchant that is indicative of probabilities that a particular merchant is associated with each merchant cluster (i.e., merchant cluster membership probabilities).
In some implementations, the cluster 204 may store the information 226 in a database (not shown) as a matrix of probabilities. A profile generator 228 included in the clustering system 200 can access the output information 226 for use in generating profile variables associated with merchant clusters. For example, each transaction in the data set may be divided by a transaction allocator 230 into merchant clusters according to the probability that the merchant belongs in each cluster.
A profile variable generator 232 can compute profile variables for each cluster, and those variables along with other variables may be used to train models that predict, for example, bank card fraud. Additionally, for each merchant in a transaction, the amount may be divided by a transaction spending amount allocator 234 according to cluster probability membership. The profile variable generator 232 may then compute profile variables as mentioned above. The cluster profile variables 236 and other variables (not shown) can be used as inputs to a model which predicts the likelihood of fraud.
The system 600 includes a processor 610, a memory 620, a storage device 630, and an input/output device 640. Each of the components 610, 620, 630, and 640 are interconnected using a system bus 650. The processor 610 is capable of processing instructions for execution within the system 600. The processor may be designed using any of a number of architectures. For example, the processor 610 may be a CISC (Complex Instruction Set Computers) processor, a RISC (Reduced Instruction Set Computer) processor, or a MISC (Minimal Instruction Set Computer) processor.
In one implementation, the processor 610 is a single-threaded processor. In another implementation, the processor 610 is a multi-threaded processor. The processor 610 is capable of processing instructions stored in the memory 620 or on the storage device 630 to display graphical information for a user interface on the input/output device 640.
The memory 620 stores information within the system 600. In one implementation, the memory 620 is a computer-readable medium. In one implementation, the memory 620 is a volatile memory unit. In another implementation, the memory 620 is a non-volatile memory unit.
The storage device 630 is capable of providing mass storage for the system 600. In one implementation, the storage device 630 is a computer-readable medium. In various different implementations, the storage device 630 may be a floppy disk device, a hard disk device, an optical disk device, or a tape device.
The input/output device 640 provides input/output operations for the system 600. In one implementation, the input/output device 640 includes a keyboard and/or pointing device. In another implementation, the input/output device 640 includes a display unit for displaying graphical user interfaces.
The features described can be implemented in digital electronic circuitry, or in computer hardware, firmware, software, or in combinations of them. The apparatus can be implemented in a computer program product tangibly embodied in an information carrier, e.g., in a machine-readable storage device or in a propagated signal, for execution by a programmable processor; and method steps can be performed by a programmable processor executing a program of instructions to perform functions of the described implementations by operating on input data and generating output. The described features can be implemented advantageously in one or more computer programs that are executable on a programmable system including at least one programmable processor coupled to receive data and instructions from, and to transmit data and instructions to, a data storage system, at least one input device, and at least one output device. A computer program is a set of instructions that can be used, directly or indirectly, in a computer to perform a certain activity or bring about a certain result. A computer program can be written in any form of programming language, including compiled or interpreted languages, and it can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.
Suitable processors for the execution of a program of instructions include, by way of example, both general and special purpose microprocessors, and the sole processor or one of multiple processors of any kind of computer. Generally, a processor will receive instructions and data from a read-only memory or a random access memory or both. The essential elements of a computer are a processor for executing instructions and one or more memories for storing instructions and data. Generally, a computer will also include, or be operatively coupled to communicate with, one or more mass storage devices for storing data files; such devices include magnetic disks, such as internal hard disks and removable disks; magneto-optical disks; and optical disks. Storage devices suitable for tangibly embodying computer program instructions and data include all forms of non-volatile memory, including by way of example semiconductor memory devices, such as EPROM, EEPROM, and flash memory devices; magnetic disks such as internal hard disks and removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks. The processor and the memory can be supplemented by, or incorporated in, ASICs (application-specific integrated circuits).
To provide for interaction with a user, the features can be implemented on a computer having a display device such as a CRT (cathode ray tube) or LCD (liquid crystal display) monitor for displaying information to the user and a keyboard and a pointing device such as a mouse or a trackball by which the user can provide input to the computer.
The features can be implemented in a computer system that includes a back-end component, such as a data server, or that includes a middleware component, such as an application server or an Internet server, or that includes a front-end component, such as a client computer having a graphical user interface or an Internet browser, or any combination of them. The components of the system can be connected by any form or medium of digital data communication such as a communication network. Examples of communication networks include a local area network (“LAN”), a wide area network (“WAN”), peer-to-peer networks (having ad-hoc or static members), grid computing infrastructures, and the Internet.
The computer system can include clients and servers. A client and server are generally remote from each other and typically interact through a network, such as the described one. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.
Although a few implementations have been described in detail above, other modifications are possible. For example, the clustering is not limited to clustering merchants or purchasers. In other implementations, the clustering system can be used to perform machine language learning. For example, association grounded semantics (AGS) is a theory of assigning meaning (semantics) to natural language based on the association of each word with all other words. AGS theory holds that each word in a natural language derives its meaning from the words with which it occurs. Thus, a model of word co-occurrence is a model of the meaning of a word. Two words which have the same co-occurrence statistics with other words must have the same meaning because they are substitutable.
In some implementations, soft co-clustering as previously described may permit an understanding of a language without rules composed by an expert. Instead, a grammar can be created from a statistical model, which may—in some implementations—be self improving, robust with respect to inconsistencies in training, and hold some promise of becoming complete.
For example, in a language learning implementation, the subjects can be documents, the verb can be “contains,” and the objects can be words. The interpretation of soft co-clustering would be a clustering of documents according to terminology and a clustering of words according to the context of their occurrence.
In yet other implementations, information other than spending amount or number of transaction can be associated with the merchant clusters. For example, spending frequency and amount statistics can be divided based on fraud or non-fraud categorizations as well as by merchant cluster.
In addition, the logic flows depicted in the figures do not require the particular order shown, or sequential order, to achieve desirable results. In addition, other steps may be provided, or steps may be eliminated, from the described flows, and other components may be added to, or removed from, the described systems. Accordingly, other implementations are within the scope of the following claims.