Method and apparatus for updating records in a database system based on an improved model of time-dependent behavior

FIELD OF THE INVENTION

The invention relates generally to techniques for processing information in a database system, and more particularly to techniques for updating customer signatures or other types of records in a database system.

BACKGROUND OF THE INVENTION

It is desirable in many business applications of database systems to track separately the transaction behavior of each customer. This tracking may be implemented using a customer-specific record referred to herein as a customer signature. For example, a customer signature for buying behavior may contain information on the likely place of purchase, value of goods purchased, type of goods purchased, and timing of purchases for the given customer. The signature may be updated whenever the customer makes a transaction, and, because of storage limitations, the updating may be able to use only the new transaction and the summarized information in the current customer signature.

The above-described customer signatures may be provided using relative frequency distributions. Such distributions are also commonly known as histograms. Updating histograms sequentially is not difficult when observations are randomly sampled. If {circumflex over (π)}

n

is a vector of current histogram probabilities and X

n+1

is a characteristic of a current transaction, represented as a vector of 0's except for a 1 in a cell containing an observed value, then the sequentially updated vector of histogram probabilities is

{circumflex over (π)}

n+1

=(1

−w

n+1

){circumflex over (π)}

n

+w

n+1

X

n+1

, (1)

where w

n+1

=1/(n+1) and {circumflex over (π)}

0

=0. Updating thus requires only the most recent transaction, the number of transactions made so far and the current summary. This is known as an unweighted average, because {circumflex over (π)}

n

weights each observed transaction X

1

, . . . , X

n

equally.

In cases of time-dependent customer behavior, a histogram updated using an unweighted average is generally inappropriate because recent transactions have no more influence on the histogram than old transactions do. Such time-dependent behavior is tracked better by an exponentially weighted moving average (EWMA). An updated EWMA vector {circumflex over (π)}

n+1

is given by equation (1) with w

n+1

=w for a fixed weight w, 0<w<1, that controls the extent to which {circumflex over (π)}

n+1

is affected by a new transaction and the speed with which a previous transaction is “aged out.” The initial probability estimate {circumflex over (π)}

0

must be specified, and can be determined, e.g., from historical data on other customers. Under some conditions, an EWMA approximates the corresponding posterior mean under a Bayesian dynamic model, as described in M. West et al., “Bayesian Forecasting and Dynamic Models,” Springer-Verlag, 1989. Additional details regarding EWMA can be found in B. Abraham et al., “Statistical Methods for Forecasting,” John Wiley & Sons, New York, N.Y., 1983.

It is important to note that the unweighted averages and EWMAs are generally appropriate only when variables are randomly sampled. However, timing variables like day-of-week are typically not randomly sampled. As a result, standard sequential estimates of their distributions can be badly biased. For example, if the transaction rate on Monday is high and the most recent transaction occurred early Monday morning, then the next transaction is likely to occur on Monday and unlikely to occur on Tuesday or any day of the week other than Monday. Because unweighted averages and EWMAs increase the estimated probability for a histogram cell every time that cell is observed, the estimated probability for Monday first rises with every transaction made on Monday and then falls with every transaction made before the following Monday.

As is apparent from the foregoing, a need exists for techniques for updating customer signatures or other records of time-dependent behavior in the presence of timing variables that are not randomly sampled.

SUMMARY OF THE INVENTION

The present invention provides improved techniques for updating customer signatures or other types of records in a database system.

In accordance with one aspect of the invention, a customer signature or other type of record in a database system is updated using an event-driven estimator based on a model of time-dependent behavior. A current version of a record affected by a given transaction is retrieved from a memory of the system and an event-driven estimator of at least one of a transaction rate and a period probability for the record is determined based at least in part on a dynamic Poisson timing model. The dynamic model has a number of periods and corresponding period-based transaction rates associated therewith. The event-driven estimator may be configured so as to generate an estimated transaction rate {circumflex over (λ)}

j,n

for period j and a given transaction n, and then to generate an estimated period probability {circumflex over (π)}

j,n

for period j as {circumflex over (λ)}

j,n

/Σ

k

{circumflex over (λ)}

k,n

. An updated version of the record may then be generated based on the event-driven estimator.

In accordance with another aspect of the invention, a number of different techniques may be used to update the event-driven estimator. As one example, the event-driven estimator may be updated for every transaction in a specified period and at the end of each oft he periods. As another example, the event-driven estimator may be updated for every transaction, regardless of which of the periods any particular transaction falls in, but not at the end of each of the periods. As yet another example, the event-driven estimator may be updated for a given one of the periods only when there is a transaction occurring within that period.

In accordance with a further aspect of the invention, the estimated transaction rate {circumflex over (λ)}

j,n

provided by the event-driven estimator under the dynamic Poisson timing model satisfies

{\hat{λ}}_{j, n}^{- 1} = {\begin{matrix} (1 - w) {\hat{λ}}_{j, n - 1}^{- 1} + {wZ}_{j, n,} & if transaction n falls in period j \\ {\hat{λ}}_{j, n - 1}^{- 1} + \frac{w}{1 - w} Z_{j, n,} & if transaction n does not fall in period j, \end{matrix}

wherein w denotes a specified updating weight, Z

j,n

is an elapsed time in period j since a previous transaction n−1, and {circumflex over (λ)}

j,n

−1

is a reciprocal rate that estimates the average time between transactions in period j.

Advantageously, the event-driven estimator of the present invention is both computationally efficient and memory efficient. It provides better sequential estimates of timing distributions used for customer signatures or other records than the conventional unweighted average and exponentially weighted moving average (EWMA) described previously, and yet is nearly as simple to compute as an EWMA.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1

shows an exemplary database system in which record updating may be implemented in accordance with the invention.

FIG. 2

shows a distributed database system in which record updating may be implemented in accordance with the invention.

FIG. 3

is a flow diagram of an example record updating process in accordance with the invention.

FIG. 4

compares the performance of an example event-driven estimator (EDE) in accordance with an illustrative embodiment of the invention with conventional unweighted average, exponentially weighted moving average (EWMA) and maximum likelihood estimator (MLE) performance.

FIG. 5

is a diagram illustrating the structure of example data for a Poisson timing process with two periods.

FIG. 6

shows absolute relative error curves for first period probability for a conventional MLE and a number of example EDEs in accordance with the invention, for a Poisson timing model with constant rates that are fixed for all periods.

FIG. 7

is a tabular listing of sample standard deviations for the absolute relative error curves of FIG.

6

.

FIG. 8

shows absolute relative error curves for first period probability for a conventional MLE and a number of example EDEs in accordance with the invention, for a Poisson timing model with constant rates which vary across periods.

FIG. 9

shows absolute relative error curves for first period probability for a conventional MLE and a number of example EDEs in accordance with the invention, for a dynamic Poisson timing model.

FIG. 10

is a tabular listing of sample standard deviations for a portion of the absolute relative error curves of FIG.

9

.

FIG. 11

shows absolute relative error curves for first period probability for a conventional MLE and a number of example EDEs in accordance with the invention, for a Poisson timing model with deterministic shifts in the period rates.

DETAILED DESCRIPTION OF THE INVENTION

The present invention will be illustrated below in conjunction with exemplary database systems. It should be understood, however, that the invention is not limited to use with any particular type of database system or system configuration, but is instead more generally applicable to any data storage application in which it is desirable to provide efficient updating of database records involving time-dependent behavior. The term “database system” should be understood to include any type of system in which data items or other types of records are stored and processed in one or more physical locations. A “record” as the term is used herein is intended to include not only information stored in a single memory or other storage device, but also a set of related information that is stored in a distributed manner across multiple storage devices. The term “memory” as used herein is intended to encompass single storage devices as well as co-located or distributed arrangements of multiple storage devices.

Example Database Systems

FIG. 1

shows an exemplary database system

10

in which the record updating techniques of the invention may be implemented. The system

10

includes a processor

12

connected to communicate with a main memory

14

and a secondary memory

16

. The processor

12

can be implemented as a personal, micro or mainframe computer, a workstation, a microprocessor, a central processing unit, an application-specific integrated circuit (ASIC) or other digital data processor, as well as various portions or combinations of these and other devices. The main memory

14

is typically an electronic memory associated with the processor

12

. The secondary memory

16

is typically an external magnetic or optical disk-based memory, but may also be a magnetic tape memory, an electronic memory, or other suitable data storage device. The main and secondary memories of the system

10

may represent storage locations of data warehouses, data marts or other large databases having storage capacities of up to a few terabytes, smaller localized database memories, or even memories associated with desktop or portable computers, depending upon the particular application in which the invention is implemented.

The record updating techniques of the invention, which will be described in greater detail below, may be implemented at least in part as one or more software programs stored in portions of one or both of the memories

14

and

16

, and executed by processor

12

. The software programs may be configured to generate and maintain customer signatures or other records for a database or databases associated with one or both of the memories

14

and

16

. For example, such programs may be used to generate and maintain histograms for each of a number of different customers or other users associated with the system

10

. The system

10

may also utilize processor

12

and memories

14

,

16

to provide conventional database processing capabilities such as those described in, for example, H. F. Korth and A. Silberschatz, “Database System Concepts,” Third Edition, McGraw-Hill, N.Y., 1997, and R. Elmasri and S. B. Navathe, “Fundamentals of Database Systems,” Benjamin/Cummings, Redwood City, Calif., 1989, both of which are incorporated by reference herein.

FIG. 2

shows an exemplary distributed database system

20

in which the record updating techniques of the invention may be implemented. The system

20

includes N physical sites

22

-

i

, i=1, 2, . . . N connected by a network

24

. The network

24

may be a local area network, a metropolitan area network, a wide area network, a global data communications network such as the Internet, a private “intranet” network or any other suitable data communication medium. The sites

22

-

i

may be centralized database systems such as data warehouses or data marts, remote customer sites such as automatic teller machines or desktop personal computers, portable computers or other mobile computing devices, or any other type of data processors. Each of the N sites

22

-

i

includes a processor

26

-

i

and a memory

28

-

i

. The memory

28

-

i

may be, for example, a large centralized a database system, or a memory of a desktop or portable personal, depending upon the nature of the corresponding physical site

22

-

i

. One or more of the memories

28

-

i

may be configured to include both a primary memory

14

and a secondary memory

16

as shown in FIG.

1

. The processors

26

-

i

may each run a local database management system (DBMS). Like the processor

12

of

FIG. 1

, the processors

26

-

i

can be implemented as personal, micro or mainframe computers, workstations, microprocessors, central processing units, ASICs or other digital data processors, as well as various portions or combinations thereof.

The memories

28

-

i

may utilize electronic, magnetic or optical storage media, or various combinations thereof, and as previously noted may represent storage locations of data warehouses or data marts having storage capacities of up to a few terabytes, or memories of desktop or portable computers. The storage and retrieval of data items from a memory

28

-

i

of a given site

22

-

i

are controlled by the corresponding processor

26

-

i

using the local DBMS. The sites

22

-

i

of system

20

may each run multiple transactions, such as read and write transactions, during which data items stored at other sites are accessed and processed in accordance with insert, delete and modify operations. One or more oft he sites

22

-

i

of

FIG. 2

may execute a software program which generates and maintains histograms for records in a database or databases associated with one or more of the memories

28

-

i.

Event-Driven Estimator

The present invention in an illustrative embodiment provides a fast, memory-efficient sequential estimator suitable for use in updating customer signatures or other records in database systems of the type described in conjunction with

FIGS. 1 and 2

above. The sequential estimator of the illustrative embodiment is referred to herein as an event-driven estimator (EDE), since it is updated each time a transaction occurs. It is based on a dynamic Poisson model of time-dependent behavior that has periodic rates that may evolve over time. The EDE approximates the posterior mean under the dynamic Poisson timing model and has good asymptotic properties. Simulations show that it also has good finite sample properties. An application to sequentially tracking the day-of-week calling patterns of a random sample of about 2,000 telecommunications customers over a three month period will be described below and demonstrates that the dynamic Poisson model assumptions in the illustrative embodiment are adequate and that the corresponding EDE is useful in practice.

The EDE estimates the transaction rate λ

j,n

for period j at the time of a given transaction n, and then estimates the period probability {circumflex over (π)}

j,n

for period j by

{\hat{λ}}_{j, n} / \sum_{k} {\hat{λ}}_{k, n},

where {circumflex over (λ)}

j,n

denotes the estimated transaction rate. As will be described in greater detail below, the estimated transaction rate, {circumflex over (λ)}

j,n

, for λ

j,n

under the above-noted dynamic Poisson model satisfies

\begin{matrix} {\hat{λ}}_{j, n}^{- 1} = {\begin{matrix} (1 - w) {\hat{λ}}_{j, n - 1}^{- 1} + {wZ}_{j, n,} & if transaction n falls in period j \\ {\hat{λ}}_{j, n - 1}^{- 1} + \frac{w}{1 - w} Z_{j, n,} & if transaction n does not fall in period j, \end{matrix} & (2) \end{matrix}

where Z

j,n

is the time that has elapsed in a period j since the previous transaction n−1.

FIG. 3

is a flow diagram of a record updating process that may be implemented in the database systems of

FIGS. 1 and 2

in accordance with the present invention. This process involves updating a record in the form of a customer signature using the above-noted EDE, but can be readily extended to other types of records. In step

30

, a transaction occurs involving a particular customer. The current customer signature for the particular customer is retrieved from memory in step

32

. This memory may be main memory

14

of

FIG. 1

or one of the memories

28

of

FIG. 2

or any other suitable storage device. In step

34

, the transaction rate and/or period probabilities for the customer signature are estimated using the EDE based on the above-noted dynamic Poisson timing model. In step

36

, the EDE from step

34

is used to generate an updated customer signature, and the resulting updated customer signature is then stored in memory. After step

36

, the process returns to step

30

to await another transaction.

Advantageously, the EDE utilized in the updating process of

FIG. 3

provides better sequential estimates of timing distributions used for customer signatures than the conventional unweighted average and exponentially weighted moving average (EWMA) described previously, and yet is nearly as simple to compute as an EWMA.

FIG. 4

shows a set of plots comparing the performance of the EDE with that of an unweighted average, an EWMA and a maximum likelihood estimator (MLE). The Monday probability associated with a day-of-week timing variable is plotted for each of these sequential estimators as a function of transaction number using simulated transaction data. The periodic behavior of these estimators is an artifact that reflects only the order in which transactions are made. The sequential probability estimates for Monday cover a four-week period, and transactions for each day of the week are assumed to follow a Poisson (50) distribution. The nominal probability of each day, 1/7, is indicated by a dashed line in each panel of the figure.

The first panel of

FIG. 4

shows the unweighted average. The second panel of the figure shows the EWMA with w=0.02 and an initial probability of 1/7 for each weekday. Both the unweighted average and the EWMA estimated probability for Monday peaks at the end of Monday and then slowly falls during the rest of the week until the first transaction made on the following Monday. The pattern dampens for unweighted averaging over time because eventually each observation is given only a small uniform weight. For comparison, the third panel shows the MLE for the simulated Poisson model. These estimates cannot be computed without a full week of data and do not adapt as behavior evolves.

The fourth panel of

FIG. 4

shows the EDE of the illustrative embodiment of the invention, using weight w=0.02 and initial rates λ

j,0

=40, j=1, . . . , 7. Clearly, the EDE avoids the extreme periodic bias of the unweighted average and EWMA. Moreover, the EDE is very close to the exact MLE for the simulated Poisson process. As previously noted, however, the MLE is not appropriate for sequentially estimating timing distributions for a volatile database because in the

FIG. 3

example it cannot be computed until a full week after the first transaction has passed and it does not adapt to evolving behavior.

The EDE exhibits a number of advantageous characteristics of the EWMA, i.e., simplicity, ability to adapt to evolving behavior, and minimal storage requirements. Moreover, the EDE is approximately optimal under a dynamic Poisson timing model, in the sense that it approximates the posterior mean under a particular dynamic Poisson timing model. Computing the EDE is similar to computing an EWMA, except that the rate for each period, not just the rate for the observed period, is updated whenever there is a transaction. The updated estimate for the observed period is a weighted average of the previous reciprocal rate and the time since the last transaction in period j. The updated reciprocal rate estimate for any other period increases by the amount of time that has been spent in the period since the last transaction. The reciprocal rate {circumflex over (λ)}

j,n

−1

estimates the average time between transactions in period j.

Poisson Timing Model for Deterministic Transaction Rates

A Poisson timing model will now be described for the case of transaction rates that are constant within a given period or otherwise evolve deterministically over time. Suppose that calendar time is broken into a sequence of cycles, each of duration τ, and that each cycle is broken into J periods with fixed, but possibly unequal, lengths. For example, a cycle may last a week and have the J=3 periods {Monday, . . . , Friday}, {Saturday} and {Sunday}. Transaction i occurs at a calendar time T

i

during a period R

i

ε{1, . . . , J} of cycle C

i

. The time required to complete period j is δ

j

, and the time required to complete one cycle is

τ = \sum_{j = 1}^{J} δ_{j} .

Thus, a transaction at time T

i

falls in cycle C

i

=└T

i

/τ┘+1, where └x┘ denotes the greatest integer in x. It falls in period R

i

=j if Δ

j−1

<T

i

−(C

i

−1)τ≦Δ

j

, where

Δ_{j} = \sum_{m = 1}^{j} δ_{m}

is the time from the start of the cycle to the end of period j and Δ

0

=0. Let S

i

=T

i

−(C

i

−1)τ−Δ

R

i

−1

be the time spent in period R

i

up to T

i

. The probabilities of interest are the probabilities of periods 1, . . . , J during cycle k when the times of transactions in each period of each cycle follow a Poisson process.

Let N

k,j

(u) be the number of transactions up to time u within period j of cycle k; i.e., the number of transactions between calendar times (k−1)τ+Δ

j−1

and (k−1)τ+Δ

j−1

+u. Suppose that {N

k,j

(u)} is a truncated Poisson process with rate λ

k,j

. That is, if the interval [u, u+v] falls entirely in period j of cycle k, then

N

k,j

(

u+v

)−

N

k,j

(

u

)˜Poisson (λ

k,j

v

).

If N

k,j

(u) is independent of N

k,j

(v) for k≠k′ or j≠j′, then the number of transactions N

k,j

=N

k,j

(δ

j

) during period j of cycle k has a Poisson (λ

k,j

δ

j

) distribution. Also, the number of transactions

N_{k} = \sum_{j} N_{k, j}

within cycle k has a Poisson (λ

k

) distribution, where

λ_{k} = \sum_{j = 1}^{J} λ_{k, j} δ_{j} .

This is referred to as the Poisson timing model because the number of transactions N

t

up to a calendar time t that fall in period j

t

of cycle k

t

has a Poisson distribution with parameter

\sum_{k < k_{t}} λ_{k} + \sum_{j < j_{t}} λ_{k, j} δ_{j} + λ_{k_{t}, j_{t}} s

where s=t−(k

t

−1)τ−Δ

j

di t

−1

is the time spent in period j

t

. The conditional distribution of N

k,j

given N

k

is Binomial (N

k

,λ

k,j

δ/λ

k

) because

\begin{matrix} \Pr (N_{k, j} = m | N_{k} = M) = \frac{\Pr (N_{k, j} = m, N_{k} - N_{k, j} = M - m)}{\Pr (N_{k} = M)} \\ = \frac{\Pr (N_{k, j} = m) \Pr (N_{k} - N_{k, j} = M - m)}{\Pr (N_{k} = M)} \\ = (\begin{matrix} M \\ m \end{matrix}) {(\frac{λ_{k, j} δ_{j}}{λ_{k}})}^{m} {(1 - \frac{λ_{k, j} δ_{j}}{λ_{k}})}^{M - m}, \\ m = 0, \dots, M . \end{matrix}

The period probabilities π

k,1

, . . . , π

k,j

for cycle k then satisfy

\begin{matrix} π_{k, j} = \Pr (N_{k, j} = 1 | N_{k} = 1) = \frac{λ_{k, j} δ_{j}}{λ_{k}} . & (3) \end{matrix}

The model as described so far assumes that the transaction rate in each period of each cycle is an arbitrary constant λ

k,j

. It is often more reasonable to assume that the transaction rates are constant across cycles, so that λ

k,j

=λ

m,j

for all k, m, or to assume that the transaction rates evolve smoothly. For example, λ

k+1,j

=(1−α)λ

k,j

+α for a small α in (0, 1). This kind of assumption justifies a sequential updating scheme in which transactions from previous cycles affect the estimates for the current cycle. The next section extends the above-described Poisson timing model for transaction rates that evolve deterministically over time to dynamic transaction rates that evolve randomly over time.

Poisson Timing Model for Dynamic Transaction Rates

As in the previous section, let N

k,j

be the number of transactions during period j of cycle k. It is now assumed that the transaction rate is dynamic, so a different rate applies to each transaction. Let T

k,j,0

i=1, . . . , N

k,j

, be the time of the i

th

transaction that falls in period j of cycle k. Let T

k,j

=(k−1)τ+Δ

j−1

be the start of period j in cycle k and T

k,j,N

k,j

+1

=(k−1)τ+Δ

j

be the end of period j in cycle k.

FIG. 5

shows the structure of the data for an example of the above-described dynamic Poisson timing process with two periods. Note that T

i,j,k

is either the start of a period, the time of a transaction, or the time from the last transaction in the period to the end of the period.

The waiting times Y

k,j,i

=T

k,j,i

−T

k,j,i−1

i=1, . . . , N

k,j

, between transactions in the same period of the same cycle are uncensored. At time T

k,j,N

k,j

+1

, the waiting time Y

k,j,N

k,j

+1

from the last transaction in period j until the end of period j can be considered to be a censored waiting time because the period could no longer be observed. Let Z

k,j,N

k,j

+1

be the uncensored waiting time for the end of period j in cycle k, that is, the transaction time that would have been observed if the period had continued to be observed. Including Y

k,j,N

k,j

+1

in the model allows the information that no transaction was seen in a period to be used to update the rate for that period.

Under a Poisson timing model, Y

k,j,1

. . . , Y

k,j,N

k,j

are independent exponential random variables and Y

k,j,N

k,j

+1

is a right-censored exponential. Under a dynamic Poisson timing model, the mean of the exponential distribution evolves randomly. Here it is assumed that the mean in effect at time T

k,j,i

is a random perturbation of the rate in effect at the time of the last transaction in period j prior to T

k,j,i

. Thus, the transaction rate changes when and only when there is a transaction. The dynamic Poisson timing model is more precisely defined as follows:

Y

k,j,i

˜exponential(λ

k,j,i

),

i

=1

, . . . , N

k,j

Y

k,j,N

j

+1

=min(

Z

k,j,N

k,j

+1

(

k

+1)τ+Δ

j

−T

k,j,N

k,j

),

Z

k,j,N

k,j

+1

˜exponential(λ

k,j,N

k,j

+1

)

\begin{matrix} λ_{k, j, i} = {\begin{matrix} ε_{k, j, i} λ_{k, j, i - 1,} & i = 2, \dots, N_{k, j} + 1, N_{k, j} \geq 1, & or i = 1, k = 1 \\ λ_{k, j, i - 1,} & i = 1, k > 1 \end{matrix} & (4) \end{matrix}

where λ

k,j,0

=λ

k−1,j,N

k−1,j

+1

and λ

1,j,0

is a known constant. Note that the transaction rate for a period stays constant until a transaction occurs in that period, and then the rate for the next period (which controls the waiting time until the next transaction) is generated. Thus, the transaction rate that applies to the first transaction in a period j after time 0 is generated at time 0. Dynamic model (4) is an example of a type of Bayesian dynamic model. Conventional Bayesian dynamic models are described in M. West et al., “Bayesian Forecasting and Dynamic Models,” Springer-Verlag, 1989.

Because the mean of ε

k,j,i

one and its variance is 1/α, the transaction rate is more stable for larger α. The current transaction rate is also more variable when the previous transaction rate was high than when it was low. To see this, let C

k,j,(i)

denote the cycle of the last transaction in period j prior to time T

k,j,i

, that is,

C_{k, j, (i)} = {\begin{matrix} \max {l < k : N_{l, j} \geq 1}, & N_{k, j} (T_{k, j, i} - T_{k, j, 0}) \leq 1 \\ k, & N_{k, j} (T_{k, j, i} - T_{k, j, 0}) \geq 2, \end{matrix}

where N

k,j

(T

k,j,i

−T

k,j,0

) represents the number of transactions in period j of cycle k up to time T

k,j,i

. By convention, C

k,j,(i)

=0 if no transactions occurred in period j prior to T

k,j,i

. Let λ

k,j,(i)

denote the rate in effect at the time of the last transaction in period j of cycle C

k,j,(i)

. It follows from dynamic model (4) that (λ

k,j,k

|λ

k,j,(i)

)˜Γ(α,α/λ

k,j,(i)

), which has variance λ

2

k,j,(i)

/α.

The posterior distribution of the current transaction rate can be found as follows. Let W

k,j,i

denote the time elapsed in period j between the last transaction in period j and time T

k,j,i,−1

, so

W_{k, j, i} = {\begin{matrix} \sum_{l = C_{k, j, (i)}}^{k - 1} Y_{l, j, N_{l, j} + 1}, & C_{k, j, (i)} < k \\ 0, & C_{k, j, (i)} = k, \end{matrix}

with the convention that Y

k,j,1

=0. For 1<i≦N

k,j

, the posterior distribution of λ

k,j,i

given the previous transaction rate λ

k,j,(i)

and all the observed data Y

k,j,1

={Y

1,j,1)

, . . . , Y

k,j,i

} up to and including time T

k,j,i

is Γ(α+1, α/λ

k,j,(i)

+W

k,j,i

+Y

k,j,i

) When i=N

k,j

+1, the time Y

k,j,i

after the last transaction in a period until the end of the period is a censored exponential and the posterior distribution of λ

k,j,i

given λ

k,j,(i)

and the Y

k,j,i

is Γ(α, α/λ

k,j,(i)

+W

k,j,i

+Y

k,j,i

). Thus,

E (λ_{k, j, i} | λ_{k, j, (i)}, Y_{k, j, i}) = {\begin{matrix} {[(1 - w) λ_{k, j, (i)}^{- 1} + w (W_{k, j, i} + Y_{k, j, i})]}^{- 1}, & 1 \leq i \leq N_{k, j} \\ {[λ_{k, j, (i)}^{- 1} + \frac{w}{1 - w} (W_{k, j, i} + Y_{k, j, i})]}^{- 1}, & i = N_{k, j} + 1, \end{matrix}

where w=(α+1)

−1

. This posterior mean is not directly useful for the sequential estimation problem addressed in the illustrative embodiment because it depends on the unknown period rate λ

k,j,(i)

and all the past observations. It does, however, resemble the EDE given by equation (2) above.

Derivation of the EDE

Taylor expansions of an approximation to the posterior mean of a transaction rate λ

k,j,i

given the initial rate λ

j,0

and Y

k,j,i

can be used to derive an iterative procedure for updating λ

k,j,i

. Namely, take

{\hat{λ}}_{k, j, i}^{- 1} = {\begin{matrix} (1 - w) {\hat{λ}}_{k, j, (i)}^{- 1} + w (W_{k, j, i} + Y_{k, j, i}), & k \geq 1, 1 \leq i \leq N_{k, j} \\ {\hat{λ}}_{k, j, (i)}^{- 1} + \frac{w}{1 - w} (W_{k, j, i} + Y_{k, j, i}), & k \geq 1, i = N_{k, j} + 1, \end{matrix}

where {circumflex over (λ)}

1,j,1

=E(λ

1,j,0

|λ

1,j,1

). For k>1 or i>1, {circumflex over (λ)}

k,j,i

approximates the posterior mean E(λ

k,j,i

|λ

j,0

, Y

k,j,i

). Because the transaction rates {circumflex over (λ)}

k,j,i

are updated at the end of each period j or at each new transaction in period j, the previous equations can be rewritten as

\begin{matrix} {\hat{λ}}_{k, j, i}^{- 1} = {\begin{matrix} (1 - w) {\hat{λ}}_{k, j, i - 1}^{- 1} + {wY}_{k, j, i}, & k \geq 1, 1 \leq i \leq N_{k, j} \\ {\hat{λ}}_{k, j, i - 1}^{- 1} + \frac{w}{1 - w} Y_{k, j, i}, & k \geq 1, i = N_{k, j} + 1, \end{matrix} & (5) \end{matrix}

which uses only the previous estimate and the current transaction to update the estimators. The iterative estimators for the period probabilities are then

{\hat{π}}_{k, j, i} = {\hat{λ}}_{k, j, i} δ_{j} / (\sum_{m = 1}^{J} {\hat{λ}}_{k, m, i} δ_{m}) .

The weight w in the iterative estimate may be selected as proportional to the variance of the multiplicative noise factor in the dynamic model (4): w=α(α+1)

−1

var(ε

k,j,i

). If var(ε

k,j,i

) is large, then λ

k,j,i

evolves quickly and more weight must be given to the current observation Y

k,j,i

, which leads to a noisier sequence of estimates {{circumflex over (λ)}

k,j,i

}. Conversely, if (ε

k,j,i

) is small, then λ

k,j,i

evolves slowly and more weight should be given to previous waiting times, giving more stable {{circumflex over (λ)}

k,j,i

}. The simulations to be described in detail below, however, suggest that larger w, on the order of 1/{square root over (α)}, may provide better performance.

The iterative estimate (5) is updated whenever there is a transaction in period j or a period j ends. This is referred to as full iterative estimation because each transaction rate is updated as soon as it changes or as soon as a period to which it applies ends. Thus, full iterative estimation requires updating the signature for each customer at the end of every period. Retrieving all signatures at the end of every period can be impractical, however. It will be shown below that the EDE, which is updated only when there is a transaction, is equivalent to the full iterative estimator (5) whenever there is a transaction. Although the EDE is out-of-date whenever no transaction occurs in a period, this limited staleness is not of major significance in many practical applications.

Now suppose the transaction rates can be updated only at the time of a transaction, and whenever there is a transaction the estimated transaction rates for all periods since the last transaction are updated. For example, suppose that J=3 and the current transaction falls in period

2

of cycle

10

. If the previous transaction fell in period

2

of cycle

8

, then λ

9,3,1

and λ

10,1,1

as well as λ

10,2,1

would be updated at time T

10,3,1

. Note that missed periods are “found” only at the first transaction time in a period. Because updating occurs only at transaction times, the subscripts are by transaction number rather than by cycle and period. The transaction times are denoted by T

1

, . . . , T

n

, their periods by R

1

, . . . , R

n

, the time from the start of period R

n

to T

n

is S

n

, and the transaction rates in effect at transaction n are λ

1,n

, . . . , λ

j,n

.

Let M

j,n

≧0 be the number of missed periods of type j between T

n−1

and T

n

, i.e., the number of cycles during which the transaction rate for period j was not updated. In the example above, M

3,n

=M

1,n

=2 and M

2,n

=1. Let Z

j,n

be the waiting time for period j at transaction n, so that

Z_{j, n} = {\begin{matrix} T_{n} - T_{n - 1} & R_{n} = R_{n - 1} = j and C_{n} = C_{n - 1} \\ M_{j, n} δ_{j} + S_{n} + δ_{j} - S_{n - 1}, & R_{n} = R_{n - 1} = j and C_{n} > C_{n - 1} \\ M_{j, n} δ_{j} + δ_{j} - S_{n - 1}, & R_{n - 1} = j, R_{n} \neq j \\ M_{j, n} δ_{j} + S_{n}, & R_{n - 1} \neq j, R_{n} = j \\ M_{j, n} δ_{j} & R_{n - 1} \neq j, R_{n} \neq j . \end{matrix}

Finally, define the updated EDE for period j at transaction n by

\begin{matrix} {\hat{λ}}_{j, n}^{- 1} = {\begin{matrix} (1 - w) {\hat{λ}}_{j, n - 1}^{- 1} + {wZ}_{j, n,} & R_{n} = j \\ {\hat{λ}}_{j, n - 1}^{- 1} + \frac{w}{1 - w} Z_{j, n,} & R_{n} \neq j \end{matrix} & (6) \end{matrix}

for j=1, . . . , J.

With equation (6), all estimated transaction rates are brought up-to-date at each transaction time, but some estimated rates may be out-of-date between transactions because inactive periods have not yet been accounted for. The reciprocal of the updated rate for period j is a weighted average of the previous estimated reciprocal rate for period j and the time Z

n,j

spent in period j since the last transaction when the current transaction falls in period j. If the current transaction does not fall in period j, then the updated reciprocal rate for period j is not a weighted average. Instead, the previous estimate is increased by a term that is proportional to the time in period j with no activity.

Because all transaction rates are brought up-to-date at the time of a transaction, regardless of which period the transaction falls in, the EDE of the period probability for period j is

\begin{matrix} {\hat{π}}_{j, n} = \frac{{\hat{λ}}_{j, n} δ_{j}}{\sum_{i = 1}^{J} {\hat{λ}}_{i, n} δ_{i}} . & (7) \end{matrix}

A case-by-case analysis shows that at time T

n

the JEDEs from equation (6) are identical to the J iterative estimates from equation (5). For example, suppose that R

n

=j, R

n−1

=i≠j and M

j,n

>0. Then the transaction at time T

n

is the first in period j of cycle C

n

. With the full iterative estimator, the estimated reciprocal transaction rate for period j would have been updated by a censored exponential of length δ

j

at the end of each of the previous M

j,n

intervals of type j. Therefore, just before time T

n

, the full iterative estimator would be the reciprocal of

{\hat{λ}}_{j, n - 1}^{- 1} + \frac{w}{1 - w} M_{j, n} δ_{j} .

At time T

n

, this estimator would be updated to

(1 - w) ({\hat{λ}}_{j, n - 1}^{- 1} + \frac{w}{1 - w} M_{j, n} δ_{j}) + {wS}_{n} = (1 - w) {\hat{λ}}_{j, n - 1}^{- 1} + {wZ}_{j, n},

which is identical to the reciprocal of the EDE after transaction n.

The EDE (6) is computed from the current estimated rate {circumflex over (λ)}

j,n

and the time T

n−1

of the last transaction. This is almost as memory-efficient as the conventional EWMA estimate (1), which requires storing only the first J−1 of the {circumflex over (π)}

j

's.

An Application: Estimating Day-of-Week Calling Patterns for Fraud Detection

Simply stated, fraud detection is the discrimination of legitimate transactions from fraudulent ones. Because customers are often extremely diverse, the discrimination generally must be tailored to each customer separately. An important step in fraud detection, then, is tracking the legitimate behavior of each customer in real-time, so that there is an accurate basis of comparison for discriminating fraud. The section describes an example application of the present invention which involves tracking day-of-week calling patterns for a random sample of about 2,000 callers who made between 50 and 800 completed calls during peak hours over a three month period. Peak hours are defined in this example to be 9:00 a.m. to 11:00 a.m. and 1:00 p.m. to 4:00 p.m. Monday through Friday. Not all customers made calls for the entire three months; the time between a customer's first and last calls ranged from eight to thirteen weeks.

The above-described dynamic Poisson model holds for a customer if the times Y

k,j,i

between calls i and i−1 during peak hours on day-of-week j in week k behave as independent exponential(λ

k,j,i

) random variables. The day-of-week calling patterns for non-peak calls may be different. To avoid the complications of nesting hours-within-days, the non-peak calls are ignored in this example. The calling rates λ

k,j,i

for the customer are unknown and might change throughout the three months, but this example will initially be used to test the simple Poisson model for customer transaction rates that are approximately constant over the three month period, so Y

k,j,i

˜exponential (λ

j

).

A chi-squared goodness-of-fit test for the exponential distribution can be applied to the waiting times for each day of week for each customer separately, giving a total of 10,000 goodness-of-fit p-values. This type of test is described in greater detail in W. J. Conover, “Practical Nonparametrix Statistics,” John Wiley & Sons, New York, 1980. Under the null hypothesis that the Poisson timing model is adequate, the sep-values are uniformly distributed in the interval (0,1). If the simple Poisson timing model with constant transaction rates is adequate for this set of customers, then the points when plotted should lie on a straight line. Taking an intercept from a regression of theoretical U(0,1) quantiles on the empirical quantiles as an estimate oft he fraction of customers for whom the constant rate Poisson model is inadequate shows that the simple model does not fit for only about 5.5% to 7.5% of customers across weekdays. This degree of lack-of-fit may be acceptable in the present example and in many other practical applications.

Even if the simple Poisson model with constant rates does not fit, the EDEs of the day-of-week probabilities may still be adequate. To check if that is the case here, the empirical distribution π of a customer's calls per weekday may be taken over the three month period as the underlying day-of-week distribution for the customer. The average absolute relative error of the EDE {circumflex over (π)}

n

with respect to π is then computed as a function of call number by taking

ε_{n} = 100 \sum_{j = 1}^{5} \frac{&LeftBracketingBar; π_{j} - {\hat{π}}_{j, n} &RightBracketingBar;}{5 π_{j}} .

Percentiles of the average error at call n over all customers that make at least n calls then describe the performance of the EDE. It has been determined in the present example that, considering the entire sample of customers as a whole, the EDE clearly outperforms the EWMA estimate. After about 100 calls, the median EDE average absolute relative error is smaller than the 0.25 quantile of the EWMA average absolute relative error; after 300 calls, the 0.75 EDE quantile is smaller than the 0.25 EWMA quantile. The median average absolute relative error stabilizes around 5% for the EDE and around 15% for the EWMA. As expected, the average absolute relative errors for the subset of customers for which the constant rate Poisson model is inadequate tend to be larger than those for all 2,000 customers considered as one group, but the EDE still clearly dominates the EWMA. The median average absolute relative error curve oscillates around 8% for the EDE and around 22% for the EWMA. In this application, then, the EDE is relatively robust to departures from the constant rate Poisson timing model.

Approximate Behavior of the EDE

The approximate behavior of the EDE will now be described in greater detail. The approximate conditional moment will be described first, followed by the approximate conditional distribution under a Poisson model with constant transaction rates.

Equation (6) is convenient for updating the rate estimates, but not for establishing their properties. For that, it is more convenient to rewrite the EDE as a sum of terms that are defined at the transactions that fall in period j, plus an additional term that reflects the inactive time spent in period j, if the current period is unequal to j. To simplify the notation, subscript j is suppressed in this section.

Let N

+,i

denote the number of transactions that fall in period j up to and including time T

i

and N

−,i

denote the number of transactions in period j up to but not including time T

i

, so

N_{+, i} = \sum_{m = 1}^{i} I (R_{m} = j) and N_{-, i} = N_{+, i} - I (R_{i} = j),

where I(A)=1 if condition A is true and 0 otherwise. Let L

i

be the index of the i

th

transaction in period j, so L

i

=min {m:N

+,m

=i}, with the convention that L

0

=0. Note that L

i

is only defined for i=0, . . . , N

+,n

at transaction n. Finally, let W

+,i

, be the total time spent in period j between time T

i

and the previous transaction in a period j (or since time 0, if there are no previous transactions in period j). Then

W_{+, i} = \sum_{m = L_{N_{-, j}} + 1}^{i} Z_{j, m} .

In particular, W

+,L

1

, . . . , W

+,L

N+,n

are the waiting times between transactions that fall in period j up to time T

n

.

Equation (6) shows that when the current transaction does not fall in period j, the reciprocal of the EDE is increased by a constant multiple of the inactive time since the last transaction. Averaging occurs only when there is a transaction in period j. It is thus possible to accumulate the inactive time and update the transaction rate for period j only at the time of a transaction in period j, Thus, the EDE can be rewritten as

{\hat{λ}}_{j, n}^{- 1} = {\begin{matrix} (1 - w) {\hat{λ}}_{j, L_{N_{-, n}}}^{- 1} + {wW}_{+, n,} & R_{n} = j \\ {\hat{λ}}_{j, L_{N_{-, n}}}^{- 1} + \frac{w}{1 - w} W_{+, n,} & R_{n} \neq j . \end{matrix}

Since {circumflex over (λ)}

j,L

i

−1

=(1

−W

){circumflex over (λ)}

j,L

i−1

+wW

+,L

i

(with {circumflex over (λ)}

j,L

0

=λ

j,0

), it follows that

\begin{matrix} {\hat{λ}}_{j, n}^{- 1} = {\begin{matrix} \sum_{i = 1}^{N_{+, n}} {(1 - w)}^{i - 1} {wW}_{+, L_{i}} + {(1 - w)}^{N_{-, n}} λ_{j, 0}^{- 1}, & R_{n} = j \\ \sum_{i = 1}^{N_{+, n}} {(1 - w)}^{i - 1} {wW}_{+, L_{i}} + {(1 - w)}^{N_{+, n}} λ_{j, 0}^{- 1} + \frac{w}{1 - w} W_{+, n,} & R_{n} \neq j . \end{matrix} & (8) \end{matrix}

To summarize, the rate estimates obtained by (i) updating at every transaction in period j and at the end of every period j, or (ii) updating at the time of each transaction, regardless of which period it falls in, but not at the end of each period j, or (iii) updating only when there is a transaction in period j are identical at the time of a transaction in period j. Updating scheme (ii) is most useful for computing, but (iii) is most useful for establishing properties of the rate estimator.

Approximations to the moments and the distribution of {circumflex over (λ)}

j,n

will now be described. The main interest here is to understand the forecasting properties of {circumflex over (λ)}

j,n

given a sequence of transaction rates λ

j,n

={λ

j,1

, . . . , λ

j,N

+,n

} and N

+,n

. To simplify the notation, the dependence on the conditioning variables is suppressed, but all moments and densities should be understood as conditional on λ

j,n

and N

+,n

. For example, E({circumflex over (λ)}

j,n

−1

) denotes E({circumflex over (λ)}

j,n

−1

|λ

j,n

, N

+,n

).

Given λ

j,n

, it can be shown that the waiting times W

+,L

i

, i=1, . . . , N

+,n

, between transactions in period j are independent exponential (λ

j,L

i

) random variables under the dynamic Poisson timing model. The only other waiting time that needs to be considered is W

+,n

when R

n

≠j, where

W_{+, n} = δ_{j} - S_{L_{N_{+, n}}} + δ_{j} \sum_{m = L_{N_{+, n}} + 1}^{n} M_{j, m} .

Conditional λ

j,n

and N

+,n

, the last sum has a geometric (1−exp(−λ

j,L

N+,n+1

δ

j

)) distribution and is independent of S

L

N+,n

. If the transaction rate for period j changes so slowly that it is approximately constant within cycles, then δ

j

−S

L

N+,n

is approximately distributed as a truncated exponential (λ

j,L

N+,n+1

) in the interval [0, δ

j

]. Combining these results implies that W

+,n

given λ

j,n

and N

+,n

is approximately (λ

j,L

N+,n+1

) when R

n

≠j.

When R

n

=j, it follows from (8) that, given λ

j,n

and N

+,n

,

\begin{matrix} E ({\hat{λ}}_{j, n}^{- 1}) = \sum_{i = 1}^{N_{+, n}} {(1 - w)}^{i - 1} w λ_{j, L_{i}}^{- 1} + {(1 - w)}^{N_{+, n}} λ_{j, 0}^{- 1} var ({\hat{λ}}_{j, n}^{- 1}) = \sum_{i = 1}^{N_{+, n}} {(1 - w)}^{2 (i - 1)} w^{2} λ_{j, L_{i}}^{- 2} . & (9) \end{matrix}

Similarly, when R

n

≠j,

\begin{matrix} E ({\hat{λ}}_{j, n}^{- 1}) ≅ \sum_{i = 1}^{N_{+, n}} {(1 - w)}^{i - 1} w λ_{j, L,}^{- 1} + {(1 - w)}^{N_{+, n}} λ_{j, 0}^{- 1} + \frac{w}{1 - w} λ_{J, {L_{i}}_{N_{+, n} - 1}}^{- 1} var ({\hat{λ}}_{j, n}^{- 1}) ≅ \sum_{i = 1}^{N_{+, n}} {(1 - w)}^{2 (i - 1)} w^{2} λ_{j, L_{i}}^{- 2} + {(\frac{w}{1 - w})}^{2} λ_{J, N_{+, n} + 1.}^{- 2} & (10) \end{matrix}

If λ

j,n

=λ

j

for all n, then for R

n

=j

E

({circumflex over (λ)}

j,n

−1

)=[1−(1

−w

)

N

+,n

]λ

j

−1

+(1

−w

)

N

+,n

λ

j,0

−1

N

+,n

→

→∞

λ

j

−1

(11)

var ({\hat{λ}}_{j, n}^{- 1}) = \frac{w}{2 - w} [1 - {(1 - w)}^{2 N_{+, n}}] λ_{j}^{- 2} \overset{N_{+, n} ⟶ \infty}{\to} \frac{w}{2 - w} λ_{j}^{- 2} .

When R

n

≠j, the approximations in (10),become exact and simplify to

\begin{matrix} \begin{matrix} E ({\hat{λ}}_{j, n}^{- 1}) = \frac{1}{1 - w} [1 - {(1 - w)}^{N_{+, n} + 1}] λ_{j}^{- 1} + {(1 - w)}^{N_{+, n}} {λ_{j, 0}^{- 1}}^{\underset{⟶}{N_{+, n} ⟶ \infty}} \frac{1}{1 - w} λ_{j}^{- 1} \\ var ({\hat{λ}}_{j, n}^{- 1}) = &AutoLeftMatch; \frac{w}{(2 - w) {(1 - w)}^{2}} [1 - {(1 - w)}^{2 (N_{-, n} + 1)}] λ_{j}^{- 2} \overset{N_{+, n} ⟶ \infty}{\to} [\frac{w}{(2 - w) {(1 - w)}^{2}}] λ_{j}^{- 2} . \end{matrix} & (12) \end{matrix}

The approximate conditional distribution under a Poisson model with constant rates will now be described. If the transaction rate for period j is constant across cycles, then, given λ

j,n

and N

+,n

, the EDE of a reciprocal rate is a linear combination of independent, identically distributed exponential random variables, plus an exponentially decreasing term that depends on λ

j,0

. If all coefficients in the linear combination were the same, the linear combination would be conditionally distributed as a gamma random variable. The conditional distribution of {circumflex over (λ)}

j,n

−1

can therefore be approximated by an exponentially decreasing constant plus a gamma distribution with parameters that match the first two moments of {circumflex over (λ)}

j,n

−1

. That is,

{\hat{λ}}_{j, n}^{- 1} \approx Γ (\frac{{[E ({\hat{λ}}_{j, n}^{- 1}) - {(1 - w)}^{N_{+, n}} λ_{j, 0}^{- 1}]}^{2}}{var ({\hat{λ}}_{j, n}^{- 1})}, \frac{[E ({\hat{λ}}_{j, n}^{- 1}) - {(1 - w)}^{N_{+, n}} λ_{j, 0}^{- 1}]}{var ({\hat{λ}}_{j, n}^{- 1})}) + {(1 - w)}^{N_{+, n}} λ_{j, 0}^{- 1} .

From equations (11) and (12),

\begin{matrix} {\hat{λ}}_{j, n}^{- 1} \approx {(1 - w)}^{N_{+, n}} λ_{j, 0}^{- 1} + {\begin{matrix} Γ (\frac{{\frac{2 - w}{w} [1 - {(1 - w)}^{N_{+, n}}]}^{2}}{1 - {(1 - w)}^{2 N_{+, n}}}, \frac{2 - w}{w} \frac{1 - {(1 - w)}^{N_{+, n}}}{1 - {(1 - w)}^{2 N_{+, n}}} λ_{j}), & R_{n} = j \\ Γ (\frac{{\frac{2 - w}{w} [1 - {(1 - w)}^{N_{+, n} + 1}]}^{2}}{1 - {(1 - w)}^{2 (N_{+, n} + 1)}}, \frac{(2 - w) (1 - w)}{w} \frac{1 - {(1 - w)}^{N_{+, n} + 1}}{1 - {(1 - w)}^{2 (N_{+, n} + 1)}} λ_{j}), & R_{n} &NotHumpEqual; j \end{matrix} & (13) \end{matrix}

Letting N

+,n

→∞, the right side of (13) asymptotically simplifies to

\begin{matrix} {\hat{λ}}_{j, n}^{- 1} \approx {\begin{matrix} Γ (\frac{2 - w}{w}, \frac{2 - w}{w} λ_{j}), & R_{n} = j \\ Γ (\frac{2 - w}{w}, \frac{(2 - w) (1 - w)}{w} λ_{j}), & R_{n} \neq j . \end{matrix} & (14) \end{matrix}

Simulations indicate that the approximation (13) provides good performance for a wide range of w, even for small N

+,n

. It is sufficient to simulate the behavior of {circumflex over (λ)}

j,n

at λ

j

under the Poisson timing model because, with constant rates, the distribution of

λ_{j} \sum_{i = 1}^{N_{+, n}} {(1 - w)}^{i - 1} w W_{+, L_{i}}

in equation (8) is free of λ

j

and the contribution of the initial rate λ

j,0

decreases exponentially. Using the statistical language S described in R. A. Becker et al., “The New S Language: A Programming Environment for Data Analysis and Graphics,” Wadsworth, Belmont, Calif., 1988, 2500 simulated values of {circumflex over (λ)}

j,n

−1

−(1−w)

N

+,n

λ

j,0

−1

were generated, with w=0.02, N

+,n

=10, for both R

n

=j and R

n

≠j, and the empirical distribution of the 2500 values of λ

j

[{circumflex over (λ)}

j,n

−1

−(1−w)

N

+,n

λ

j,0

−1

] were compared with the gamma approximation. The comparison indicated that the empirical cumulative distribution function (CDF) is close to the approximate gamma CDF. Similar results were obtained for other values of w and N

+,n

.

The approximate distribution of λ

j

[{circumflex over (α)}

J,n

−1

−(1

−w)

N

+,n

λ

j,0

−1

] can be used as a pivot to construct approximate conditional prediction intervals for the transaction rates. Let γ

j,n

(α,w) denote the 100α

th

percentile of the gamma approximation (13) for {circumflex over (λ)}

j,n

−1

with λ

j

=1. Then an approximate conditional prediction interval of level 1−α for λ

p

based on {circumflex over (λ)}

j,n

−1

is

[\frac{γ_{j, n} (α / 2, w)}{{\hat{λ}}_{j, n}^{- 1} - {(1 - w)}^{N_{+, n}} λ_{j, 0}^{- 1}}, \frac{γ_{j, n} (1 - α / 2, w)}{{\hat{λ}}_{j, n}^{- 1} - {(1 - w)}^{N_{+, n}} λ_{j, 0}^{- 1}}] .

For large N

+,n

, similar intervals can be obtained using (14).

Approximate conditional moments for the estimated period probabilities {circumflex over (π)}

j,n

can be obtained from equations (9) and (10) (or equations (11) and (12)) by applying the delta method to equation (7). The delta method is described in B. Efron et al., “An Introduction to the Bootstrap,” Chapman & Hall, New York, 1993. When transaction rates are constant across cycles, better approximate prediction intervals for period probabilities can be obtained from a parametric bootstrap using the approximate distributions (13) or (14) and the independence of the transaction rate estimators.

In practice, transaction rates tend to vary slowly with time, reflecting changes in the environment and the individual, so the assumption of constant transaction rates across cycles may be reasonable for only a limited number of cycles. The approximations continue to hold locally, but N

+,n

needs to be redefined as the number of transactions during period j for the past max (0, C

n

−k) where k is the number of cycles over which λj,n is nearly constant.

Finite Sample Performance of the EDE

Simulations were used to compare the performance of the above-described EDE having example updating weights of w=0.02, 0.05, and 0.1 with the performance of the conventional MLE for different Poisson timing models. In each simulation, there are 12 cycles (or weeks) of 7 periods (or days), and each period has the same transaction rate. Thus, the period probabilities are all 1/7. In the following description, a simulation based on the Poisson timing model with constant period is described first, followed by a simulation based on the dynamic Poisson timing model. Finally, a simulation based on a Poisson timing model with a deterministic shift in the period rates at the end of the 12th cycle is described. In the latter simulation, at the end of the 12th cycle, one period rate is increased by 25%, another decreased by 25%, and the rates for the other periods are unchanged. All period rates then remain constant for another 12 cycles.

The performance of the estimators changes with time, as more transactions are made and, under the dynamic model or shift model, as the underlying transaction rates change. If θ(t) is a parameter of interest (a rate or a period probability) at time t and {circumflex over (θ)}(t) its estimate, then the absolute relative error at time t may be defined as

ε[{circumflex over (θ)}(

t

)]=100

E

(|{circumflex over (θ)}(

t

)−θ(

t

)|)/θ(

t

).

For each of the simulations noted above, 10,000 paths of the transaction process were simulated under the corresponding Poisson timing model, each path covering 12 cycles. For each simulated path, period rates and probabilities were estimated at each transaction and the corresponding absolute relative errors were evaluated on a fixed, equally spaced grid of 100 time points, using linear interpolation. This allows the absolute relative error curves from different transaction paths to be combined. Additional information presented includes the mean of the 10,000 absolute relative error curves, and the standard deviations of the mean absolute relative error curves. The latter provides an assessment of the precision in the simulation results.

For the Poisson model with constant rates, four scenarios, with identical transaction rates λ fixed at constant values of 1, 5, 10, or 20 for each of seven periods, giving period probabilities of 1/7 for each period, were simulated. Estimated transaction rates for each period were initialized for each of the seven periods independently by randomly selecting a value in the interval [0.75λ, 1.25λ] according to a uniform distribution. This initialization procedure implies that, on average, the transaction rates are initialized at the correct value, but the initial rate for any period can be off by as much as 25%. In practice, initial transaction rates are generally derived from past customer data, so initial rates averaged over customers tend to be unbiased. Therefore, the simulated initialization procedure is realistic.

FIG. 6

shows the absolute relative error curves for the MLE and the EDE, and

FIG. 7

lists the corresponding standard deviations, for the simulation of the constant rate Poisson timing model. Each panel of

FIG. 6

corresponds to a different base transaction rate. The standard deviations values listed in

FIG. 7

are average values determined over the above-noted time grid, with the associated ranges given in parentheses following the average values.

The results shown in

FIGS. 6 and 7

for the simulation for the constant rate Poisson timing model indicate the following:

1. The simulated absolute relative error of the MLE for the constant rate Poisson timing model decreases as the number of transactions increases.

2. The smallest updating weight, w=0.02 in this example, gives the best EDE under the constant rate model. During the first 12 cycles, the EDE with w=0.02 is better than the MLE for λ=1 and λ=5; better or equivalent to the MLE for λ=10; and worse than the MLE for λ=20, after 4 cycles.

3. The EDE with any of the three weights is better than the MLE during the initial cycles, even though the initial value assigned to λ is only correct on average. In contrast, the MLEs for the period probabilities are undefined until the first transaction in the last period of the first cycle is observed or the first cycle ends.

4. The absolute relative error rates for the EDE appear to stabilize at a value that does not depend on λ.

These conclusions remain valid when the transaction rates for each period are constant across cycles, but are different among periods.

FIG. 8

shows the simulated absolute relative error curves for the estimated probability of period 1 under the same scenarios used to produce the results in

FIG. 6

, but with the period transaction rates λ

1

, . . . , λ

7

set to λ

1

=0.8λ, λ

2

=λ, λ

3

=1.2λ, λ

4

=0.5λ, λ

5

=3λ, λ

6

=2λ, and λ

7

=0.5λ. As before, the base transaction rates are λ=1, 5, 10, and 20, and each panel of the figure corresponds to a different base transaction rate. The absolute relative error curves shown in

FIG. 8

are nearly identical to those in

FIG. 6

, and the corresponding simulation standard deviations (not shown) are nearly identical to those listed in FIG.

7

.

For the dynamic Poisson timing model, four scenarios with initial transaction rates of λ

0

=1, 5, 10, and 20 transactions per period are considered. The EDE is initialized as in the constant rate model simulation previously described. The multiplicative noise term responsible for the evolution of the transaction rates in the dynamic Poisson model is simulated from a Γ(α, α) distribution with large α (α≧400), which means that the transaction rates evolve slowly. For large α, Γ(α, α)≅N(1, 1/α), so the endpoints of a 99% prediction interval for the relative change are approximately ±2.58/{square root over (α)}. The endpoints of the approximate 99% prediction intervals for the values of α used in the simulation corresponding to high (α=400), moderate (α=1600), and low (α=4000) noise levels are ±0.129, ±0.064, and ±0.041, respectively.

FIG. 9

shows the absolute relative error rate curves for the first period (j=1) probability for MLE and the EDE with weights w=0.02, 0.05, and 0.1 over 12 cycles of the dynamic Poisson timing model, for each combination of the example λ

0

and α values. Each panel of the figure corresponds to a different initial transaction rate λ

0

, which is the same for all periods, and a different noise parameter α.

FIG. 10

lists the standard deviations for the absolute relative error curves with α=400, corresponding to the highest relative change in the transaction rates. Again, the standard deviation values are averaged over the time grid, with the ranges of the standard deviations over the time grid included in parentheses. The standard deviations for the absolute relative error curves corresponding to α=1600 and α=4000 fall between the values shown in FIG.

7

and those shown in FIG.

10

. The results shown in

FIGS. 9 and 10

for the simulation for the dynamic Poisson timing model indicate the following:

1. When α=400 (high noise), the EDE performs better than the MLE. The absolute relative errors for the EDE stabilize around 20%, independent of λ

0

, but the absolute relative error for the MLE appears to increase with time. The EDE with w=0.05 is best, but the other values of w are nearly as good.

2. When α=1600 (moderate noise), the EDE with w=0.02 and the EDE with w=0.05 have similar absolute relative error curves, but the other estimators have larger absolute relative errors. The absolute relative error for the MLE increases over time for λ

0

=20.

3. When α=4000 (low noise), the EDE with w=0.02 has the best absolute relative error curve. The MLE has a similar absolute relative error curve (flat around 10%) for λ

0

=10 and λ

0

=20.

The simulation results for the Poisson model with deterministic shifts in rates will now be described. Under this scenario, the period rates for the Poisson timing model are identical and constant for the first 12 cycles. As previously noted, at the end of the 12th cycle, the first period rate is increased by 25% and the rate for the fourth period is decreased by 25%, so the cycle rate remains the same. The period rates then stay constant for the next 12 cycles. The rates used for the first 12 cycles are λ=1, 5, 10, and 20 transactions per period. Each EDE is initialized using the procedure used above for the constant rate model.

FIG. 11

shows the simulated absolute relative error curves for the first period probability for the MLE and the EDE with weights w=0.02, 0.05, and 0.1 over 24 cycles of the dynamic Poisson timing model with deterministic shifts in the period rates at the end of the 12th cycle. Each panel of the figure corresponds to a different transaction rate for the first 12 cycles, which is the same for all periods. The results shown in

FIG. 11

indicate the following:

1. The EDE is able to absorb the shift in the rate considerably faster than the MLE.

2. Estimators with larger w adapt faster, but the absolute relative error rate stabilizes at a large value.

3. The EDE with w=0.02 offers the best trade-off between ability to adapt to change quickly and low average absolute relative error over the period.

The standard deviations (not shown) for the absolute relative error curves are very similar to the ones listed for the constant rate model in FIG.

7

.

In summary, the MLE under the Poisson timing model with constant rates is generally not appropriate under dynamic Poisson models with high or moderate levels of noise and under Poisson models with shifts in the transaction rates. However, the EDE of the illustrative embodiment of the invention with w=0.02 performs well under a variety of scenarios.

The present invention in the illustrative embodiments described herein provides a fast, memory-efficient procedure for sequentially estimating timing distributions. The procedure uses only the times of the current and the last transaction to update the transaction rates and the period probabilities and can therefore be used with stream transaction data that are continuously collected over time. The EDE of the present invention is competitive with a conventional MLE under a Poisson timing model with constant rates and considerably better than the conventional MLE when transaction times follow a dynamic Poisson timing model. In the application to tracking the calling behavior of 2,000 customers described above, the Poisson timing model was shown to be reasonable for most individuals. The EDE performed well even for those customers for which the Poisson timing model may not be adequate. Similar performance improvements have been observed in numerous other applications.

It should be understood that the above-described embodiments of the invention are intended to be illustrative only, and the invention may be implemented in numerous other embodiments. For example, although the simulation results for the illustrative embodiments described above indicate that an appropriate weighting for the EDE under a dynamic Poisson model is about 1/{square root over (α)}, other embodiments may use other weights determined to be more appropriate for a given application. In addition, although the illustrative embodiments use fixed updating weights, the invention can also be used with updating weights that change with time. An one possible example of such an arrangement, if a transaction rate cannot be initialized reliably, then the updating weight can be larger for the first several transactions to allow the estimated rates and probabilities to move rapidly away from the initial value. Later, if the transaction rates become stable, the updating weight can be decreased to reduce the variability in the EDE. As another example, the updating weight may depend on the current estimate of the transaction rate. Furthermore, alternative embodiments of the invention may utilize different system configurations, different types of records and different updating processes. These and numerous other alternative embodiments within the scope of the following claims will be apparent to those skilled in the art.

Number	Name	Date	Kind
5870752	Gibbons et al.	Feb 1999	A
5907602	Peel et al.	May 1999	A
5912905	Sakai et al.	Jun 1999	A
6012064	Gibbons et al.	Jan 2000	A
6381589	Leon	Apr 2002	B1

Method and apparatus for updating records in a database system based on an improved model of time-dependent behavior

Information

Patent Number

Date Filed

Date Issued

Inventors

Original Assignees

Examiners

Agents

CPC

US Classifications

Field of Search

US

International Classifications

Term Extension

Abstract

Description

Claims

PRIORITY CLAIM

US Referenced Citations (5)

Non-Patent Literature Citations (4)

Provisional Applications (1)

Entry
F. Chen et al., “Incremental Quantile Estimation For Massive Tracking,” Proceedings of KDD-2000, pp. 516-522, 2000.
M. West et al., “Introduction to the DLM: The First-Order Polynomial Model,” Bayesian Forecasting and Dynamic Models, Chapter 2, Springer Verlag, pp. 37-55, 1989.
B. Abraham et al., “Locally Constant Mean Model and Simple Exponential Smoothing,” Statistical Methods for Forecasting, John Wiley and Sons, pp. 85-89, 1983.
P.B. Gibbons et al., “Fast Incremental Maintenance of Approximate Histograms,” Proceedings of the 23rd VLDB Conference, Athens, Greece, 5 pages, 1997.