Patent Grant
8086655
The present invention relates generally to the field of data stream publishing, and more particularly, to data perturbation of data stream values, with application in privacy preservation.
Time series data are prevalent in a wide range of domains and applications, such as financial, retail, environmental and process monitoring, defense and health care. Additionally, massive volumes of data from various sources are continuously collected. However, data owners or publishers may not be willing to exactly reveal the true values due to various reasons, most notably privacy considerations. A widely employed and accepted approach for partial information hiding is based on random perturbation. See, for example, R. Agrawal et al., “Privacy Preserving Data Mining,” In SIGMOD, 2000, which introduces uncertainty about individual values. Consider the following examples:
A driver installing a vehicle monitoring system (see, for example, D. Automotive, “CarChip,” http://www.carchip.com/, and W. P. Schiefele et al., “SensorMiner: Tool Kit for Anomaly Detection in Physical Time Series,” Technical Report, http://www.interfacecontrol.com/, 2006) may not wish to reveal his exact speed. How can he, for example, avoid revealing small violations of the speed limit (say, by 3-5 mph) but still allow mining of general driving patterns or detection of excessive speeding?
A financial services company may wish to provide a discounted, lower-quality price ticker with a specific level of uncertainty, which is not useful for individual buy/sell decisions but still allows mining of trends and patterns. How can they ensure that the level of uncertainty is indeed as desired?
Similarly, a financial institution (see, for example, Y. Zhu et al., “StatStream: Statistical Monitoring of Thousands of Data Streams in Real Time,” In VLDB, 2002) may not wish to reveal amounts of individual transactions over time, but still allow mining of trends and patterns. How can they control the level of uncertainty (or, privacy) in the published data and ensure that nothing more can be inferred?
Prior work on numerical and categorical data has focused on the traditional relational model, where each record is a tuple with one or more attributes. Existing methods can be broadly classified into two groups and work (i) either by direct perturbation of individual attributes separately (see, for example, R. Agrawal et al., “Privacy Preserving Data Mining,” In SIGMOD, 2000; D. Agrawal et al., “On the Design and Quantification of Privacy Preserving Data Mining Algorithms,” In PODS, 2001; and W. Du et al., “Using Randomized Response Techniques for Privacy-Preserving Data Mining,” In KDD, 2003) or of entire records independently (see, for example, H. Kargupta et al., “On the Privacy Preserving Properties”; Z. Huang et al., “Deriving Private Information from Randomized Data,” In SIGMOD, 2005; K. Liu et al., “Random Projection-Based Multiplicative Data Perturbation for Privacy Preserving Distributed Data Mining,” IEEE TKDE, 18(1), 2006; and K. Chen et al., “Privacy Preserving Data Classification with Rotation Perturbation,” In ICDM, 2005), (ii) or by effectively swapping or concealing values among an appropriately chosen small group of “neighboring” records (see, for example, L. Sweeney, “k-anonymity: A Model for Protecting Privacy,” IJURKS, 10(5), 2002; C. C. Aggarwal et al., “A Condensation Approach to Privacy Preserving Data Mining,” In EDBT, 2004; E. Bertino et al., “Privacy and Ownership Preserving of Outsourced Medical Data,” In ICDE, 2005; and A. Machanavajjhala et al., “l-diversity: Privacy Beyond k-anonymity,” In ICDE, 2006).
Although some of the prior work on relational data has considered certain forms of privacy breaches that are possible by exploiting either the global or local structure of the data (see, for example, A. Machanavajjhala et al., “l-diversity: Privacy Beyond k-anonymity,” In ICDE, 2006; Z. Huang et al., “Deriving Private Information from Randomized Data,” In SIGMOD, 2005; H. Kargupta et al., “On the Privacy Preserving Properties of Random Data Perturbation Techniques,” In ICDM, 2003; and K. Chen et al., “Privacy Preserving Data Classification with Rotation Perturbation,” In ICDM, 2005), the additional aspect of time poses new challenges, some of which are related to fundamental properties of time series (see, for example, D. L. Donoho et al., “Uncertainty Principles and Signal Recovery,” SIAM SIAP, 49(3), 1989). In particular: (i) sophisticated filtering techniques may potentially reduce uncertainty thereby breaching privacy; (ii) time series can be “described” in a large number of ways (in a sense, a univariate time series is a single point in a very high-dimensional space [see, for example, C. C. Aggarwal, “On k-anonymity and The Curse of Dimensionality,” In VLDB, 2005]—for example, if the series has 1000 points, there are many 1000-dimensional bases to choose from); (iii) time series characteristics may change over time and, in a streaming setting, new patterns may start emerging in addition to old ones changing (for example, it is not possible to know about quarterly or annual trends while still collecting the first week of data), making both static, global as well as fixed-window analysis unsuitable.
In accordance with the aforementioned and other objectives, the embodiments of the present invention are directed towards methods and apparatus for data perturbation of data stream values, with application in privacy preservation. The trade-offs between time series compressibility and partial information hiding are examined, as well as their fundamental implications on how uncertainty about individual values by perturbing them should be introduced. More specifically, if the perturbation does not have the same compressibility properties as the original data, then it can be detected and filtered out, reducing uncertainty. Thus, by making the perturbation “similar” to the original data, the structure of the data can be preserved better and breaches are made more difficult. However, as data become more compressible, a fraction of overall uncertainty can be removed if true values are leaked, revealing how they were perturbed. These notions are formalized, the above trade-offs on real data are studied, and practical schemes which strike a good balance and can also be extended for on-the-fly data hiding in a streaming environment are developed.
For example, in one aspect of the present invention, techniques for perturbing an evolving data stream are provided. The evolving data stream is received. An online linear transformation is applied to received values of the evolving data stream generating a plurality of transform coefficients. A plurality of significant transform coefficients are selected from the plurality of transform coefficients. Noise is embedded into each of the plurality of significant transform coefficients, thereby perturbing the evolving data stream. A total noise variance does not exceed a defined noise variance threshold.
In additional embodiments of the present invention the online linear transformation may be a Fourier transform or a wavelet transform. The defined noise variance threshold may be predetermined by at least one of a user and application requirements. The noise may be additively embedded or random. The plurality of significant transform coefficients may include transform coefficients that have an absolute magnitude exceeding a defined threshold. The noise may be embedded into each significant transform coefficient in proportion to a magnitude of each significant transform coefficient.
In further aspects of the present invention and apparatus for perturbing an evolving data stream is provided as well as a method for making a computer implemented process to enable perturbing of an evolving data stream.
These and other objects, features and advantages of the present invention will become apparent from the following detailed description of illustrative embodiments thereof, which is to be read in connection with the accompanying drawings.
a and b are a series of diagrams illustrating time-frequency properties, according to an embodiment of the present invention;
a and b is a series of graphs illustrating intuition via perturbation most resilient to any true value leaks and most resilient to any linear filtering, according to an embodiment of the present invention;
The embodiments of the present invention relate to univariate time series, examine the trade-offs of methods for partial information hiding via data perturbation, and propose a practical approach that is evaluated against both filtering attacks and, also, true value leaks. Additionally, they are suited for time-evolving (i.e., non-stationary) series and can be adapted for on-the-fly data hiding in a streaming setting.
The main idea is exemplified by the two extreme cases, which are explained in more detail in below with regard to
In the first case, each time instant is perturbed independently, while in the second case all time instants are perturbed in the same way. But what if groups (or windows) of values are perturbed in the same way within a group, but differently across groups? How should these groups be chosen? Based on this insight, these questions are addressed using both Fourier and wavelet transforms.
Therefore, the embodiments of the present invention expose and study the relationship between data representation, compressibility and privacy preservation via perturbation, in the context of time series. The embodiments of the present invention also introduce the notion of compressible perturbation, which determines the best way to perturb the data depending on the perturbation magnitude and the properties of the data. Further, these embodiments examine the trade-off between breaches that exploit compressibility via filtering operations and breaches that rely on leaks of true (i.e., unperturbed) values. Finally, the embodiments of the present invention present schemes that are based on the Fourier transform and on wavelets. The wavelet-based scheme is also amenable to streaming time series.
The trade-offs between privacy and compressibility are presented, as well as the efficiency and effectiveness of the approach on real time series.
Referring initially to
Wavelets are best introduced with the Haar transform, because of its simplicity. A more rigorous introduction to wavelets along with an introduction to the Fourier transform can be found, for example, in D. B. Percival et al., “Wavelet Methods for Time Series Analysis,” Cambridge Univ. Press, 2000. Given a series with N points, ν0,t:=xt is defined to start the Haar DWT construction. At each iteration, or level, l=1, 2, . . . , log2 N two operations on νl−1,t are performed to compute the coefficients at the next level:
The scaling factor of 2−1/2 ensures that the total energy (i.e., sum of squares of all values) is preserved. The coefficients of level l+1 are half as many as those of l and correspond to a time window twice the size. Construction is stopped when wl,t consists of one coefficient, which happens at l=log2 N+1. The total number of wavelet coefficients across levels is N−1.
There are several families' wavelet transforms that follow the above recursive pyramid algorithm, using a pair of filters, one high-pass and one low-pass. For example, in Haar wavelets, this pair consists of the simple first-order differencing and averaging filters, respectively. More generally, for each L≧1, Daubechies-L (or DB-L) wavelets use an L-th order difference filter for the high-pass operation and the corresponding low-pass filter (for more details, see, for example, D. B. Percival et al., “Wavelet Methods for Time Series Analysis,” Cambridge Univ. Press, 2000). These filters have 2L non-zero coefficients.
Referring now to
In the above example, note that estimation of both νl,t and wl,t requires only the two last scaling coefficients from the previous level, at νl−1,2t and νl−1,2t+1. In general, Daubechies-L DWT requires the last 2L scaling coefficients from the previous level. Thus, it is possible to perform the DWT incrementally as new points arrive, by buffering only 2L numbers for each of the l≦log2 N levels. The total time required is still proportional to N, i.e., constant per new value.
Because of their time/frequency decomposition properties, wavelets have been successfully used in signal estimation and denoising (see, for example, D. L. Donoho et al., “Adapting to Unknown Smoothness Via Wavelet Shrinkage,” J. Am. Stat. Soc., 90, 1995; and D. L. Donoho, “De-noising Via Soft Thresholding,” IEEE TOIT, 41(3), 1995).
Assume that the representation of a time series with N points in some basis is given. This representation consists of N numbers and can be obtained by applying an orthonormal transform (specifically, change of coordinates) to the original series {xt}t=1N. Also assume that the noise is i.i.d. (specifically, white) and its variance σ is known. Given the above, the ideal denoiser is simple: any coefficient whose magnitude is below σ is discarded as noise, otherwise it is retained. Then, the important questions are: (i) how to choose an appropriate basis, (ii) how to estimate σ when it is not known, and (iii) what to do with the retained coefficients.
For the first question, it is ideal to want the basis that compresses the signal into the smallest possible number of coefficients or, equivalently, has the largest possible number of zero coefficients. This implies that the remaining, non-zero coefficients will have a large magnitude, making them easy to distinguish from noise coefficients. Of course, it is not possible to know this optimal representation for a single series; differently put, the optimal basis for a specific realization of a series is always just the series itself, which is not very useful. Therefore, it is desirable to choose a representation that is appropriate for a class of signals. As already mentioned, wavelets successfully compress man0y real-world series (see, for example, D. L. Donoho, “Progress in Wavelet Analysis and WVD: A Ten Minute Tour,” In Y. Meyer and S. Rogues, editors, Progress in Wavelet Analysis and Applications, Frontiéres, 1993), because of their time/frequency decomposition properties and are thus an appropriate choice.
Having chosen wavelets to represent the series, it can be shown that the risk-optimal estimate of the noise variance is the median over t of the absolute magnitude, |w1,t|, of the first-level coefficients (see, for example, D. L. Donoho et al., “Adapting to Unknown Smoothness Via Wavelet Shrinkage,” J. Am. Stat. Soc., 90, 1995). Additionally, the best way to perform thresholding is to shrink each retained coefficient towards zero, rather than keeping them intact. This is also known as soft thresholding and its application to the wavelet representation is known as wavelet shrinkage.
Referring now to
For both extremes it is assumed that, in the worst case, an attacker may have full knowledge of the true data, but in different ways. In the first, an attacker is allowed direct access to an arbitrary number of true values (in the time domain). In the second extreme, the attacker is allowed to know the shape of the series with arbitrary accuracy (specifically, the attacker may know the one-dimensional subspace spanned by the series itself). It is always assumed that an attacker uses linear functions/filters to obtain estimates of the true data (see, for example, Z. Huang et al., “Deriving Private Information from Randomized Data,” In SIGMOD, 2005; and F. Li et al, “Hiding in the Crowd: Privacy Preservation on Evolving Streams Through Correlation Tracking,” In ICDE, 2007).
a(1-3) illustrates the perturbation that is resilient to any number of true value leaks. In this case, each time instant must be perturbed independently of others, in order to prevent any inferences across values. This requirement is always satisfied by white noise, specifically, independent, identically distributed random values. A realization of a white noise process is shown in the bottom panel of
b(1-3) illustrates the perturbation that is resilient to knowledge of the exact shape of the series. In this case, the perturbation must be completely indistinguishable from the original series. In other words, it should be perfectly correlated with the original series. Clearly, this is guaranteed if the perturbation is an exact copy of the original series, except for rescaling of all values by the same factor. The result is shown in FIG. 3(b1), with the same perturbation magnitude as in the previous example. As expected, any kind of linear filtering is unable to separate the perturbation from the true series—
Referring now to
Referring now to
A common measure of uncertainty is standard deviation, specifically, root mean square value of a series. Standard deviation is used to measure two important aspects: (i) discord between perturbed and original data, and (ii) remaining uncertainty about the true values, after attempts to recover them. It is desirable for the discord to be as low as possible and, in particular, at most equal to a chosen threshold. The utility of the published data drops as the discord increases (see, for example, A. Evfimievski et al., “Limiting Privacy Breaches in Privacy Preserving Data Mining,” In PODS, 2003; and D. Kifer et al., “Injecting Utility into Anonymized Datasets,” In SIGMOD, 2006). On the other hand, given the discord, it is desirable for the remaining “true” uncertainty to be as high as possible, ideally equal to the discord. These notions are formally defined below.
Additive Perturbation—Given a series xt, for t≧1, a corresponding perturbation series nt is chosen with zero mean, E|nt|=0, and publish the series yt:=xt+nt, for all t≧1.
Discord—Discord σ is the standard deviation of the perturbation, i.e.,
σ2:=Var[yt−xt]=Var[nt]=E[nt2].
The discord threshold is given and determines both the maximal loss of information that is tolerable, as well as the maximum uncertainty that can be introduced. In fact, these two quantities should be equal and this is precisely the goal. However, they may not be equal, because an adversary can apply techniques that reduce the uncertainty.
Given the discord threshold, the available perturbation latitude is always fully exploited, specifically, the goal will be to add a perturbation amount equal to the threshold. Thus, from now on, the discord and its threshold are not distinguished, using σ to denote both.
Given the published values yt, for t≧1, an adversary may attempt to obtain an estimate of the true values, which may reduce the overall uncertainty. The discord (specifically, uncertainty originally introduced by the data publisher) is the standard deviation of the difference between true and published values. Similar to this, the remaining uncertainty is measured with the standard deviation of the difference between true values and the adversary's estimates. This remaining uncertainty is a measure of privacy achieved under each attack setting.
Two attempts are considered for estimating the true values, each with different, worst-case assumptions about the background knowledge available. In both cases, it is assumed that an adversary applies linear functions or filters to obtain an estimate of the true values.
The fast one relies on linear filtering methods, which attempt to separate the perturbation from the true data. The filtering technique employed is described above and has been proven very successful in a wide range of domains and applications (see, for example, D. L. Donoho et al., “Adapting to Unknown Smoothness Via Wavelet Shrinkage,” J. Am. Stat. Soc., 90, 1995; and D. L. Donoho, “De-Noising Via Soft Thresholding,” IEEE TOIT, 41(3), 1995).
Filtering Uncertainty—Let {tilde over (y)}t be the result of a linear filtering operation on the published series yt. The filtering uncertainty is the remaining uncertainty after this operation, i.e.,
{tilde over (σ)}2:=Var[{tilde over (y)}t−xt].
In practice, the standard deviation {tilde over (σ)} of the filter's output is estimated by applying the filtering operation on a finite time series consisting of N points and using the sample estimate of the standard deviation,
With a slight abuse of notation, the sample estimate is also denoted with {tilde over (σ)}{tilde over ( )}instead of {tilde over (s)}.
In this case, an adversary has the background knowledge that the signal has a compact representation in some space, and more specifically, that the largest fraction of its energy is concentrated on a few transform coefficients. This is a very common assumption in signal estimation and recovery (see, for example, D. L. Donoho, “Compressed Sensing,” IEEE TOIT, 52(4), 2006; and D. L. Donoho, “De-Noising Via Soft Thresholding,” IEEE TOIT, 41(3), 1995), and amounts to a constraint on the “shapes” that the series is allowed to have. All practical applications of signal recovery need to make an assumption about the actual transform. Wavelet-based techniques have been shown most successful for a wide range of real-world signals (see, for example, D. L. Donoho et al., “Adapting to Unknown Smoothness Via Wavelet Shrinkage,” J. Am. Stat. Soc., 90, 1995), performing at least as well as Fourier-based techniques.
The second kind of attempt to partially remove the perturbation relies on true value leaks. By construction yt=xt+nt, and, if nt is Gaussian white noise, this is precisely the model for least-squares linear regression. This observation leads naturally to the next definition.
Leak Uncertainty—Let ŷt be the linear regression estimate obtained by fitting a line to the true vs. perturbed values, i.e., ŷt=αyt+β where ŷt are chosen so as to minimize the residual error Σt(xt−{tilde over (y)}t)2. This RMS error is the measure of true value leak uncertainty, i.e.,
{tilde over (σ)}2:=Var[{tilde over (y)}t−xt].
In practice, it is desirable to estimate {tilde over (σ)} from a finite sample. The least-squares estimators of α and β are
are the sample means. The sample estimate of the residual variance is
Since a and b are unbiased estimators, their expectation over all finite samples is E|a|=α and E|b|=β.
Leak uncertainty is the minimum error that can be achieved by any linear function for estimating the true values, even if it is assumed that an adversary knows all true values. Therefore, the measure is a worst-case estimate of privacy loss, under the assumptions that an adversary uses linear estimation techniques and has access to any number of true values.
Furthermore, the distribution of N{tilde over (s)}2/{tilde over (σ)}2 is χ2 with N−2 degrees of freedom (see, for example, M. H. DeGroot et al., “Probability and Statistics,” Addison Wesley, 3rd ed. edition, 2002). Therefore, even if a small subset of M<N samples was used to estimate ŝ, its expectation over all leaks of size M would still be E[ŝ2]={circumflex over (σ)}2(M−2)/M≈{circumflex over (σ)}2. The standard deviation Dev[ŝ2] drops quickly, in proportion to {circumflex over (σ)}2/M. Finally, again with a slight abuse of notation, from now on {circumflex over (σ)}2 will be used instead of ŝ.
For single time series, trends and patterns often refer to bursts (see, for example, Y. Zhu et al., “Efficient Elastic Burst Detection in Data Streams,” In KDD, 2002) and dominant periods (see, for example, M. Vlachos et al., “Structural Periodic Measures for Time-Series Data,” DMKD, 12(1), 2006). Such analysis is largely performed on the spectrum of the signal. Whether a perturbation preserves these key properties depends on (i) how much perturbation is added, which is the discord, and (ii) how the perturbation is added. In most perturbation methods, the first is a parameter determined by the end user. Additionally, both of perturbation techniques, naturally preserver the spectral and “smoothness” properties of the original signal, by design. Hence, the proposed perturbation techniques will be useful in preserving both privacy and utility of time series.
Two potential breaches are considered with different assumptions about background knowledge. In the first case, it is assumed that an adversary knows that a series has a compact representation in some linear subspace (for example, few non-zero wavelet or Fourier coefficients). In the second case it is assumed that an adversary knows any number of true values, in the time domain. In both cases it is assumed that linear estimation techniques are used. Practical techniques are proposed to address both challenges and the techniques are evaluated under the two different attack models on a number of real datasets.
As pointed out, the simple solution of perturbing the series with white noise does not work, because white noise is incompressible under any representation (or basis). As a result, the added perturbation is “diluted” over coefficients that are not important in representing the series. Consequently, a large portion of the white noise can be removed, leading to a significant decrease in remaining, true uncertainty over individual true values. Thus, the goal is to avoid this problem, by appropriately adapting the perturbation to the original series.
The perturbation nt for each value at time t will be chosen based on a given discord a and, of course, the series {xt} itself. Since (i) it is impossible to design a method that is optimally resilient against both filtering and leak attacks, and (ii) filtering is possible at any and all time instants since it requires no prior knowledge about the true data, resilience is used for filtering as the primary guide in designing the techniques, but also taking leak attacks into consideration and the methods are evaluated with respect to both potential attacks. The general steps to construct the perturbation are:
(S0) Choose a “description” or basis.
(S1) Perturb only those coefficients that are “important” (to be made precise later) in the chosen description.
(S2) Determine by how much to perturb them.
The first step consists of applying an orthonormal transform which, given the N time domain values xt, for 1≦t≦N, will produce another set of N coefficients, ci for 1≦i≦N. Next, it is assumed for the moment that Gaussian white noise is added with variance σ2. This simply means that each coefficient is perturbed by a random number c, drawn according to a Gaussian distribution with zero mean and standard deviation σ, ci←G
However, it has been established that this approach is susceptible to filtering attacks. Therefore, a basis is chosen that successfully compresses a large class of time series, in the sense that it concentrates its energy into a few transform coefficients. Recall that the ideal denoiser, given a basis, discards all coefficients below the (true or estimated) noise variance. Therefore, any noise embedded into such coefficients is “wasted,” as it can be easily separated from the dominant coefficients. This observation leads to the conclusion that only those coefficients with magnitude greater than σ are “important” for perturbing the data in a way that is resilient to filtering attacks.
Therefore, instead of allocating the N available noise units into all N coefficients, they are allocated to the set of coefficients whose magnitude exceeds σ. Let I:={i:|ci|≧σ} be the set of their indices. However, in order to ensure that Var[nt]=σ2, the variance of the random number that will be added to each ci, for iεI needs to change as well. For example, a simple choice would be a random number with variance ρi:=N/K to each of them, where K:=|I| is the number of coefficients that exceed σ. This ensures that E[Σici′2/N]=E[ΣΣiεIci2′]/N+E[Σi∉Ici′2]/N=Kσi2σ2/N+(N−K)·0/N=K(N/K)σ2/N+0=σ2, since each ci′εC is perturbed independently. In other words, the expected sample variance of the perturbation series will be σ2 as desired. More generally, any ρi is chosen such that Σiρi2=N.
Referring now to
In this section, two batch perturbation methods are proposed that rely on pure frequency or time/frequency representations of the series. In particular, the first is based on the well-established Fourier representation of the entire, length-N series. The second is based on the wavelet representation. First, Fourier and wavelet perturbation are studied in a batch setting and the wavelet-based scheme is revisited, adapting it to a streaming setting.
Referring now to
Intuitively, each sinusoid is perturbed by randomly changing its magnitude and phase (lines 4-5 in
where χN−k+1* denotes the complex conjugate of χN−k+1. The DC coefficient χ1 is always real and equal to the series mean. If N is odd, then χN/2+1 is also real (case not shown in
Because of Equation (1), essentially the first half of the Fourier transform carries all the necessary information. The per-frequency energy (or, more accurately, its square root) is computed in line 1 of
Referring now to
The allocation of “noise units” into the important frequencies is done in proportion to N/K as well as in proportion to the energy content of each perturbed frequency (factor of pk2/P in line 3 of
Fourier-based perturbation generally performs well for series that are dominated by a few frequencies which do not change over time. If the series has discontinuities or frequency shifts, then Fourier may perform worse, because phenomena localized in time are spread across frequencies. This effect would allow a potential attacker to remove more uncertainty, roughly in proportion to the magnitude of such discontinuities (either in time or in frequency) and in inverse proportion to the number of frequencies.
Finally and more importantly, the Fourier transform of a growing series cannot be updated incrementally. One potential solution might be to use the short-time Fourier transform (STFT), but a fixed-size time window is undesirable. Next, a wavelet-based perturbation method is developed. Wavelets employ multiple window sizes to decompose the series and are also amenable to streaming estimation.
Referring now to
Wavelets have been successful in a wide range of settings (see, for example, T. Li et al., “A Survey on Wavelet Applications in Data Mining,” SIGKDD Explorations, 4(2), 2002) and are more resilient to changes in series' characteristics. They decompose the series into translated and dilated, localized waves at multiple scales, which correspond to a particular time and frequency window. Short windows are employed for high frequencies (specifically, short periods) and longer windows for lower frequencies (specifically, longer periods).
The localization of bases in time has the additional desirable characteristic that, intuitively, each period is perturbed independently of others. For example, assume that by following an automobile, its true speed is learned over a period of 15 minutes. However, if periodic trends shorter than 15 minutes are perturbed independently, collected true values can tell nothing about the future perturbation at scales of up to 15 minutes. For periodic trends in the next scale of 30 minutes, perhaps the information learned will be useful for another 15 minutes, but not longer, and so on for scales of 60 minutes, etc.
Finally, the DWT can be computed in O(N) time, as opposed to O(N log N) time required by FFT (see, for example, D. B. Percival et al., “Wavelet Methods for Time Series Analysis,” Cambridge Univ. Press, 2000). Thus, even in a batch setting they are computationally more efficient. Furthermore, wavelets can be estimated incrementally, using just O(log N) total space and O(1) amortized time per value. Since they have several desirable benefits, wavelets are focused on.
It is a goal to choose an effective perturbation that is hard to remove, but it is desirable to perturb values as they arrive, before seeing the entire series, which grows indefinitely. Furthermore, it is desirable to minimize or eliminate publishing delay.
The Fourier transform needs, by definition, the entire series which is clearly not possible in this case. One solution is to partition the series into fixed-size windows and apply Fourier on each of them. However, if a small window is used, it is not possible to capture trends with period larger then the window length. For example, if a 5-minute window is used to perturb driving speed, it is still possible to leverage hourly or daily driving patterns to reduce uncertainty. If a large window is used, then it may be necessary to delay publishing the data until the window is filled up, so it is analyzed and perturbed. Alternatively, the frequencies could be used from the previous window to perturb the current one. However, if the window is large, it may not capture trends that have substantially changed in the new window. For example, a car might have been on the highway driving with a constant speed during the last hour, but has now entered a city and is in stop-and-go traffic. If a single one-hour window is used, the perturbation will follow the wrong trends.
Thus, the time/frequency decomposition of wavelets, which use multiple windows proportional to the period is desirable. In this case, the information of the last, for example, 5 minutes, is used to decide if and how to perturb, during the next 5 minutes, patterns up to that long. However, the information of the last 10 minutes is used to make the same decision for smoother, longer patterns (up to 10 minutes) during the next 10 minutes, and so on. However, steps (S1-2) of
Revisiting step (S1). If it is desirable to make an exact decision whether to perturb a coefficient wl,t based on its actual magnitude (lines 2 and 3-4 in
A maximum delay (equivalently, a maximum level willing to wait for) may be imposed, but that is effectively the same as using a fixed-length window. Instead, the noise is embedded into the next coefficient of the same level, specifically, ωl,t+1′ is used instead of ωl,t′ in lines 3 and 4. Said differently, the important coefficients in step (S1) are chosen based on the magnitude of previous coefficient at same band. For example, referring now to
This simple one-step prediction is effective, since the only interest is whether a coefficient exceeds σ, rather than in its exact value. More specifically, periodic trends result in uniformly large coefficients at the corresponding wavelet level. Bursts also tend to affect more than one consecutive coefficient—if not, that is the only case that may be missed. However, such very short bursts generally occur at small scales and can safely be ignored.
Revisiting step (S2). The number K of coefficients exceeding σ (line 1 of
N←N+1
if |wl,t|≧σ then
K←K+1
ρ←λρ+(1−λ)(N/K)
The order in which wavelet coefficients are incrementally computed is shown in
The inverse DWT can be computed incrementally in O(1) time per value, using O(log N) space. The forward transform can be performed incrementally because it is a post-order traversal of the coefficient tree (see
The methods of the present invention are evaluated on several series from the UCR Time Series Data Mining Archive (TSDMA) (see, for example, E. Keogh et al., “Ucr Time Series Data Mining Archive,” http://www.cs.ucr.edu/˜eamonn/TSDMA/)—see
The prototype is built in Matlab 7, running on a Pentium M 2 GHz with 2 GB memory. The Wavelet Toolbox is used for batch wavelet transforms, as well as for wavelet denoising (SureShrink [see, for example, D. L. Donoho et al., “Adapting to Unknown Smoothness via Wavelet Shrinkage,” J. Am. Stat. Soc., 90, 1995], with DB-4 wavelets and the rigorous version of single-level noise estimation). One experimental run is performed for several different values of the discord σ, ranging from 5% to 40% of the total series standard deviation, at steps of 5%. For each experiment and for each method, ten perturbation trials are run. Each trial produces a different random perturbation. The baseline method is white noise (i.i.d. Gaussian random perturbation) and included are (i) batch wavelet perturbation (DWT), (ii) its streaming version (Streaming DWT), and (iii) Fourier perturbation (FFT, comparing two noise allocation schemes—all figures are with per-band weighting as in
It is first examined how much uncertainty can be removed by either a filtering or a true value leak attack on data perturbed with each method. In particular, the fraction of uncertainty removed is examined, i.e.,
{tilde over (f)}(σ):=(σ−{tilde over (σ)})/σ and {circumflex over (f)}(σ):=(σ−{circumflex over (σ)})/σ,
for several different values of the discord a (ranging from 5% to 40%). Both the maximum (specifically, worst-case value) and average of {tilde over (f)} and {circumflex over (f)} are estimated across the ten perturbation trials in each experiment.
Referring now to
Note that, by construction, reconstruction from true value leaks does not help at all for white noise (even though not visible, all bars are zero). However, filtering can very successfully remove from 20-30% of the perturbation (for Light) up to almost 90% (for SP500). Thus, the need to take into account the characteristics of the series by using an appropriate, concise description is clear beyond doubt.
Having established this, it is observed that all three of the proposed methods perform similarly. The streaming, wavelet perturbation method performs slightly better than the other two in some occasions. The reason is that it may initially overestimate the “density” ρ=N/K, particularly for series that have a large number of coefficients below the discord σ. This results in adding slightly more noise which, however, is never beyond 1-3% more than desired. Fourier perturbation may perform somewhat worse on certain data. However, as described below, it may exhibit sensitivity to certain data characteristics and, in particular, the presence of sharp discontinuities. Overall, however, all three methods perform well on a wide variety of series and stay close to the optimal diagonal.
Finally, for wavelet-based perturbation, the average and maximum uncertainty reduction are closer to each other. In some cases the discrepancy between the two is larger for Fourier. Thus, even though all three methods have similar average behavior, wavelets perform more consistently.
In order to measure the uncertainty u(σ) that remains after attempted attacks of any type, the fraction of the perturbation that remains in the worst case (i.e., after the most successful of the two attacks) is also shown. In particular,
u(σ):=min{σ(1−{tilde over (f)}(σ)),σ(1−−{circumflex over (f)}(σ))},
where {tilde over (f)}(σ) and {circumflex over (f)}(σ) are estimated over ten trials, as explained before.
Referring now to
First, it is clear in these plots as well that white noise performs very poorly, allowing a very large reduction of uncertainty. All three of the proposed methods perform similarly. In Light, which exhibits sharp discontinuities, the largest fraction of the energy concentrated on daily and half-daily periods. Most of the remaining energy is smeared across frequencies, due to the frequent jumps. Thus, this concentration of energy on a few frequencies allows somewhat larger uncertainty reduction via leaks, due to the regularity of the perturbation.
Noise allocation in proportion to frequency band energy, and is compared to equal allocation. The comparison is performed for both Fourier and wavelet perturbation. By default,
Referring now to
On Chlorine, which consists mainly of a few, unchanging frequencies, Fourier perturbation performs similarly under both allocation schemes—see
Referring now to
The experimental evaluation clearly shows that white noise is insufficient for effective perturbation, particularly under the filtering attacks which are the primary concern. Thus, it is necessary to take the structure of the data into consideration, by seeking an effective, concise description of real data. Three methods are proposed which perform similarly on average. For series with stable spectral content limited to a small number of frequencies, all methods perform similarly. If the spectral content changes, then Fourier performs well on average but is less consistent overall. The perturbation method of the present invention that uses time/frequency wavelet analysis performs as well as or better than Fourier and is also suitable for streaming estimation.
Two potential breaches are considered, with different assumptions about background knowledge, each of which captures situations that may arise in practice. In particular, the first set of assumptions is most common in signal estimation and recovery applications, and essentially imposes either “global smoothness” constraints (via the background assumption of compact representation in the frequency domain) or “local smoothness” constraints (via the assumption of compact representation in the wavelet domain). The second set of assumptions deals with true value leaks and efforts for linear estimation of other true values, based on those that were leaked. In this case the worst-case view is taken that an arbitrary number of true values may be leaked. The leak uncertainty is a statistical measure of the maximum possible loss of privacy under these assumptions.
The embodiments of the present invention focus on practical aspects and the methods are extensively evaluated under both attack models, demonstrating that both are important in practice. In addition, the experimental evaluation presents both average-case results, in
In general, filtering attacks based on background knowledge about the “smoothness” properties of the data are the most important in practice. This is clear in all cases of
In summary, two novel aspects of partial information hiding and privacy are focused upon. Two real-world scenarios are considered; design robust and practical techniques which are also suitable for a streaming setting. For each aspect, the techniques of the present invention are evaluated extensively on real data.
From the first, seminal work on privacy preservation via partial data hiding (see, for example, R. Agrawal et al., “Privacy Preserving Data Mining,” In SIGMOD, 2000; and L. Sweeney, “k-anonymity: A Model for Protecting Privacy,” IJUFKS, 10(5), 2002) until today, there is an increasing realization that subtle potential privacy breaches may arise when any regularity or structure is present in the entire collection of values considered as a single, complex data object (see, for example, H. Kargupta et al., “On the Privacy Preserving Properties of Random Data Perturbation Techniques,” In ICDM, 2003; Z. Huang et al., “Deriving Private Information from Randomized Data,” In SIGMOD, 2005; X. Xiao et al., “Personalized Privacy Preservation,” In SIGMOD, 2006; and A. Machanavajjhala et al., “l-diversity: Privacy Beyond k-anonymity,” In ICDE, 2006). The embodiments of the present invention address these challenges for time series data. They also consider true value leaks as well as filtering attempts, study the fundamental trade-offs involved in addressing both and propose a practical, effective method that is based on the wavelet transform, which has been widely successful in capturing the essential characteristics of data (see, for example, T. Li et al., “A Survey on Wavelet Applications in Data Mining,” SIGKDD Explorations, 4(2), 2002).
Referring now to
As shown, the computer system may be implemented in accordance with a processor 1710, a memory 1712, I/O devices 1714, and a network interface 1716, coupled via a computer bus 1718 or alternate connection arrangement.
It is to be appreciated that the term “processor” as used herein is intended to include any processing device, such as, for example, one that includes a CPU (central processing unit) and/or other processing circuitry. It is also to be understood that the term “processor” may refer to more than one processing device and that various elements associated with a processing device may be shared by other processing devices.
The term “memory” as used herein is intended to include memory associated with a processor or CPU, such as, for example, RAM, ROM, a fixed memory device (e.g., hard drive), a removable memory device (e.g., diskette), flash memory, etc.
In addition, the phrase “input/output devices” or “I/O devices” as used herein is intended to include, for example, one or more input devices (e.g., keyboard, mouse, scanner, etc.) for entering data to the processing unit, and/or one or more output devices (e.g., speaker, display, printer, etc.) for presenting results associated with the processing unit.
Still further, the phrase “network interface” as used herein is intended to include, for example, one or more transceivers to permit the computer system to communicate with another computer system via an appropriate communications protocol.
Software components including instructions or code for performing the methodologies described herein may be stored in one or more of the associated memory devices (e.g., ROM, fixed or removable memory) and, when ready to be utilized, loaded in part or in whole (e.g., into RAM) and executed by a CPU.
Although illustrative embodiments of the present invention have been described herein with reference to the accompanying drawings, it is to be understood that the invention is not limited to those precise embodiments, and that various other changes and modifications may be made by one skilled in the art without departing from the scope or spirit of the invention.
| Number | Name | Date | Kind |
|---|---|---|---|
| 6332030 | Manjunath et al. | Dec 2001 | B1 |
| 6681029 | Rhoads | Jan 2004 | B1 |
| 7127079 | Keating et al. | Oct 2006 | B2 |
| 20040068399 | Ding | Apr 2004 | A1 |
| 20070239444 | Ma | Oct 2007 | A1 |
| 20080010671 | Mates | Jan 2008 | A1 |
| 20080209568 | Chang et al. | Aug 2008 | A1 |
| Number | Date | Country | |
|---|---|---|---|
| 20090077148 A1 | Mar 2009 | US |
