Patent Application
20080021899
A secure dot product is applied 110 to the private query sample 101 and the private labeled samples 102 to determine securely distances 111 between the private query sample and the plurality of private labeled samples. A secure k-rank protocol is applied 120 to the distances 111 to determine a nearest distance 121 of a kth nearest labeled sample having a particular label. Then, a secure Parzen window protocol is applied 130 using the nearest distance 121 to label the private query sample according to the particular label 131.
In the above method, n private query samples can partitioned 115 into √{square root over (n)} clusters, each cluster including √{square root over (n)} private labeled samples, and private shares of the distances between the private query sample and the private labeled samples in one of the clusters can be obtained before applying the protocols 110, 120 and 130.
Density Estimation
We describe density estimation methods. All computations are performed over a prime field F=Zp for some prime number p. A size of the prime field p is exponential as to the cryptographic security parameter. In all cases, we need to evaluate if an expression is larger or smaller than zero. This cannot be done over the prime field F. Therefore, the parameters are integers. We let S be some upper bound on every intermediate, absolute value in the computation that is known to both Alice and Bob. Such a value S is implicit from the choice of representation of the inputs over the integers. Then, Alice and Bob select the prime field F with cardinality bigger than 2S. Due to wrapping (modulo F), negative numbers over the integers map to numbers bigger than |F|−S, and positive numbers are smaller than S.
Private Computation
Let ƒ: {0, 1}*×{0, 1}→{0, 1}*×{0, 1} be a function. We denote the first item of f(x1, x2) by ƒA(x1, x2), and the second item by ƒB(x1, x2). Let π be a two-party protocol for evaluating the function ƒ, and we denote the two parties by A for Alice, and B for Bob.
If Alice and Bob follow the protocol, then the parties are called semi-honest, or honest but curious. The views of A and B while applying the protocol π(x1, x2) are, respectively
viewAπ=(x1; rA; mA,1, . . . , mA,t) and
viewRπ=(x2, rB, mB,1, . . . , mB,t),
where for i=A, B, we denote by ri the random input of party i, and by mi,j the jth message received by party i. The output received by party i at the end of the protocol π(x1, x2) is outputiπ (x1, x2).
By private computation, we mean that everything that party i learns from viewiπ can only be determined from the input and the output. Formally, we require that viewiπ can be simulated from the input and output of party i.
We say that the protocol π privately evaluates a deterministic function ƒ if there exist a probabilistic processes SIMA and SIMB such that, for i=A, B
(SIMi(x1,ƒi(x1, x2))≅viewiπ (x1, x2).
Oblivious Transfer Protocol
The well known oblivious transfer (OT) protocol is a cryptographic process that enables Alice to select one item from Bob's database of items. Alice obtains this item without revealing to Bob which item was selected and without learning anything about the rest of the items in the database. The most common variant of OT is
The OT process can be constructed from any known enhanced trapdoor permutation.
The Millionaire's Problem
Alice has a number x, and Bob has a number y. Alice and Bob would like to compare the two numbers and determine who has the larger number, without revealing anything else about the numbers themselves. This is a generalization of the well known millionaire's problem where two parties desire to know who has the most money, without revealing the amount each party owns.
Protocol 1—Secure Millionaire Protocol
To compare the two numbers, Alice and Bob define two comparison values {,
} ε F, which are used to compare the result, typically {
,
}={0, 1}.
Input
Alice has a number x ε {0, 1}m, and Bob has a number y ε {0, 1}m.
Output
Alice obtains a, Bob obtains b, such that
This is the secure millionaire's protocol.
Protocol 2—Secure Dot Product Protocol
We use a secure dot product protocol to determine a distance between two samples in the prime field Fd, see below. The samples can be finger prints, biological sample, images, and the like.
Alice has a vector X ε Fd, and Bob has a vector Y ε Fd, Alice and Bob privately determine shares of the dot (inner) product. The secure dot product protocol is well known.
Input
Alice has X ε Fd, and Bob has Y ε Fd.
Output
Alice obtains a, Bob obtains b, such that a+b (mod F)=XT Y, where T is the conventional notation for a transpose operator.
Secure Rank Protocol
Alice and Bob can use the secure dot product protocol to determine respectively first and second private shares of a distance between the private query sample and labeled samples in the database of the classifier. Now, Alice and Bob need to determine labeled samples in a small neighborhood label the private query sample.
A size of the neighborhood can be either defined by a radius, as is done in Parzen window classification, or by order, as is done in k-nn classification. We can convert the order to the radius by determining the distance of the kth nearest sample, and then applying the Parzen window classification using this distance.
Alice and Bob have respectively first and second private shares of the distance d, of the nearest neighbor to the private query sample, and now wish to determine first and second private shares of the second nearest neighbor. Actually, Alice and Bob have random shares to the squared distances. This does change the ranking of the kth item.
The above can be expressed as determining a smallest distance, in a list of distances, subject to a constraint that this distance should greater than d1. This rules out selecting the nearest neighbor again because the distance d1 to the nearest neighbor is not greater than the threshold. By repeatedly updating the threshold, and applying the secure rank protocol for k-1 times, we determine the kth nearest neighbor. To preserve privacy, the threshold parameters are given as random shares.
We use two intermediate protocols in the secure rank protocol. The first, called the private shared-minimum protocol, is a modification of the millionaire's protocol in the case where both parties have private shares to the numbers to be compared. The output is a random share of the minimum distance. The second intermediate protocol adds a threshold parameter, and requires that the output is greater that this threshold.
Private Share Minimum Protocol
In this well known protocol, Alice and Bob have private shares of x and y, and want to obtain private shares of a minimum of x.
Formally, Alice has xA, yA and Bob has xB, yB such that xA+xB=x (mod F), and yA+yB=y (mod F). After the secure evaluation, Alice has zA and Bob has zB, such that za+zb=z (mod F), where z=min (x, y). Because x is an item of F, after adding xa+xb (mod F), we obtain exactly x, and not a value that is congruent to x. The same is true for y. Thus, we can compare x, y as integers, and obtain the random shares za, zb which are congruent to the minimum of (x, y), and yet are random in the prime field F.
We can construct a Boolean circuit for evaluating this function. The circuit sums xa+xb, and ya+yb, and subtracts |f| if the result is bigger than |F|. Then, we compare x, y, and output random shares of the minimum. We can use Yao's circuit for this private computation.
To extend the private share minimum protocol to more than two numbers, one can either iterate this protocol over the pairs of numbers or construct a circuit that directly compares more than two numbers.
Threshold Private Shared Minimum Protocol
The well known private threshold shared minimum protocol differs from the private shared minimum protocol by adding another input, the threshold, such that on input x, y, t, the output is the minimum of x, y that is also at least t. If x, y<t, then the output is t. The inputs are given as respective first and second private shares of Alice and Bob, and the output is given as random shares of the minimum.
We can construct a circuit for this function. The random shares are added modulo F, x, y, and are compared to t. If x, y>t, then the output is just the minimum of x, y. If x<t<y, then the output is y. If x, y<t, then the output is t.
Secure k-Rank Protocol
The well known secure k-rank protocol applies the protocol for k-1 times, updating the threshold after every iteration. Formally, Alice and Bob have respectively first private shares a1, . . . , an and second b1, . . . , bn, and a list of squared distances d1, . . . , dn. All distances are unique positive numbers. Then, the distance of the nearest neighbor is determined by setting the threshold to zero.
Protocol 3—Secure Parzen Window Protocol
A density of a Parzen-window can be estimated by
where the n labeled samples X1, . . . , Xn are independently and identically distributed (i.i.d), r is a radius of a volume of V=(2r)d, and ρ(u) is a window function defined as
That is, the Parzen window protocol p(X) measures a density of labeled samples in a hypercube centered around the private query sample X. Classifiers based on Parzen-window estimation, estimate densities of each class and classify the private query sample by the label corresponding to a maximum posterior probability of the labeled samples.
Bob has n private labeled samples in a d-dimensional space, {Xi, yi}ni=1, where c is a maximum number of classes, and yi are corresponding labels of the samples in the database. Alice has the private query sample X. Alice has the values c and n. Alice wants to privately label the private query sample X using the secure Parzen window protocol. Alice and Bob have respectively first and second private shares rA and rB, of a Parzen window with a radius r=rA+rB.
Given {Xi, yi}ni=1 private labeled samples in the database, we determine the label y of the majority of labeled samples that are within the radius r from the private query sample X.
The protocol for the secure Parzen window protocol proceeds as follows.
Bob knows only the radius r of the window, so Alice sets rA=0, and Bob sets rB=r. The results are private shares for Alice and Bob, and one party sends a private share to the other party to obtain the result.
In step 3, Alice and Bob apply a variant of the secure k-rank protocol that determines the maximum, not the minimum of a list of distances.
Obtaining the Private Query Sample
Bob desires to search for records in Alice's database. In this case, it is not enough to detect that a particular record exist in the database. Bob is interested in the information associate with the private query sample X. The secure Parzen window protocol only reveals the label of private query sample X, not its value or associated data record. This problem is solved as follows.
For simplicity of this description that the classification problem is a binary classification. At the beginning of the last step of the above secure Parzen window protocol, Alice has first private shares a1, b1, and Bob has second private shares a2, b2. Both parties agree that if a predicate a1+b1<a2+b2 is true, then the private query sample X is a possible candidate record, and Alice should provide the record to Bob, otherwise, Bob should learn nothing. To solve this Alice and Bob apply the private share minimum protocol. At the end of the private shared minimum protocol, Bob has a random permutation p of the predicate. Note that we reverse the roles of Alice and Bob in this case. Alice can now construct a two-way OT table that Bob can index with p. One entry in the table contains the private query sample, the other entry contains an empty string.
Protocol 4—Secure k-nn Protocol
A secure k-nn protocol maximizes
P(ym|X)=maxiP(yi|X) (4)
where y1, . . . , yc are the possible states. The k-nn protocol classifies the query sample X by measuring the density of each class and taking the maximum density. This is very similar to the secure Parzen window protocol, only now the size of the neighborhood is data dependent and not determined ahead of time. Therefore, we determine the distance of the kth nearest neighbor as a preprocessing step and then apply the secure Parzen window protocol. We determine the distance of the kth nearest neighbor by applying the shared minimum protocol k-1 times. Alice and Bob start with tA=tB=0, and use the output of each iteration as input for the next invocation of the threshold shared minimum protocol. After k-1 rounds, Alice and Bob have private shares of the correct distance, and they can invoke the secure Parzen window protocol to complete the secure k-nn protocol.
Given {Xi, yi}ni=1 labeled samples, we determine the label of the majority of labeled samples that are the k nearest neighbors of the private query sample X.
Input
Alice has the private query sample X. Bob has a database of labeled samples X1, . . . , Xn with labels y1, . . . , yn ε {1, . . . , c}, and a parameter k.
Output
Alice obtains the label of the majority of the samples within the k nearest neighbors of the private query sample X.
Nearest Match Protocol
In some cases, Alice might be interested in querying a database for the labeled sample nearest to her private query sample. For example, if Alice has a finger-print, she might be interested in querying a database to obtain the nearest match to her private query sample. This can be achieved by using our k-nn protocol with k=1. With slight modification of the secure k-nn protocol Alice can obtain all the attributes of the nearest match, not just its label.
Approximate Nearest Neighbors Protocols
Nearest neighbor methods scale linearly with the number of samples in the database. As a result, approximations are often used to speed up the computations. The approximations can be used with any of the protocols described above. We first describe the non-secure approximate nearest neighbor protocol and then describe how to make the approximate protocol secure.
Non-Secure Approximate Nearest Neighbors Protocol
Bob can partition his labeled samples into l=√{square root over (n)} clusters, each having l samples at most. There are several ways for Bob to do this. One way is to apply a k-means process to the data and determine l centroids. Then, Bob attaches at most l samples to each centroid. Alternatively, Bob can use a k-d tree to recursively partition the space until each node in the k-d tree contains l samples or less. Then, each node is represented by the centroid of all its samples. Partitioning the space to accelerate retrieval is mainly useful in low-dimensional spaces. In case the data are in high-dimensional spaces, geometric hashing methods are more appropriate.
Secure Approximate Nearest Neighbors Protocol
The secure approximation nearest neighbor protocol includes a number of stages. First, we determine the nearest centroid to the private query sample. Then, we determine private shares of the distances of the labeled samples associated with this centroid to the private query sample. Finally, we apply the secure-APX-nn protocol.
The key observation is that Bob generates his private share ahead of time, and for each centroid Bob generates Alices's private shares. When Alice and Bob agree on the nearest centroid, Alice obtains her private shares from Bob. Then, Alice and Bob apply the secure-k-nn protocol.
Formally, Bob has n labeled samples X1, . . . , Xn. Denote l=√{square root over (n)}, and let C1, . . . , Cl be the centroids determined by Bob. Let Xij represent the jth sample associated with centroid i. Then, the private share of Alice for the private query sample Xij is given by Aij=Xij−Bi.
Protocol 5—Secure-APX-nn Protocol
Given X1, . . . , Xn samples, we determine approximately, the k nearest neighbors of the private query sample X.
Input
Alice has the private query sample X, and Bob has a database of labeled samples X1, . . . , Xn, and a parameter k.
Output
Alice and Bob obtain private shares of approximate k nearest neighbors of the private query sample X.
The embodiments of the invention provide methods for privacy preserving k-nn classification. The methods enable one party to have its data classified by another party, securely. This is done by developing a secure nearest neighbor search protocol that is then used for several secure density estimation methods, including Parzen window classification as well as exact and approximate k-nn classification.
It is to be understood that various other adaptations and modifications may be made within the spirit and scope of the invention. Therefore, it is the object of the appended claims to cover all such variations and modifications as come within the true spirit and scope of the invention.
