Patent Grant
9355271
This disclosure is directed to the fields of cryptography and data security in general and, more particularly, to systems and methods for providing searchable symmetric encryption.
In modern networked computing systems, user data files are often stored in networked connected data server computing devices and accessed by remote client devices through a data network such as the Internet or another suitable network. For example, numerous network-connected data storage services, sometimes referred to as “cloud” storage services, provide network-connected data storage that client computing devices use to store data files. In some instances, a client stores data on a network storage system instead of keeping a copy of the file in a local data storage device such as a hard drive or solid-state memory device.
One disadvantage of online network services is that client data may be exposed to third parties, such as network intruders, who should not be permitted access to the client information. For example, a security vulnerability in the software of a network data storage server could enable an attacker to gain access to sensitive information in files that the client has stored on the server. As is known in the art, a client computing device preserves the privacy of data files that are transmitted to the data storage server by encrypting the data files prior to storing the data files on the network storage server. The client computing device uses one or more cryptographic keys to perform the encryption, and the users of the server computing device do not have access to the cryptographic keys. When the client uses an appropriate encryption method, an attacker or other party who gains unauthorized access to the encrypted files cannot produce the original contents of the files from the encrypted files in a practical manner.
While encryption enables a client computing device to maintain the privacy of data in encrypted files that are stored on a remote server, the encryption process also presents difficulty when the client attempts to search or otherwise identify the contents of the encrypted files. As described above, in some configurations the client does not store unencrypted copies of the data files in local storage both because the local data storage device capacity may be limited in comparison to the data storage server and because the server implements redundancy and backups to preserve the encrypted files from loss. Since most security models do not place complete trust in the server, the client cannot rely on the server to decrypt and search the files without divulging the contents of the files to potential attackers.
Existing techniques including Dynamic Symmetric Searchable Encryption (DSSE) enable clients to send search queries to a server to identify encrypted files that include key words used in a search query. In a DSSE scheme, the client generates and stores one or more symmetric cryptographic keys that are not shared with the server. The client uses one key or set of keys to generate a search database of search terms corresponding to the plain text contents of the encrypted files that are stored on the server. The server stores the search database and performs searches on the encrypted files in response to requests from the client. The structure of the search database and the requests from the client do not identify the search terms that are the subject of each search request. The client uses a different key or set of keys to perform the actual encryption of the files prior to sending the encrypted files to the server. In one embodiment the search terms include commonly used words in English or words in other languages that are included in the plain text versions of the encrypted data files. In other embodiments, search terms can take the form of searchable binary data segments that may be included in multimedia files such as recorded audio, photographic, or video data files. The server stores a search database that enables the server to identify files that contain a particular search term. In existing DSSE schemes, the client generates a search query for the server that does not divulge the search term to the server and the server uses the search database to identify encrypted files that include the search term without having to decrypt the encrypted files. The client optionally retrieves one or more of the encrypted files that include the search term to decrypt the encrypted files and perform additional processing without divulging the contents of the encrypted files to the server.
A DSSE scheme is defined mathematically with the following operations:
DSSE=(Gen, Enc, Dec, SrchToken, Search, Add, AddToken, DelToken, Delete) such that:
Some existing DSSE schemes operate with a chosen keyword attack 2 (CKA-2) level of security. A DSSE scheme that meets the criteria of CKA-2 security model has the following properties. The following definition of CKA-2 security is known to the art and is further documented in “Parallel and Dynamic Searchable Symmetric Encryption,” by Seny Kamara and Charalampos Papamanthou. Let D be a DSSE scheme. D meets the CKA-2 security model definition if D has the following properties against a stateful attacker A using a simulator S where 1 and
2 are functions that describe the leakage of information to A:
While existing DSSE schemes enable clients to search encrypted data without requiring divulging search terms or the contents of encrypted files to the server, existing DSSE schemes still have some drawbacks. In particular, existing DSSE schemes leak “size pattern” information, which is to say that an attacker can identify the overall number of search term to document pairs that are stored in the search database. Beyond drawbacks based on leakage of information, some existing DSSE schemes are also computationally intensive and require extensive data storage capacity to store the search databases. Consequently, systems and methods for performing DSSE that meet the CKA-2 security model criteria and further improve the privacy of client data beyond the requirements of CKA-2 while improving performance and search database storage overhead would be beneficial.
In one embodiment, a method of searching encrypted data has been developed. The method includes generating with a client computing device a search index identifier using a predetermined encryption process to generate an encrypted key using a first secret cryptographic key and a predetermined hash function to generate the search index identifier from the encrypted search term, generating with the client computing device a first single use cryptographic key with reference to a second secret cryptographic key, and a first counter value associated with the search index identifier, generating with the client computing device a second single use cryptographic key with reference to a second secret cryptographic key, and a second counter value associated with the search index identifier, transmitting with the client computing device the search index identifier, first single use cryptographic key, and second single use cryptographic key to a server computing device, identifying with the server computing device a first set of encrypted data in a search table with reference to the search index identifier, generating with the server computing device a set of decrypted data from the first set of encrypted data, the server computing device using the first single use cryptographic key to decrypt a first portion of the first set of encrypted data and the server computing device using the second single use cryptographic key to decrypt a second portion of the first set of encrypted data, identifying with the server at least one encrypted file stored in a memory associated with the server computing device with reference to the decrypted data, the at least one encrypted file containing an encrypted representation of the search term, transmitting with the server computing device a plurality of file identifiers corresponding to the identified encrypted files to the client computing device, generating with the server computing device a second set of encrypted data from the decrypted set of data from the search table and the second single use cryptographic key, and storing the second set of encrypted data in the search table in associated with the search index identifier to replace the first set of encrypted data in the search table.
In another embodiment, a method of updating an encrypted search database for an encrypted file has been developed. The method includes identifying with a client computing device a plurality of search terms in a file stored in a memory of the client computing device, generating with the client computing device a plurality of encrypted search terms from the plurality of search terms using a first cryptographic key, generating with the client computing device a plurality of search index identifiers corresponding to each search term in the plurality of encrypted search terms using a predetermined hash function, each search index identifier in the plurality of search index identifiers corresponding to a set of entries in an encrypted search table, generating with the client computing device a plain-text set of data corresponding to a plurality of entries for the file in the search table, the set of data including a first plurality of entries corresponding to the search index identifiers having a first value indicating that a corresponding search term is present in the file and a second plurality of entries having a second value indicating that a corresponding search term is not present in the file, generating with the client computing device an encrypted set of data from the plain-text set of data corresponding to entries for the file in the search table, encrypting with the client computing device contents of the file using a file encryption/decryption key stored only in a memory associated with the client computing device, encrypting with the client computing device a file identifier of the file, transmitting with the client computing device the encrypted set of data, the encrypted file identifier, and the encrypted file to a server computing device, updating with the server computing device the search table with the encrypted set of data in a memory associated with the server computing device, storing with the server computing device the encrypted file in the memory associated with the server computing device.
For a general understanding of the environment for the device disclosed herein as well as the details for the device, reference is made to the drawings. In the drawings, like reference numerals designate like elements.
As used herein, the terms “single use cryptographic key” or “single use key” are used interchangeably and refer to a cryptographic key that is generated in a “fresh” state and used to encrypt one or more sets of data and are used to decrypt the encrypted data only once before the single use cryptographic key is considered “stale”. As described in more detail below, the term “fresh” refers to a single use cryptographic key that a client computing device generates for the purposes of encrypting search index data that are associated with a single search term in a larger encrypted search table. The single use key remains fresh as long as the client retains the secrecy of the single use key. In particular, the single use key remains fresh as the client encrypts search table data associated with the search term for one or more files and does not transmit the single use key to a server computing device that stores the encrypted search table. To perform a search operation, the client transmits the single use key to the server and the server uses the single use key to decrypt the encrypted search table data to perform the search. The term “stale” refers to a single use cryptographic key once the server has received the single use cryptographic key from the client. The server uses the stale key for a re-encryption operation and then deletes the stale key from memory. After transmitting the single use key to the server, the client does not use the stale single use cryptographic key for any further encryption operations. Instead, the client computing device generates another fresh single use cryptographic key that is associated with the search term for additional encryption operations until the client performs the next search for the search term.
In the discussion below, reference is made to counters that are associated with search terms that a client computing device queries in a search database located in a remote server computing device and in association with files that the client encrypts and stores in the server. A counter refers to a numeric value that is initialized to a predetermined value (e.g. 0 or 1) and is subsequently changed (often by adding 1 to the counter value) when the client computing device or server computing device performs an encryption process. For example, as described in more detail below, the client computing generates a plurality of single use encryption keys using a secret cryptographic key, a numeric index associated with an encrypted search term, and a counter value associated with the encrypted search term. To generate new and different single use cryptographic keys for the same search term using a single secret key, the client increments the counter value and appends the incremented counter value to the numeric index associated with the search term. Consequently, the encryption key always encrypts a different set of data to generate the plurality of single use keys. A similar process occurs for encryption of a set of search index data that is associated with a file. When the client updates the file to add or delete one or more search terms, the client increments a counter value that is associated with the file. The client then uses a secret cryptographic key to encrypt each entry in the search index using a concatenated set of data including the value of the entry (e.g. 0 if the search term is not present in the file, 1 if the search term is present in the file), a numeric index associated with a hashed value of an encrypted version of the file identifier, and the counter value. When the client changes the file, the client also increments the counter to guarantee that each entry in the search index is re-encrypted using a different set of data. This prevents an attacker from comparing a previous version of the encrypted file index to a new version of the encrypted file index to identify the search terms that changed when the client updated the file.
As used herein, the term “random oracle” refers to a function H(x) that takes an input x and generates an output of a predetermined number of bits that appear to be random. The input x can have an arbitrary length of one or more bits. The random oracle H is a type of one-way or “trapdoor” function where the output cannot be used to reconstruct the original input x in a practical manner. The random oracle function H returns the same output when invoked for a given input value. In the examples below, a random oracle function H generates a single-bit output that is used for encryption and decryption of single-bit entries in a search table. However, other random oracle embodiments generate outputs with a larger number of bits. A practical embodiment of a random oracle function is a one-way hash function such as the SHA2 or SHA3 families of cryptographically secure hash functions. Digital processing devices including microprocessors and controllers implement the cryptographically secure hash functions and the other functions of the random oracle using stored program instructions and, in some embodiments, dedicated processing hardware the performs some or all of the functions of the random oracle. Cryptographically secure hash functions typically produce large outputs (e.g. 256 or 512 bit outputs). A single-bit random oracle, however, truncates the output to use only one bit, such as the most significant or least significant bit in the output of the hash function to produce single-bit randomized output.
As used herein, the term “search index identifier” refers to a numeric datum that is used to identify a particular set of data in a search table that corresponds to entries for a search term. For example, a numeric row number of an encrypted search table identifies a row of encrypted search table entries that each store an encrypted identifier that indicates if a particular encrypted file includes an encrypted representation of the search term that corresponds to the search index identifier. As described in more detail below, a server stores an encrypted search table and receives search requests from a client that only include the numeric search index identifier instead of the actual search term. The server identifies a set of encrypted search table data using the search index identifier, decrypts the set of search table data, and returns file identifiers and file contents of encrypted files that include the search term. The search table is a two-dimensional table that is also referenced with a file index identifier. As used herein, the term “file index identifier” refers a numeric datum that is used to identify a particular set of data in the search table that corresponds to one encrypted file. In the encrypted search table, the set of data corresponding to a file includes encrypted table entries that, after decryption, identify the presence or absence of a set of search terms in one particular file.
The client 104 includes a client processor 108 and a memory 112. The processor 108 is a microprocessor or other digital logic device that executes stored program instructions and the memory 112 includes both volatile data storage devices such as random access memory (RAM) and non-volatile data storage devices such as magnetic disks and solid state drives. Some embodiments of the client processor 108 include parallel execution hardware that incorporates multiple processing cores or other parallel processing components to perform file encryption and decryption, search term encryption, file update operations, and other operations that are implemented as part of a DSSE scheme concurrently. Examples of client computing devices include portable and desktop personal computers (PCs), smartphones, tablet computing devices, wearable computing devices, and any other suitable digital computing device.
In the client 104, the memory 112 stores a static hash table 114 of counter values for search terms 114 that are used in searches and another static hash table 116 of counter values for encrypted files. A counter value in table 114 corresponds to a search term that the client 104 uses as a subject of a search in the encrypted files 156 that are stored in the server memory 152. The encrypted files 156 include encrypted representations of at least some search terms that the client requests from the server 144, although the server 144 is unable to extract plain text search terms from the encrypted files 156. The client processor 108 increments the counter associated with each search term after performing a search for the corresponding search term in the server 144. The client processor 108 increments the counter associated with a file after performing an update that adds or removes at least one search term from the file before encrypting and transmitting the file to the server 144. The client memory 112 stores secret key data 118 including the keys k1, k2, and k3. In the embodiment of
During operation, the client computing device 104 encrypts one or more plain text files 120 and generates encrypted search term indices for the files using the key k2. The client 104 transmits the encrypted version of the plain text file 120 and the search term indices to the server 144. The client 104 also identifies encrypted files that match specific search terms on the server 144, and retrieves the encrypted files. The client 104 decrypts the retrieved files, and optionally updates the files to add or delete search terms. The client 104 then re-encrypts the file, generates an updated set of encrypted search term indices, and transmits the updated encrypted file and updated encrypted search term indices to the server 144.
The server 144 includes a server processor 148 and a memory 152. The processor 148 in the server 144 is a microprocessor or other digital logic device that executes stored program instructions to perform searches and file storage and retrieval services for the client 104. While not a requirement, in some embodiments the server processor 148 has greater computational power than the client processor 108. Some embodiments of the server processor 148 include parallel execution hardware that incorporates multiple processing cores or other parallel processing components to perform searches and other operations that are implemented as part of a DSSE scheme concurrently. The memory 152 in the server 144 includes both volatile data storage devices such as random access memory (RAM) and non-volatile data storage devices such as magnetic disks and solid state drives. While not a requirement, in some embodiments the server memory 152 has a larger capacity than the client memory 112 to enable the server memory 152 to store a large number of encrypted files. While
The server memory 152 stores an encrypted search table 154, a set of encrypted files 156, and a copy of the file counter hash table 116 that is also stored in the memory 112 of the client 104. The encrypted search table is a two-dimensional table with one dimension corresponding to individual search terms in the encrypted files and another dimension including entries that correspond to individual files in encrypted files 156. In the illustrative embodiments described herein, each row of the table 154 includes encrypted entries for a single search term that is either present or absent from a particular file, and each column of the table 154 includes entries that correspond to different search terms that are either present or absent from a single file. Search queries for different search terms address the table 154 through numeric search indices and the server 144 cannot identify the underlying search term based on only the search index. As described in more detail below, the client 104 converts a search term to an appropriate numeric index for the table 154 using an encryption process that prevents the server 144 from identifying the contents of the search term from the search index number. The server 144 uses the search index value to select a row of encrypted search data from the table 154. The client 104 also generates a two single use cryptographic keys that the server 144 uses to decrypt the contents of the selected row of the table 154 and re-encrypt the contents of the selected row after identifying files that include the search term associated with the row.
In the system 100, the client 104 communicates with the server 144 through a network 180. Both the client 108 and server 144 include network communication devices, such as wired network devices (e.g. Ethernet or other suitable wired network interface) and wireless network devices (e.g. Bluetooth or IEEE 802.11 wireless LAN and 3G or 4G wireless WAN). In the discussion below, the client 104 and server 144 are assumed to communicate using authenticated and encrypted communication processes that are known to the art and are not described in further detail herein. Thus, an eavesdropping computing device that monitors traffic through the network 180 cannot determine the contents of communications between the client 104 and server 144. An “attacker” refers to a computing device or entity that has access to the server 144 and the ability to read at least portions of the data stored in the server memory 152 in a manner that is not approved by the client 104. The attacker has a goal of extracting information about the N encrypted client files 156 to reduce or eliminate the privacy of the content of these files. The attacker also observes the contents of the encrypted search table 154 and data that are generated during operations in the server 144 to generate information about the encrypted files 156. The attacker is also presumed to have the ability to monitor network communications at the server 144 to circumvent the encryption of communication messages between the client 104 and the server 144. While the attacker can observe communications from the client 104, the attacker does not have direct access to the contents of the client memory 112.
During operation, the client processor 108 and the server processor 148 executed stored program instructions for a predetermined hash function to generate the numeric search term index values for the hash tables 114 and 154 (row index), and the file index values of the hash tables 116 and 154 (column index). In the system 100, the predetermined hash function is typically not a cryptographically secure hash function, because the data being hashed are limited to encrypted search terms and encrypted file identifiers that are already encrypted. Furthermore, an attacker on the server 144 never sees the encrypted search term data because the client 104 does not transmit the encrypted search term data to the server 144. While the attacker on the server 144 can access the encrypted file identifiers and identify the corresponding columns in the tables 116 and 154 for each encrypted file, this information does little to help the attacker identify the contents of the encrypted files or search terms. Consequently, the client 104 and server 144 use any suitable hash function that produces no collisions between different encrypted search terms or encrypted file identifiers using “perfect” hash functions, or employs a hash function with a low probability of collision coupled with secondary hashing or chain hashing to handle collisions. In some embodiments, the client 104 and server 144 use multiple predetermined hash functions or families of hash functions during operation, and the term “predetermined hash function” refers to any suitable combination of hash functions that are used by both the client 104 and the server 144 in addition to using a single predetermined hash function. Of course, a cryptographically secure hash function could be used to generate the index values for the tables 114, 116, and 154, but such a function is not a requirement for the operation of the system 100.
In the embodiment of
In
During operation, the client 104 and server 144 apply the predetermined hash function to the encrypted file name to generate an index number for the file hash table 116. Each entry in the file hash table 116 includes a counter value (cntj) that is associated with the file at the numeric index j. The client 104 initializes the counter value to a predetermined number (e.g. 0 or 1) when the file is first encrypted, indexed, and stored in the server memory 152. The client increments the counter value in the file counter hash table 116 when the client 104 updates the file and stores an updated version of the file to the server 144. The client uses the file counter value during the encryption process for the search table entries that are stored in the search table 154, and all of the search entries are re-encrypted using a new counter value whenever the file is updated. The client 104 changes the counter value and uses the updated counter value during encryption to ensure that all entries corresponding to different search terms are re-encrypted and have the potential to change during the re-encryption process. Thus, an attacker who monitors the encrypted search table entries for a file cannot identify the particular search terms that have been added or removed from the file during a file update process. In the system 100, the server 144 also uses the counters that are associated with each file during the decryption and re-encryption processing for row data using the selected counter values for the encrypted files 156.
In
As noted above, in a DSSE process the client generates a plain-text search table δ. In the system 100, the client 104 does not store the plain-text search table δ in the memory 112, and the server 144 never receives the plain-text search table δ. As described in more detail below, in some configurations the client only stores the plain-text search table δ for a plurality of n files in an ephemeral manner prior to encryption and transmission of the encrypted table I to the server 144. In another configuration, the client 104 never constructs a single plain-text table δ for all of the n files. Instead, the client 104 operates on a single file to extract search terms from the file, generate a one dimensional (vector) plain-text search table for the individual file, encrypt the search table, and update a larger two-dimensional encrypted search table on the server 144 with the encrypted search data and the encrypted file. The client 104 optionally performs the same operation on individual files or smaller groups of files to form the encrypted search table 154 and encrypted files archive 156 in an incremental manner.
In some embodiments, the search term hash table 114, file counter hash table 116, and encrypted search table 154 are implemented as sparse tables using techniques for storage and retrieval of data in sparse tables that are known to the art. For example, in some embodiments the search terms are a static set of terms from an existing language, such as English, with a predetermined number of words (e.g. approximately 1 million words in English). The client 104 encrypts the search terms using the key K2 and then uses a predetermined hash function to convert the encrypted search terms to numeric index values in the search term hash table 114. However, in many instances the numeric space of the hash function is much larger than the number of non-trivial entries that populate the hash table. For example, even a comparatively small 32-bit hash space has 232 entries, which means that a dictionary of 1 million search terms only fills approximately 1 entry out of every 4,200 entries in the hash space. Consequently, the search term table 114 often includes a large number of trivial (unfilled) entries between entries that correspond to the numeric hashed values of the encrypted search terms. For similar reasons, the file counter hash table 116 and the encrypted search table 154 may be sparse tables. The tables illustrated in
In process 300, the client computing device 104 generates a search term query for the server computing device 144. The client 104 selects a search term to use in the query (block 304). In some embodiments, the client processor 108 receives a search term from a user through an input device such as a keyboard, touchscreen interface, speech input device, or other suitable input device. The search term is, for example, a word or other predetermined set of data that corresponds to an entry in the search table 154 that is stored in the server memory 152. In some embodiments, the search terms are contained in a predetermined dictionary that provides a predetermined number of potential search terms. Some of the search terms may not be contained in any of the files that are being searched, which may make storage of the search table 154 somewhat inefficient in comparison to a search table that only includes rows for search terms that are included in at least one file. However, the fixed number of search terms and corresponding fixed number of rows in the search table 154 prevents an attacker from identifying if a file update has added a new search term that was not present in the search table 154 prior to the file update or if a search term has been removed from all of the encrypted files 156. The process 300 is compatible with either a fixed number of search terms or a variable number of search terms in the encrypted search table 154.
Process 300 continues as the client 104 generates an encrypted version of the search term using the encryption key k2 and a predetermined encryption function (block 308). The encryption of the search term wi is set forth in the following equation: sw
During process 300, the client 104 generates or retrieves a stale single use key
Process 300 continues as the client 104 sends a search token τw that includes the encrypted search table index i, first single use key
Process 300 continues as the server 144 receives the search token τw and decrypts the encrypted entries corresponding to the search index identifier i using the first single use key
Once the server 144 has generated the decrypted search table data I′[*] for the entire row that is reference by search index i, the server 144 identifies the encrypted files (c) that include the encrypted representation of the search term that corresponds to the index i (block 332). In the embodiment of the system 100, the decrypted plain-text search table entries have a value of 1 if a file at file index identifier j in the search table includes the search term or a 0 if the file does not include the search term. The server 144 uses the same hash function as the client 104 to associate the encrypted file identifiers for each of the encrypted files with the file index identifiers j. In some embodiments, the server 144 caches a reverse lookup information that includes the file identifier, such as a file name of each encrypted file, in the hash table 116 in association with the file index identifier j of each file.
Process 300 continues as the server 144 re-encrypts the decrypted search results and stores the re-encrypted search result data in the encrypted table 154 to replace the previous contents of the row i (block 336). The server processor 148 uses the key rj to re-encrypt each entry in the row i in conjunction with the file index and file counter for each of the j entries in the row. The server processor 148 uses the random oracle to generate an encryption bit and then performs an exclusive-or operation to re-encrypt the previously decrypted table entry value For example, a re-encrypted element I″[j] is generated from the plain-text bit I′[j] according to the following equation: I″[j]←I′[j]⊕H(ri∥j∥cntj). The server 144 also resets the state bits st of all the entries in the row to 0 during the re-encryption process since each entry in the row i is now encrypted using only the key ri. If the client updates one or more files prior to the next search that is performed on the row i, then the server 144 updates the state bits of the modified entries to 1 to indicate that those entries should be decrypted using a fresh single use key that the client 104 will transmit to the server during a subsequent search operation.
A special case for the processing described above with regards to blocks 316-336 occurs when the client 104 has never performed a search for the search term at index i in the encrypted search table 154 since the initial generation of the encrypted table 154. The client identifies if there have been no previous search operations in response to the counter value in the hash table 114 for search index i being set to a predetermined initial value (e.g. 0 or 1). When no previous search has occurred, the original single use key ri for row i in the search table 154 is fresh. Any updates that have occurred to row i the table 154 have used only the fresh key ri for encryption. Consequently, during the first search operation, the client only generates or retrieves the single key ri and the server 144 decrypts all entries in the row i using only the single use key ri. The server 144 subsequently re-encrypts using the same key ri. After the server 144 receives the single use key ri, the single use key ri is now stale(
As described above, during the process 300 the server 144 performs decryption and re-encryption of entries in a row i of the search table 154 on an individual basis. That is to say, the server process 148 decrypts each entry along the row i using the stale key
During process 300, the server 144 transmits either the full contents and file identifiers or only the file identifiers of the encrypted files (c) 156 that the server 144 has identified from the decrypted search table data 154 to the client (block 340). When the server 144 transmits the full contents of the files, the client 104 decrypts the encrypted file contents using the secret symmetric key k1 and decrypts the file identifiers using the secret symmetric key k2. In an embodiment where the client 104 only receives the file identifier information in the initial search, the client 104 optionally requests one or more of the encrypted files and the server 144 transmits the requested encrypted files to the client 104 for further decryption and processing (block 344).
During operation of the system 100, the search operation of process 300 reveals or “leaks” some information about the encrypted search table 154 and the encrypted files 156 to potential attackers. An attacker that has the ability to monitor the activities of the server during process 300 can learn the decrypted values of the entries in a single row i of the table 154 when the client searches for the key word that is associated with the search index i. Once again, the attacker does not learn the actual plain-text contents of the search term and does not learn the plain-text contents of any of the encrypted files c. However, the attacker does learn that a particular set of encrypted files includes a search term that corresponds to the search index identifier i. Consequently, the re-encryption process does not prevent the attacker from identifying the plain-text contents of the row i after the plain-text information has leaked. However, the re-encryption process is still useful to prevent transient attackers who only gain access to the server 144 after process 300 is performed. Additionally, as described below in
The embodiment of the system 100 and the process 300 described above describes individually encrypted entries in the encrypted search table 154. During a row decryption process, the server 144 decrypts and re-encrypts each row entry individually using the first single-use key
In
In the embodiment of
During operation with the block cipher embodiment of
The client 104 and server 144 perform the process 300 using the block encryption/decryption embodiment of
In addition to performing search operations, the client 104 can store an encrypted file in the server 144 and update the encrypted search table 154.
During process 400, the client computing device 104 generates a new file or updates a plain-text copy of a file that is stored in an encrypted form on the server computing device 144 (block 404). In
Process 400 continues as the client 104 generates a list of search terms in the file after generating the new file or updating an existing file (block 408). In the system 100, the client processor 108 executes a text extraction program that performs parsing, tokenization, word-stemming, and other text processing techniques that are known to the art to generate the list of search terms that are present in the plain text file 120. Some embodiments of the client processor 108 that incorporate parallel hardware to perform the search term extraction and identification process.
As described above, in some embodiments the system 100 operates with a fixed set of search terms (e.g. words in the English language or other language). The search term extraction process optionally includes identifying search terms present in file metadata, including search terms that are present in the plain-text file name for the file. The client does not includes non-standard terms in the file 120 or other data such as numbers in the generated list of search terms in this embodiment. In another embodiment, the system 100 expands the encrypted search table 154 when a new or updated file includes a new search term that has not been included in other encrypted files in the system 100. Expanding the encrypted search table 154 for new search terms enables more flexible searching, but an attacker can identify when a new search term is added and identify that the new search term is included in only one new file when the search term is initially added to the table 154. Process 400 can be used in conjunction with search table embodiments that are either fixed-size or that can expand to add new search terms.
Process 400 continues as the client encrypts the list of search terms in the file and hashes the encrypted key words to generate search index identifiers of the search terms in the file (block 412). For example, for a search term w the client 104 uses the secret symmetric key k2 to generate an encrypted search term sw. The client 104 then applies the hash function to the encrypted search term sw to generate the numeric search index identifier i that corresponds to the row i in the encrypted search table 154 in the server memory 152. The encryption function to generate the encrypted search term sw ensures that an attacker on the server 144 cannot identify the search term that corresponds to index i by simply applying the hash function, which is not a secret, to plain-text search terms. Thus, search term index identifiers in the encrypted table 154 are tied to the secret encryption key k2, and two different client computing devices that use different keys generate different sets of search index identifiers for the same search terms. Some embodiments of the client processor 108 that incorporate parallel hardware to perform the encryption and hashing processes for multiple individual search terms in parallel to reduce the time required to complete the update process 400.
During process 400, the client 104 generates a plain-text search table δ for the file based on the search index identifiers that are generated for the search terms in the file (block 416). The search table δ has a similar structure to the encrypted search table (I) 154 that is stored in the server memory 152, but the search table δ for a single file only includes a single linear arrangement of entries (e.g. a single column), and the contents of the entries in the search table δ are not encrypted. The client 104 assigns each entry in the search table δ that corresponds to one of the identified search term index numbers i a value of 1 to indicate that the file contains the search term and assigns a value of 0 to the remaining entries in the table δ.
Process 400 continues as the client 104 encrypts the file identifier that is associated with the file and hashes the encrypted file identifier to generate the numeric file index identifier for the file (block 420). In the system 100, the client 104 uses the key k2 to encrypt the plain-text file identifier, which is typically the filename used to address the file in filesystems that are well-known to the art. The client 104 then applies the predetermined hash function to the encrypted file identifier to generate the numeric file index identifier j for the file. The generation process of the file index identifier j is similar to the generation process of the search index identifiers i, but the file index identifier j is used to address the encrypted search entries for the file in a column j of the encrypted search table 154 instead of selecting entries for a particular search term in a row i of the encrypted search table 154.
Process 400 continues as the 104 client initializes or increments the state counter cntj that is associated with the file (block 424). If the file is a new file, the client 104 initializes the counter to a predetermined value (e.g. 0 or 1) and adds a new entry for cntj to the file counter hash table 116 at index j. For an update to an existing files, the client 104 increments the state counter cntj for the file at index j in the file index hash table 116. As described below, the incremented counter value is used during the encryption of the entries in the plain-text search table δ. The incremented counter value ensures that each entry in the search table δ is encrypted using a potentially different output from the random oracle during each file update operation. Consequently, even if an entry in the plain-text search table δ does not change during a file update process, the value of the entry in the updated encrypted search table might change (with a 50% probability) from the encrypted entry in a previous version of the encrypted search table. Thus, an attacker that has access to an earlier version of the encrypted search index j for the file and the updated version of the search index cannot identify which search terms were added or removed from the file because any of the entries in the updated encrypted file index may change in an unpredictable manner.
The client 104 uses the file index identifier, file state counter, and a plurality of the fresh single use encryption keys ri corresponding to each of the i file index identifiers to encrypt the plain-text search table δ and generate an encrypted search table Ij for the file j (block 428). For example, to encrypt a single entry δ[i], the client applies the random oracle function H to a combination of the fresh row key ri, file index identifier j, and the counter cntj. The client 104 then performs an exclusive-or operation with the plain-text file entry and the output of the random oracle to generate the encrypted search index entry Ij[i]. The encryption operation for a single entry Ij[i] is also defined with the following mathematical operation: Ij[i]←δ[i]⊕H(ri∥j∥cntj). The client repeats the encryption process for each of the i search index entries.
As described above, the client 104 generates single use cryptographic keys for each search index identifier i, and optionally caches the single use keys in the search index hash table 114. During process 400, the client 104 uses only the fresh cryptographic key corresponding to the table entry at search index i to encrypt each entry in the encrypted search table Ij. If necessary, the client 104 increments the search index counter in the hash table 114 and performs the single use key generation process using the key k3 to generate a fresh single use key if one is required. The client 104 generates the encrypted search table Ij with a similar structure as a single column j in the encrypted table 154, although in some embodiments the client 104 omits the state bit field st since the server 144 can set the state bit fields. Some embodiments of the client processor 108 that incorporate parallel hardware perform the encryption of the individual search table entries with the corresponding single use keys in parallel to reduce the time required to complete the update operation 400.
During the process 400, the client 104 encrypts the contents of the plain text file (block 432). In the system 100, the client 104 uses the secret symmetric key k1 to encrypt the plain text file data 120 to generate an encrypted file c. The client 104 uses, for example, a block cipher encryption scheme such as AES or another suitable symmetric encryption scheme to generate the encrypted file c. The encryption of the file data optionally occurs before, after, or concurrently with the generation of the encrypted search table Ij for the file as described above with reference to blocks 408-428 in the process 400.
Process 400 continues as the client 104 transmits the encrypted search table entries Ij, the encrypted file identifier, and the encrypted contents of the file c to the server 144 (block 436). In the system 100 the client 104 and server 144 establish an authenticated and optionally encrypted channel through the network 180 prior to the transmission. In the embodiment of
The process 400 can also be performed using the block cipher embodiment that is discussed above in
During the search term encryption of the process 400 using the block encryption embodiment of
The embodiments described above in
Additional definitions of terms used herein and a mathematical proof of the CKA-2 properties of the system and methods described above are set forth below.
Operators ∥ and |x| denote the concatenation operation and the bit length of variable x, respectively. xS denotes that variable x is randomly and uniformly selected from set S. For any integer l, (x0, . . . , xl)
S means (x0
S, . . . , xl
S). |S| denotes the cardinality of set S. {xi}i=0l denotes (x0, . . . , xl). The term {0, 1}* denotes the set of binary strings of any finite length. └x┘ denotes the floor value of x and ┌x┐ denotes the ceiling value of x.
q1, . . . , qn
denotes set of items qi for i=1, . . . , n. Given a bit a, ā means the complement of a. Variable κ is an integer and it is used to denote the security parameter. log x means log2 x.
=(Gen, Enc, Dec) is IND-CPA secure symmetric key encryption scheme, which includes a secret key generation process, an encryption process, and a decryption process. k1
.Gen(1κ) is a Probabilistic Polynomial Time (PPT) key generation process that accepts a security parameter κ and returns a secret key c←
.Enck
.Deck
A Pseudo Random Function (PRF) is a polynomial-time computable function, which is indistinguishable from a true random function by any PPT attacker. F:{0, 1}κ×{0, 1}*→{0, 1}κ is a keyed PRF denoted as τ←Fk{0, 1}κ and an input x, and returns a token τ. G:{0, 1}κ×{0, 1}*→{0, 1}κ is a keyed PRF denoted as r←Gk
H:{0, 1}|x|←{0, 1} is a Random Oracle (RO), which takes an input x and returns a bit as output.
fid and w denote a file with unique identifier id and a unique (key)word that exists in a file, respectively. A search term w is of length polynomial in κ, and a file fid may contain any such search term (i.e., the search term universe is not fixed). For practical purposes, n and m, denote the maximum number of files and search terms to be processed by application, respectively. f=(fid
Index (also called database in the literature) δ is a n×n matrix, where δ[i, j]ε{0, 1} for i=1, . . . , m and j=1, . . . , n. Initially, all elements of δ are set to 0. Given a matrix δ, δ[*, j] and δ[i, *] mean accessing all elements in j'th column and i'th row, respectively. δ[i, *]T denotes the transpose of i'th row of δ. I is a n×n matrix, where I[i, j]ε{0, 1}2. I[i, j].v stores δ[i, j] in encrypted form depending on state and counter information. I[i, j].st stores a bit indicating the state of I[i, j].v. Initially, all elements of I are set to 0. I[i, j].st is set to 1 whenever its corresponding fj is updated, and it is set to 0 whenever its corresponding search term wi is searched. The term I[i, j] without any additional elements denotes I[i, j].v for brevity, and the state bit I[i, j].st is referenced expressly. The encrypted index is denoted by γ and the encrypted matrix corresponds to the encrypted matrix I and a hash table.
Each file fid and search term wpair are mapped to a unique set of indices (i, j) in matrices (δ, I). Static hash tables uniquely associate each file and search term to its corresponding row and column index, respectively. Static hash tables also enable to access the index information in (average) O(1) time. Tf is a static hash table whose key-value pair is {sfj, cntj
}, where sf
i,
}, where token sw
The embodiments presented above are shown to be secure according to the following theorem:
If Enc is IND-CPA secure, (F, G) are PRFs and H is a RO then the DSSE scheme is (1,
2)-secure in ROM to implement CKA-2 security.
Proof:
A simulator S interacts with an attacker A in an execution of an IdealA,S(κ) experiment.
In this experiment, S maintains lists R,
K and
H to keep track the query results, states and history information, initially all lists empty.
R is a list of key-value pairs and is used to keep track RO(.) queries. We denote value←
R(key) and ⊥←
R(key) if key does not exist in
R.
K is used to keep track random values generated during the simulation and it follows the same notation that of
R.
H is used to keep track search and update queries, S's replies to those queries and their leakage output from (
1,
2).
S executes the simulation as follows:
I. Handle RO(.) Queries:
Function b←RO(x) takes an input x and returns a bit b as output. Given input x, if ⊥=R(x) then set b
{0, 1}, insert (x, b) into
R and return b as the output. Else, return b←
R(x) as the output.
II. Simulate (γ, c):
Given (m, n, id1, . . . , idn′
,
|cid
)←
1(δ, f), S simulates (γ, c) as follows:
Correctness and Indistinguishability:
c has the correct size and distribution, since 1 leaks
|cid
and Enc is a IND-CPA secure scheme, respectively. I and Tf have the correct size since
1 leaks (m, n). Each I[i, j] for j=1, . . . , n and i=1, . . . , m has random uniform distribution as required, since RO(.) is invoked with a separate random number zi, j. Tf has the correct distribution, since each Sf
Hence, S also does not abort.
Simulation: Assume that S receives a search query w on time t. S is given (P(δ, Query, t), Δ(δ, f, wi, t))←2(δ, f, w, t). S adds these information to
H. S then simulates τw and updates lists (
R,
K) as follows:
Given any Δ(δ, f, wi, t), S simulates the output of RO(.) such that τw always produces the correct search result for idw←Search (τw, γ). S needs to simulate the output of RO(.) for two conditions (as in III—Step 6): (i) The first search of wi (i.e., τw=(i, ri)), since S did not know δ during the simulation of (γ, c). (ii) If any file fidR (as in III—Step 6). In other cases, S just invokes RO(.) with (ri∥j∥stj), which consistently returns previously inserted bit from
R (as in III—Step 8).
During the first search on wi, each RO(.) output V[i, j]=RO(ri∥j∥stj) has the correct distribution, since I[i, *]εγ has random uniform distribution (II—Correctness and Indistinguishability argument). Let J=(j1, . . . , jl) be the indexes of files containing wi, which are updated after the last search on wi. If wi is searched then each RO(.) output V[i, j]=RO(ri∥j∥stj) has the correct distribution, since τf←(I′, j) for indexes jεJ has random uniform distribution (IV—Correctness and Indistinguishability argument). Given that S's τw always produces correct idw for given Δ(δ, f, wi, t), and relevant values and RO(.) outputs have the correct distribution as shown, A does not abort during the simulation due to S's search token. The probability that A queries RO(.) on any (ri∥j∥stj) before him queries S on τw is negligible
and therefore S does not abort due to A's search query.
IV. Simulate (τf, τf′):
Assume that S receives an update request Query=(Add, |cid
, Delete) at time t. S simulates update tokens (τf, τf′) as follows:
Correctness and Indistinguishability:
Given any (τf, τf′) for a file fid
It remains to show that (τf, τf′) have the correct probability distribution. In real algorithm, std of file fid
and therefore S also does not abort due to A's update query.
V. Final Indistinguishability Argument:
(sw
|Pr[RealA(κ)=1]−Pr[IdealA,S(κ)=1]|≦neg(κ)
It will be appreciated that variants of the above-described and other features and functions, or alternatives thereof, may be desirably combined into many other different systems, applications or methods. Various presently unforeseen or unanticipated alternatives, modifications, variations or improvements may be subsequently made by those skilled in the art that are also intended to be encompassed by the following claims.
This application claims priority to U.S. Provisional Application No. 62/026,201, which is entitled “Method For Dynamic, Non-interactive And Parallelizable Searchable Symmetric Encryption With Small Leakage And Provable Security,” and was filed on Jul. 18, 2014, the entire contents of which are hereby incorporated by reference herein. This application claims further priority to U.S. Provisional Application No. 61/892,641, which is entitled “Method For Dynamic, Non-Interactive And Parallelizable Searchable Symmetric Encryption With Secure And Efficient Updates,” and was filed on Oct. 18, 2013, the entire contents of which are hereby incorporated by reference herein.
| Number | Name | Date | Kind |
|---|---|---|---|
| 7484092 | Rajasekaran | Jan 2009 | B2 |
| 8533489 | Roeder | Sep 2013 | B2 |
| 8874930 | Chase | Oct 2014 | B2 |
| 20090300351 | Lei | Dec 2009 | A1 |
| 20100211782 | Auradkar et al. | Aug 2010 | A1 |
| 20110119481 | Auradkar et al. | May 2011 | A1 |
| 20110138190 | Chase et al. | Jun 2011 | A1 |
| 20130046974 | Kamara | Feb 2013 | A1 |
| Entry |
|---|
| Lipmaa, Helger et al., “Comments to NIST concerning AES Modes of Operations: CTR-Mode Encryption,” Sep. 2000 (4 pages). |
| Kamara, Seny et al., “Parallel and Dynamic Searchable Symmetric Encryption,” Financial Cryptography and Data Security, Okinawa, Japan, Apr. 2013 (15 pages). |
| Song, Dawn Xiaodong et al., “Practical Techniques for Searches on Encrypted Data,” IEEE Symposium on Security and Privacy, 2000 (12 pages). |
| Cash, David et al., “The Locality of Searchable Symmetric Encryption,” Apr. 30, 2014 (23 pages). |
| Cash, David et al., “Dynamic Searchable Encryption in Very-Large Databases: Data Structures and Implementation,” NDSS Symposium 2014, Feb. 22, 2014 (16 pages). |
| Stefanov, Emil et al. “Practical Dynamic Searchable Encryption with Small Leakage,” NDSS Symposium 2014, Feb. 22, 2014 (15 pages). |
| Liesdonk, Peter Van et al., “Computationally Efficient Searchable Symmetric Encryption,” SDM, 2010 (14 pages). |
| Pappas, Vasilis et al., “Blind Seer: A Scalable Private DDMS,” SP '14 Proceedings of the 2014 IEEE Symposium on Security and Privacy, May 18, 2014 (16 pages). |
| Sedghi, Saeed et al., “Adaptively Secure Computationally Efficient Searchable Symmetric Encryption,” Centre for Telematics and Information Technology, University of Twente, 2009 (17 pages). |
| Goh, Eu-Jin, “How to Search on Encrypted Data,” Stanford University, Nov. 2003 (108 pages). |
| Kamara, Seny et al. “Dynamic Searchable Symmetric Encryption,” Proceedings of the 2012 ACM conference on Computer and communications security, 2012 (24 pages). |
| Goh, Eu-Jin, “Secure Indexes,” Cryptology ePrint Archive, Oct. 7, 2003 (18 pages). |
| Curtmola, Reza et al., “Searchable Symmetric Encryption: Improved Definitions and Efficient Constructions,” 13th ACM Conference on Computer and Communications Security, 2006 (33 pages). |
| Number | Date | Country | |
|---|---|---|---|
| 20150143112 A1 | May 2015 | US |
| Number | Date | Country | |
|---|---|---|---|
| 62026201 | Jul 2014 | US | |
| 61892641 | Oct 2013 | US |
