Patent Application
20080267403
Not Applicable
Not Applicable
Not Applicable
This invention relates generally to photographic, video or audio recording, generally for surveillance or security concerns. Surveillance is becoming more and more common all over the world. People are forever under the watchful ‘eye’ of the camera even as they go through their day-to-day activities. CCTV, increasingly with audio recording, is widely used for surveillance in banks, parking lots, shopping malls, airports, and other public places. Electronic audio recording (wiretapping) is also a growing concern. In these applications there is tension and tradeoff between the privacy of those being recorded and the underlying goals of security.
There are many privacy issues in surveillance. While some “invasion” is unintentional, or even just potential, the personnel who are in-charge of scanning these video images are often either ignorant about their job or tend to misuse their powers, for example, engaging in voyeurism. While cameras in airports, school bathrooms or eldercare facility bathrooms might improve security or safety, the potential abuses prohibit their use.
In the prior art, in order to address privacy, techniques such as privacy masking or blurring have been proposed. In the prior art, either an opaque mask, external to the camera, is applied, or after capturing the data, the regions of concern are irrevocably transformed to protect the privacy. While effective for applications like videotaping for television, these techniques have less implication for privacy in surveillance. With these techniques, the portion of the video/audio information including “privacy sensitive” data is modified but the transforms render the resulting data significantly less useful for security purposes.
In other prior art, privacy is addressed by encrypting data. Generally the person who is concerned about privacy, protects the data by encrypting it. Encrypted phone or radio communication is a well established example of prior art. In general this protects privacy because it inhibits the ability of others to access the data, i.e. it inhibits surveillance.
U.S. Pat. No. 6,067,399, issued to Berger on May 23, 2000 titled “Privacy mode for acquisition cameras and camcorders”, teaches a method of privacy enhancement wherein the system detects regions of skin in images, and modifies the corresponding pixels to obscure the subjects skin areas or face, either by direct pixel manipulation or graphics overlay. It teaches of protecting the privacy by ensuring that the identity cannot be recovered even from the original source data. It also teaches of distorting the audio channel to protect identity. European Patent EP 1 081 955 A2, issued to Koji et al. on Jul. 3, 2001 titled “Monitor Camera system and method of displaying picture from monitor thereof”, teaches of another method to determine and manipulate a “privacy region” which can obscure parts of an image as seen on camera. Again the distortions are non-invertible destruction of the data that might violate privacy.
US Patent 20050270371 A1, issued to Seblak on Dec. 12, 2005 titled “Transformable privacy mask for video camera images”, teaches of an adaptive pixel-wise obscuration approach to protect privacy.
US Patent 20060206911 A1, issued to Kim et. al. on Sep. 14, 2006, titled “Security camera employing privacy protection method”, teaches of an approach whereby a privacy area processor in the camera reduces the resolution, i.e. blurs, regions of the image to protect privacy.
An important aspect of all the aforementioned prior art is that the resulting audio/video data are protected in such a manner that the resulting data can be played/viewed without modification to existing display hardware/software. Unfortunately, they also cannot recover the original data.
US Patent 20030231769 A1, issued to Bolle et al, Titled “Application independent system, method, and architecture for privacy protection, enhancement, control, and accountability in imaging service systems” teaches of transforming the data of privacy interest by a range of techniques (destruction, modulation, overlay with graphical icon). The 20030231769 A1 patent teaches that “Some extracted information in video analysis stage is permanently obscured in the transformed methods”. It separates the descriptive information to be protected into various “tracks”, which can be separately encrypted with a range of keys. It requires an authorizer that provides authorization information with the image, the descriptive information in the transformed state is capable of being decoded only if one or more authorization inputs satisfy the authorization criteria. This method teaches of separating the data to be protected and encrypting some of it, but requires an added authorizer and specialized display/decoding components.
Partial encryption has been applied to protection of video data, where the goals are to reduce computational cost by encrypting only part of a data stream but selecting that portion so as to provide overall protection. U.S. Pat. No. 5,805,700 teaches selectively encrypting compressed video data based on policy. Specifically it teaches how to selectively encrypt the basic transfer units (BTU), start code of a GOP (group of pictures) or an I- P- B-frame in a MPEG-formatted video to achieve video image degradation with substantially less processing needed than using encryption of the full data stream. The '700 patent is based on structures of the MPEG-format and shows how, if key items are encrypted, the video cannot be effectively recovered. It does not address region-based encryption or any type of privacy protection. The objectives of using partial encryption in the '700 patent is to make as much of the data useless with minimal effort, not to leave the majority of data useful for security purposes.
U.S. Pat. No. 7,167,560 issued to H. H Yu, Titled “Partial encryption of stream-formatted media”, address partial encryption of streaming data, building on the recognition that the very same qualities of streaming media data that makes it useful, also make the data especially suited to a type of encryption that represents a significantly reduced computational load. Where the encryption-caused disruption is slight, the recipient will only be aware of a slight degradation in the quality of the media. But where the encryption is more significant, there comes a degree of disruption at which the media is rendered substantially imperceptible or of such low quality as to be substantially unsuitable to the recipient. Further, the degree of disruption at which the media becomes substantially imperceptible or substantially unsuitable to the recipient corresponds to partial encryption. The '560 patent teaches an approach such that more important information data layers are encrypted first and more securely while less important data layers are encrypted second and less securely, etc., thereby achieving scalability; multi-dimensional and multi-layered encryption capability to accommodate different application needs and/or to make different quality levels of preview available to different types of users (e.g., lower level with least clear data preview for general population, higher level preview with clearer data for club members, and full playback for authorized or paid customers); fine granularity scalable encryption for a fine granularity scalable coded media data stream especially useful in real time and streaming media applications.
What is lacking in the prior art is a technique that allows privacy protection, while simultaneously allowing security/surveillance to use as much of the data as possible, and to recover the original data if needed.
What is needed, and what present invention provides, is an approach which supports privacy yet still provides a security/surveillance value for the data. Furthermore, the invention does this in such a manner that existing media tools can still manipulate and play the data.
In one embodiment of the invention, for image or video based technologies the invention is applied by detecting a region of potential privacy concern, e.g. face, skin or even motion. These regions are then encrypted, in-place, producing an image that can be viewed with standard tools, but where the regions become apparently random data. The encryption can be either completely done using a public key encryption, or a symmetric encryption, e.g. AES128, can be used, or they can be combined with the AES key being encrypted with the public key, with the encrypted AES key and region definitions being stored as a comment or other field within the media stream, or even as invisible embedded watermark data. Using the private key and a special extraction tool, the original data can be recovered. For example, if the images were needed for criminal prosecution, the encrypted “face” data might be recovered. By allowing recovery of the original data, the invention provides for improved security, while still protecting privacy.
In another embodiment, various components of a digital audio channel are identified as needing privacy protection. Those segments are then subject to encryption and reinserted into the digital stream in-place of the original data. When listened to with traditional tools, the encoded data will appear as noise. The encryption can be either completely done using a public key encryption, or a symmetric encryption, e.g. 3DES, can be used and the DES key being encrypted with the public key, with the encrypted DES key being stored back within the media stream. The unencrypted components of the data can be listened to using standard tools and may provide important evidence for surveillance. If there is sufficient cause, the keys will again allow recovery of the original data using a special tool
A lossy compression, such as jpeg or mp3 applied to encrypted data, would result in data that would no longer support recovery of an approximation to the original data. In another embodiment, the digital regions to be privacy protected will be compressed or need to be compressed, and the encryption is applied to the compressed data, adjusting for any data boundaries needed to properly interact with the compression algorithms and data formats.
A more complete understanding of the present invention, as well as further features and advantages of the present invention, will be obtained by reference to the following detailed description and drawings.
The present invention provides for privacy enhanced security were the encrypted data is stored back into the data-stream such that preexisting display technologies, e.g. web browser or digital cameras, can decode and display the privacy protected data. There is no need for a separate authorizer, there is no destruction of data, no masking of data—rather the data is encrypted and reinserted into place as part of the image encoding process. In the various embodiments of this invention, the protected data is such that it can be decoded and viewed on any standard image/video display device. For example, a protected jpeg image would be viewable with a standard image viewer and the standard compliant jpeg image carries the encrypted data with it. Because of the spatially varying partial encryption, the data is still generally useful for security purposes. The protected data, and any necessary keys, can then be supplied to a separate recovery program to decrypt the protected region, e.g. to provide data for prosecution.
In the prior art, the data stream to be “protected” was transformed in a manner that permanently lost data. Because there was no need to preserve information, the transformation process was simply to apply, even if the data was later compressed. The in-place encryption of the proposed invention must be applied after any lossy compression, because lossy encryption would destroy some of the encrypted data, rendering recovery impossible.
Compared to the approach of the 20030231769 A1 Patent, the presentation invention does not require an authorizer as part of the processing, using instead predefined public keys, which is both an advantage and potential disadvantage. It is an advantage because of simplicity during encoding. It is a potential disadvantage because it reduces the privacy model as there is no image-specific authorization and the system cannot limit who is authorized to decode the image-whomever gains access to the appropriate keys can decrypt the data even if such key access was never unauthorized.
Another point of comparison of the proposed invention with the 20030231769 A1 Patent is that there is no separated data tracks. Separated tracks can provide for rapid analysis, indexing and searching. Separated tracks also simplify the handling of compressed data, which is never encrypted, and the encryption which is not subject to compression at later stages. However, using separated tracks is a disadvantage because the added data tracks also require non-standard display/transmission technology.
In
The input stream and the designated regions are then passed on to the protector (120). The undesignated blocks pass through the protector essentially unchanged, but for designated block(s) it provides for in-place encryption, i.e. it encrypts the data and places it back at essentially the same location relative to the original data stream to produce the protected stream (130). To do this, it required that the encryption process be such that the resulting data is consistent with the data formatting rules of the stream. In particular, if the input stream is a structured data format, e.g. a precompressed data source, with a combination of structure and data fields intermixed, then the encryption should be applied only to the data.
In some embodiments, it is desired to combine the privacy protection in a device that is also providing compression, e.g. a web-camera that is going from raw sensor data to a stream of compressed jpeg images. In this case, one could view the process as first compressing to produce the input stream (100) of
A critical aspect of many embodiments of the invention is the handling of in-place encryption when using compressed data. We describe a preferred embodiment using JPEG images as the mode of compression, see Gregory K. Wallace, The JPEG Still Picture Compression Standard—IEEE Transactions on Consumer Electronics, Vol. 38, No. 1, February 1992. Those skilled in the art will recognize how to adapt the approach to other block or region-based compression schemes. We use the JPEG standard as our example because it is the most common compressed image format being used, and is commonly used in streaming web-cameras which produce MJPEG, a sequence of separately encoded JPEG images.
In one preferred embodiment, we take the approach of applying the encryption during the JPEG compression process just after the DCT quantization but before the lossless Huffman encoding. In
The final aspect of the process needs to address the encryption technologies and key management. There are many classes of encryption. A simple embodiment is well suited to personal devices such as cell phone or digital camera, or even a personal web-camera monitoring the home. The process uses a symmetric key encryption, such as the AES standard or DES standard, and generates the key by hashing a user-provided pass-phrase. The pass-phrase is should not be stored and would be reentered each time the device is used. This has the advantage of simplicity, but because of the symmetric nature of the encryption, we cannot securely store the key in the image. This embodiment is effective if the protected data is intended to be used by only a small number of individuals that can share the secret key.
In another preferred embodiment, which provides for improved security and usability, we combine a public-key encryption with symmetric key technology. In
In alternative embodiment, we also include a check sum or cryptographic hash of the original data so that we verify its validity when decoded. For simpler decoding of the regions, it can also be convenient to include in the embedded payload an indication of which regions that have been designated as protected. In JPEG streams, this can be done by including a thumbnail with a particular value for the protected regions.
The process of recovering the original data from the protected data is shown in
In an alternative embodiment, the data decryption is done as part of the standard jpeg decode and before the reconstruction of a non-compressed image for display. In this manner, an embedded device such as a cell phone, which can store the data and keys locally, can function as local viewer for the decrypted data.
The invention can also be applied to non-image data, and a particularly interesting embodiment addresses audio recording or wiretapping. There has been growing concern about the US government's wiretapping of phone calls of American citizens without first obtaining a warrant. Some in the government have argued that the time required to obtain the warrant is unacceptable when listening in on potential terrorist phone calls. Acts such as CALEA, see Communications Assistance for Law Enforcement Act of 1994. Pub. L. No. 103-414, 108 Stat. 4279, already provide an infrastructure for telecom surveillance. By applying the invention, we can directly address, increasing privacy and security. The basic concept, which we call a half-tap, is show in
The process for obtaining a warrant aids the intelligence analyst considerably. With half the communication available, the probability of finding compelling evidence if the call is truly suspicious is high.
In summary, the invention provides for determination of data needing privacy protection and for the in-place encryption of that data, even under compression, such that standard display/playback mechanisms can use the protected data streams and provide information useful for security. The protected data can, with access to the appropriate private keys, be restored to the original form, further improving security.
The methods described herein can be implemented as computer-readable instructions stored on a computer-readable storage medium that, when executed by a computer, will perform the methods described herein. The methods can also be implemented as circuits embodied in photo, video or audio processing hardware, which increases the overall security since there is reduced opportunity to access the data before encryption.
While the invention has been described in conjunction with specific embodiments thereof, it is evident that many alterations, modifications, and variations will be apparent to those skilled in the art in light of the foregoing description. Accordingly, it is intended to embrace all such alterations, modifications, and variations in the appended claims.
The present invention claims priority on provisional patent application Ser. No. 60/858,140, filed on Nov. 9, 2006, entitled “PICO: privacy through invertible cryptographic obscuration” and is hereby incorporated by reference.
| Number | Date | Country | |
|---|---|---|---|
| 60858140 | Nov 2006 | US |
