The application concerns a system, method and application for electronic data processing and exchange, and more particularly, to a system, method and application for sending data from one end user to one or more end user(s) using a media-based system.
Electronic messaging (e.g., e-mail, text) and transfer of data (e.g. images, video, documents) (collectively referred to hereafter as “data”) has become a primary means of communication for a large number of organizations, businesses and individuals. Data exists in a plethora of file formats (e.g., portable document format (pdf), joint photographic experts' group (jpeg), PowerPoint (ppt), executable file (exe), etc.) which are defined as standard methods that information is encoded for storage in a computer file. The enormous variety and application of these file formats allow for specific functionality and purpose built into each piece of data.
Currently, popular social media transmission methods, such as Facebook®, Twitter®, or Instagram support limited file formats for the transmission of data. Specifically, social media applications support only text, image, and video file formats, thus limiting the type of data that may be transmitted through social media applications. Similar to the social media methods, some business applications are also built to only store and send certain multimedia file formats (photos, videos, audio files, etc.), making it complex to add encryption to the workflow.
Furthermore, current data transmission applications are an inherently insecure means of communication given that all messages, images, documents, or other media sent between senders and recipients are transmitted over networks and rely upon the security strength of the applications and networks facilitating the transmission. This problem is exacerbated by the fact that privacy controls are in the hands of 3rd party application and services providers. Hence, when an email or file is sent, or a picture posted/stored online, the user has little-to-no control over that content above what the online service is offering. In some cases, the online services “own” your content and do not offer any way to permanently delete the images or content once they are transmitted to the online services. In addition to owning user content, this user content is not encrypted when stored on servers of these online services (e.g. Facebook, medical applications, etc.). Thus, if the server is hacked, then user content (e.g. images) can easily be downloaded and viewed since the content is not encrypted. As an analogy, the modern equivalent of sending data is sending a postcard, which allows anyone who comes in contact with the postcard to read/copy/retransmit the context without any recourse.
For the longest time, this fact has not deterred a large portion of data users to continue using conventional means for transmitting data as a conduit for sensitive, confidential material. While some users, and/or the organizations they belong to, have begun using encryption as a means to secure the transfer of data, the vast majority of users must continue to transfer sensitive information using regular, unencrypted methods. However, the online and social market has matured to a point where many people are beginning to understand the ramifications of the lack of privacy controls online. Many public figures, including many celebrities, have had very sensitive pictures and documents exposed to the public via online hacking and other illegal methods. In other situations, previously sent text messages and emails are used in legal court proceedings to implicate individuals.
In business use cases, many professional services firms (such as accounting, medical, legal, and financial firms) communicate with their clients, and internally with others in their company, via unsecured email. The unsecured documents and materials (e.g. images, text documents, etc.) communicated are generally sensitive in nature and can lead to fraud and identity theft, causing significant financial loss and personal angst. In one example, medical patients now send photos and videos to their doctors through online services. Although the transmission of these files (e.g., photos and videos) may be encrypted, these files are at times stored unencrypted on the computers/servers at the hospital. In another example, businesses may communicate private documents (e.g. financial information, intellectual property, etc.) to and from clients, while also storing these private documents on their servers. These private documents are protected by email encryption during transmission, and by firewalls when stored on the servers. However, these private documents are not themselves encrypted. Thus, if the email account or file server is hacked, these private documents can easily be downloaded and viewed by the hacker.
The drawings are merely schematic representations, not intended to portray specific parameters of the invention. The drawings are intended to depict typical embodiments, and therefore should not be considered as limiting the scope of the claims.
In one embodiment, the system includes a data transcoding device including a memory device for storing clear data containing private information, and a processor configured as a data transcoder. The processor is configured to create packets of the clear data, prepare the packets for transcoding the clear data into a multimedia data file, by determining properties of the multimedia file based on parameters of the clear data. The processor configured to generate the multimedia file by transcoding the clear data based on the determined properties.
In another embodiment, the system includes a data detranscoding device including a memory device for storing a multimedia file, and a processor configured as a data detranscoder. The processor is configured to extract multimedia data from the multimedia file, detect a packet of data within the extracted multimedia data, the indecipherable packet of data including multimedia data that represents transcoded clear data, and detranscode the packet of data to determine the clear data.
In the following detailed description, numerous specific details are set forth by way of examples in order to provide a thorough understanding of the relevant teachings. However, it should be apparent that the present teachings may be practiced without such details. In other instances, well known methods, procedures, components, and/or circuitry have been described at a relatively high-level, without detail, in order to avoid unnecessarily obscuring aspects of the present teachings.
There is a need for an elegant solution capable of securely transmitting data from any file format via online service and applications (e.g. social media platforms). There is also a need for an elegant solution capable of securely storing data from any file format either locally in memory or on a remote server (e.g. cloud). In addition, there exists a need for a data transcoding system, method and application capable of simple integration with existing/dominant email, social, online/offline storage, and web communication protocols.
The described embodiments (also referred to as “Arebus”) disclose a novel method, system and application for transcoding any form of electronic data (hereafter referred to as “Clear Data”) into a multimedia file format (e.g. Arebus generates an image, video, or music file based on the Clear Data) that is indecipherable to the human eye/ear (e.g. looks like or sounds like random noise rather than Clear Data information) and only becomes decipherable as the original Clear Data after detranscoding. As multimedia files have become the near-universal file format for transmission of data across social media platforms, the described embodiments create a method, system and application for the transfer of Clear Data through platforms that are not intended to, are limited to, or do not support the transmission of all file types. This method of transcoding allows applications/services to send and/or store multimedia files without major change to the way the application operates. Basically, output application data files may be transcoded without changing the manner in which the application outputs files (e.g. the transcoding is an add-on step to the output/input of the application). This is especially useful for applications such as social media services that are limited to transmitting multimedia files (i.e. they cannot transmit other types of files).
In addition to transcoding, the described embodiments may incorporate an encryption element into the novel method, system and application, wherein the encryption element separates data from the social media or online services upon which the data is sent or stored. An element of the example method includes encrypting the Clear Data (including messages) prior to the data being sent/stored via the internet/web. For example, once the encrypted data is transcoded into a multimedia file format (e.g. the encrypted Clear Data is used to generate a multimedia file), the transcoded data can be sent or stored on any existing service and be opened just as easily using the Arebus transcoding/detranscoding platform without the need for additional support, conventions, or integration.
The described examples provide a way to encrypt and control any type of content the user would like to protect while transmitting over 3rd party applications and services. This encryption can be done first, then the resulting encrypted file can be transcoded to multimedia content. Clear data would first be encrypted, using any encryption algorithm and methodologies, including RSA using public and private keys, Blowfish, Twofish, AES and utilizing any key size (e.g. 128-bit or 256-bit) then sent/stored via the internet/web.
The described embodiments incorporate technology offering solutions around various novel components: First, Clear Data (encrypted or unencrypted) is transcoded (described in detail below) into a universally accepted multimedia file format (e.g. the Clear Data is used to generate a new multimedia file such as an image file, video file or audio file), which can be sent, posted, or stored online through existing services. Second, (optional) encryption, contact, and key management elements, are tasked with encrypting the Clear Data, and (optionally) using an end user's library of contacts to select who and/or what can access the data. In the encrypted scenario, upon receiving/opening of encrypted file, the Clear Data is extracted from the multimedia file format, then the receiving user authenticates receipt of the data via the contact list platform/key management service, and then encrypted file is decrypted and the Clear Data is presented to the receiving user on their device.
Exemplary, but not exhaustive, examples of the Arebus system/algorithms have been provided below, with reference to corresponding Figures. These examples are for illustration purposes only, and do not in any way limit the applicability, order or right supported in the Arebus disclosure.
For example, a first private user may want to send Clear Data in the form of a private text document and a private image to a second private user via email or social media message. In general, Arebus would transcode the private text document and the private image prior to transmission of the email/social media message (convert the Clear Data into an indecipherable multimedia file). The transcoding entails converting the raw data from each file into respective indecipherable multimedia files (e.g. the text document is transcoded into an indecipherable image and the private image is transcoded into another indecipherable image). It should be noted that in some embodiments, the transcoding algorithm is considered a “lossless” algorithm where none of the Clear Data lost during the transcoding phase (e.g. all of the Clear Data is converted into the multimedia file).
The indecipherable images produced by the transcoding are then transmitted to the second private user where they are detranscoded back into the original Clear Data so that they can be viewed (e.g. convert the indecipherable multimedia file back into the Clear Data). It should be noted that prior to detranscoding, the Clear Data cannot be deciphered by the human eye. Thus, even if a hacker intercepts the email or hacks into the recipient's email account, it will be difficult to decipher the private text file and image (i.e. the hacker would see indecipherable images that look like randomly distributed pixels). Although transcoding makes the Clear Data indecipherable, encryption of the data prior to transcoding would add another layer of protection so that it will be even more difficult for the hacker to determine the Clear Data.
In a similar use case, a patient may need to take a picture of a personal medical condition and send it to a doctor at the hospital. The algorithm in
In step 100, for example, User A, using a transcoding/detranscoding device (e.g. PC, Smartphone, etc.), identifies Clear Data that can include text, pictures, and other content or files. User A then specifies who has the rights to access the Clear Data (User B in the example above) and which 3rd party service should be used to deliver that Clear Data (email, text, other channels). User B may be identified as a recipient by selecting the electronic contact information for User B, which may include identification/access to User A's contact lists located on any software/application utilized by User A. In step 102, user B's delivery information is fed to the Access Control List (ACL) which manages multiple items in Arebus including User identities, relationships between User identities/data, address book functions, groupings of identities, and User permissions among others. For example, during operation, the ACL in step 102 may determine (using a permission list) if User A has permission to utilize the encryption/transcoding algorithm. If User A is permitted to utilize the encryption/transcoding algorithm, the process proceeds to the next step. If User A does not have permission to utilize the encryption/transcoding algorithm, the process ceases.
Assuming User A is granted permission by the ACL, the Clear Data is then encrypted in step 114 through the encryption engine (hereafter “Encrypted Data”) which provides key management and selects appropriate encryption algorithms which may include symmetric key cryptographic algorithms (e.g. advanced encryption standard (AES), data encryption standard (DES), etc.) and/or public key cryptographic algorithms (e.g. Rivest-Shamir-Adleman (RSA), digital signature services (DSS), Diffie-Hellman, etc.). The Encrypted Data is then transcoded in step 104 to generate an indecipherable multimedia file (hereafter “Transcoded Data”) which is used as the universal format for transmission of Clear Data across many third-party email, text, social media, and other services. This generated indecipherable multimedia file includes data that is indecipherable to the human eye/ear (e.g. an image, video or audio file that looks/sounds like random noise). The Transcoded Data is transmitted in step 106 through a chosen third-party service, and is then received by User B.
Once received by User B, the Transcoded Data is then detranscoded), using a transcoding/detranscoding device (e.g. PC, Smartphone, etc.), in step 108 from the indecipherable multimedia file to the Encrypted Data file, which is then checked against the ACL in step 110 to authenticate User B as the intended recipient. For example, during operation, the ACL in step 110 may determine (using a permission list) if User B has permission to receive the message from User A and utilize the decryption/detranscoding algorithm. If User B is permitted to utilize the encryption/transcoding algorithm (i.e. is the intended recipient), the process proceeds to the next step. If User B does not have permission to utilize the encryption/transcoding algorithm, the process ceases. It should be noted that the ACL in step 102 and the ACL in step 110 (i.e. the ACL used for transmission and reception) include the same transcoding/detranscoding algorithms, encryption/decryption algorithms, digital signature algorithms and respective keys to perform the processing of the data. Assuming User B is permitted by the ACL, then the Encrypted Data file is decrypted in step 116 and the Clear Data is accessible by User B in step 112.
It should be noted that in
It should be noted that in
Shown in
In one example, the transcoding algorithm is performed by the end user device. For example, User A operating PC 1604 may communicate with ISP server 1600 and Arebus server 1602 to download the transcoding/encryption algorithms described with reference to
Once downloaded, PC 1604 performs multimedia file transcoding (e.g. generates an indecipherable multimedia file using the Clear Data), and optionally encryption of the Clear Data, and either transmits this indecipherable multimedia file to SmartPhone 1606 operated by User B, or simply stores this indecipherable multimedia file locally on PC 1604.
In another example, the transcoding algorithm is performed remotely in the server. For example, User A operating PC 1604 may communicate with ISP server 1600 and Arebus server 1602 to upload Clear Data to the Arebus server 1602. Arebus server 1602 then transcodes the Clear Data into an indecipherable multimedia file and then transmits the indecipherable multimedia file back to PC 1604 or to some other designated recipient(s) device such as SmartPhone 1606. In either example, the Clear Data is transcoded into a multimedia file that is indecipherable until detranscoded.
Shown in
For example, if User A wants to transcode a Clear Data file such as a private text file, User A can simply drag and drop the text file into window 1500, select a multimedia file type from menu 1508, enter a secret password into window 1504 and then click button 1506. The transcoding algorithm (either performed by PC 1604 or remote server 1602) the transcodes the text file into the selected multimedia file format (e.g. generates a JPEG file based on the Clear Data) and displays this indecipherable multimedia file in window 1502.
In contrast, the lower portion of the GUI is configured to perform detranscoding of the indecipherable multimedia file. This lower portion of the GUI includes a window 1510 to drag and drop the indecipherable multimedia file, window 1512 to enter the secret password and a button 1514 to initiate the detranscoding algorithm.
For example, if User B wants to detranscode a multimedia file (e.g. the JPEG) received from User A, User B can simply drag and drop the JPEG file into window 1510, enter the same secret password that was used for transcoding into window 1512 and then click button 1514. The detranscoding algorithm (either performed by SmartPhone 1606 itself or by remote server 1602) then detranscodes the JPEG back into the Clear Data and displays this Clear Data to User B (i.e. User B can decipher the private text file sent by User A). It should be noted that if JPEG or other compression-based file is selected, the Clear Data is supplemented with extra padding (e.g. random bits) to provide enough data to allow the JPEG compression to take place.
The remaining
A use case for the configuration in
First, in step 200, User (A) identifies Clear Data that can include text, pictures, and other content or files. User A then specifies individuals who will have the right to access the Clear Data (User B, C . . . ∞) and which third party service(s) should be used to deliver the Clear Data (through email, text, other channels) to each (User B, C . . . ∞). This information is replicated as any times as necessary (number recipients) and fed to the ACL in step 202. In step 214, the Clear Data is then encrypted through the encryption engine using appropriately selected encryption algorithms. The Encrypted Data is then transcoded in step 204 to a selected type of multimedia file format which is used as the universal format for transmission across many third-party email, text, social media, and other services. The services used for transmission can be one and the same or mixed across different services. The Transcoded Data is then transmitted in step 206 through the chosen third party service(s) and received by the intended users using the same service. Once received by (User B, C . . . ∞), the transcoded Data is detranscoded in step 208 from the multimedia file to the Encrypted Data file, which is then checked against the ACL in step 210 to authenticate each (User B, C . . . ∞) as the intended recipients. Once authenticated the Encrypted Data file is decrypted in step 216 and the Clear Data is accessible by each individual (User B, C . . . ∞) in step 212. As mentioned earlier, each recipient (Users B, C . . . ∞) can be assigned a different level of security for the Clear Data. Examples of these levels may include: “read only”, “read once”, “unlimited access”, “watermarked”, “deleted after a finite amount of time”, etc.
A use case for the configuration in
First, in step 300, User (A) identifies Clear Data that can include text, pictures, and other content or files (Clear Data). User A then specifies a group of individuals intended to have collective access to the Clear Data (User B, C . . . ∞ in the example above), and which third party service(s) should be used to deliver the Clear Data (through email, text, other channels). This information is fed into the ACL in step 302. The Clear Data is then encrypted in step 314 through the encryption engine using appropriately selected encryption algorithms. The Encrypted Data is then transcoded in step 304 to multiple multimedia file formats (e.g. picture or video) which are used as the universal format for transmission across to the individuals belonging to the group through the many third-party email, text, social media, and other services. The services used for transmission can be one and the same or mixed across different services. The transcoded data is then transmitted in step 306 through the chosen third party service(s) which are received by the individual Users (User B, C . . . ∞) using the same service. Once received, the transcoded data files are detranscoded in step 308 from the multimedia files to the Encrypted Data files, which are then checked against the ACL in step 310 to authenticate the individual Users. Once authenticated the Encrypted Data files are decrypted in step 316 and the Clear Data is consumed by individual Users in step 312.
The scenario detailed in
In step 400, User A choses the Clear Data that can include text, pictures, and other content or files. User A then specifies that only he/she have access to the Clear Data and where they would like to store the Clear Data. This Clear Data is fed to the ACL in step 402. The Clear Data is then encrypted in step 414 through the encryption engine using appropriately selected encryption algorithms to create the Encrypted Data. The Encrypted Data is then transcoded in step 404 to a multimedia file (picture or video). The Transcoded Data is then stored locally in step 406 and/or in stored the cloud (public, private, or hybrid) in step 408. When User A would like to retrieve the data stored locally from the device, User A simply detranscodes in step 410 and decrypts in step 412 the multimedia file to retrieve the clear data in step 422. When User A or User B would like to retrieve the Data stored in the cloud, the Transcoded Data is detranscoded from the multimedia file in step 416 to the Encrypted Data file, which is then checked against the ACL in step 418 to authenticate the User. Once authenticated the Encrypted Data file is decrypted in step 418, the Clear Data is accessible by User A in step 420. User B cannot decrypt the data, and thus cannot access the Clear Data unless User A shares the encryption key and encryption method. It should be noted that User A can store/retrieve the Data in/from the cloud using multiple devices (e.g. User A could upload the Data to the cloud using their PC and then download the Data from the cloud using their Smartphone). An exemplary use case of this would be an individual taking a private picture and securely storing it on their phone/cloud to protect against local/online hackers exposing the picture to third parties if accessed.
In general, Arebus would take each payload message, optionally insert ACL security policy rights, optionally encrypt, and then transcode the documents into one multimedia file prior to transmission of the private messages (see steps below). The transcoding algorithm entails converting the raw data from each file into one combined file which is then transcoding it to an indecipherable multimedia file (e.g. a text document is assigned its own ACL, another private document is assigned its own ACL, then the two are combined into one package which is then transcoded to an indecipherable image). This single indecipherable image is transmitted to each of the targeted individuals, where it is detranscoded, and based on their own permission, is accessed.
First, User (A) 500 composes multiple payloads in step 502 sets that can include text, pictures, and other content or files (Clear Data). User A then specifies a group of individuals who have collective access to each Data payload (User B, C . . . ∞ in the example above) and which 3rd Party service(s) should be used to deliver the Data (through email, text, other channels). This information is fed to the ACL in step 504. Each payload is individually encrypted through the encryption engine at step 518 and appropriate encryption algorithms are selected. The encrypted data is then transcoded in step 506 to one single multiple multimedia file (e.g. picture, video, audio, etc.) which is used as the universal format for transmission in step 508 across to the individuals belonging to the group through 3rd party email, text, social media, and other services. The services used for transmission can be one and the same or mixed across different services. The multimedia file is then transmitted through the chosen 3rd party services which are received by the individual users using the same service. Once received, in step 510, the files are detranscoded from the multimedia file to the encrypted data files, which are then checked against the ACL in step 520 to authenticate which Data payload is accessible by the individual Users. Once authenticated the specific encrypted data file is decrypted in step 512 and the Clear Data 514 is consumed by the individual Users 516. A use case for the algorithm in
Although the scenarios provided above are detailed with the use of a two-step process incorporating encryption of data and transcoding of data, one skilled in the art should appreciate that the encryption process is not essential to the transmission of Clear Data through conventional social media platforms. In fact, depicted in
For example,
The algorithms described in
Shown in
In step 1006, the user device prepares for the transcoding process by determining properties of the multimedia file. For example, the user device may: 1) serializes the special packet(s), 2) determine a minimum amount of storage needed to transcode (e.g. size of special packet+size of encrypted file; The total bits=total pixels needed in transcoded image), 3) assign an image size based on the minimum storage (e.g. 160×160 pixels for animated GIF, for single image gif/jpg/other the file is determined based on total number of pixels needed split between height and width), 3) determine padding necessary for any unused storage within the image size, and 4) determine the total length of the frame to support file size.
In step 1008, the user device performs transcoding using these determined properties. In one example, the user device may: 1) combine packet+padding+data, and 2) set pixels across a canvas, and 3) convert to an image format such as GIF, JPG, portable network graphics (PNG), etc. (e.g. make individual GIFS for each frame, then concatenate the GIFS to an animated GIF).
During detranscoding phase 1001, a user device (e.g. another PC) may: 1) in step 1010 take the image and go through each image and get the pixel data (create an bit array), for example by determining color of each pixel (black pixels=0, white pixels=1) and using fault tolerance to make sure information can be extracted (e.g. pixel values between 0-127 in red-green-blue (RGB) format=0, and pixel values >127 in RGB format=1), 2) in step 1012 detect if the special (e.g. Arebus) packet is present, 3) in step 1014 parse the Arebus packet and determine if it is valid, and parse the encrypted data, 4) in optional step 1016 check digital signature to determine if the included encrypted data is intact, 5) in optional step 1018 decrypt the encrypted data, 6) in optional step 1020 check digital signature to determine if the original data is intact, and 7) in step 1022 check remaining Arebus packet attributes to validate size and digital signature of the Clear Data. If the check in step 1022 validates the Clear Data, the Clear Data is then made accessible to the user.
A simple example of data being processed during the transcoding phase 1001, is shown in steps 1-4b of
Shown in
Shown in
In some examples, the process separates access control for the data from the social media or online services upon which the data is sent or stored. For example, once the encrypted data is transcoded into a multimedia content element (e.g. the encrypted Clear Data is used to generate a multimedia file), the transcoded data can be sent or stored on any existing service and be opened just as easily using the Arebus Transcoding/detranscoding platform without the need for additional support, conventions, or integration.
As described above, the clear data can be transcoded into other types of indecipherable multimedia files such as video files and audio files. Described below with reference to
Shown in
However, since the data is being transcoded into video, there are some differences. For example, steps 1-3 in
Shown in
However, since the data is being transcoded into audio, there are some differences. For example, to transcode the data, the steps in
However, different applications and/or services may handle multimedia files in different ways. Some applications, like email, may not alter the multimedia file in any way and transmit the file “as is” to the intended recipient. Other applications, such as social media messaging services, may alter the multimedia file in order to reduce the file size by either compressing and/or resizing the content. For example, when a user upload or sends a picture through Facebook or WhatsApp®, if the file size or dimensions are deemed too large, the service will automatically compress and/or resize the image. This creates complexities when using multimedia files as ways to transfer unaltered raw data or files.
When compressing a multimedia file, parts of the resulting image are at times blurred or “artifacts” may be added due to the compression algorithms used. Depending on which level of compression and algorithm used, the original image may be altered significantly. As an example, if text is overlaid visually onto an image (e.g. drawn on the canvas) to be later recognized by a computer algorithm, if the image is heavily compressed, portions of the resulting image can become distorted, obscuring the added text. This increases the failure rate of the computer algorithm's ability to recognize the text.
A similar issue applies when images are resized. For example, the added text in the original image can be difficult to read or be recognized if the original image is reduced. The information can at times be too small to read. The resizing process may even crop out some of the text entirely.
One example method avoids data loss even if the multimedia file is altered by drawing recognizable characters representing the original data of a file and adding different methods of redundancy (e.g., duplicating or repeating characters in different sizes across an image or animation).
In this example, all content can be represented as a file comprised of hexadecimal values. A transcoder draws/writes these binary or hexadecimal values on an image canvas which is then converted to compatible image format (e.g. GIF, JPG, PNG). These values can also be drawn on multiple canvases, which are converted to frames, which combine to make an animation or video.
The resulting multimedia file can be saved, posted online, or sent through digital transmission (email, messaging, etc.). This multimedia file represents the visual depiction of the raw data of the original content.
The values can be drawn in any shape or pattern, including in a pattern made to resemble existing recognizable images/animations. These values can be drawn on blank new canvases or incorporated/overlaid onto existing images/animations.
As an example, as shown in
If space on one canvas fills up, if required, a new canvas is created, and the drawing process continues to the new canvas which can either resemble the first design layout of the original image 1701 or of a different image. These canvases can be combined into an animation, with each canvas representing a frame. The Values can be drawn onto existing pictures, images, or frames (hereafter referred to as “Transcoded Animation”).
The raw data from the computer file 1700 can be converted or mapped to binary or any other type of characters representing the original raw data. For example, these characters can include a hexadecimal representation of the raw data, or different shapes, symbols, or scripts used for written language (e.g., Arabic, Aramaic, Greek, Armenian, Cyrillic Hebrew, Mesoamerican, Mongolian, Ge'ez, Georgian), computer font types (e.g., GNU Unifont, Anonymous Pro), or preset pixel sequences that can be distinguished from each other. This conversion would happen before drawing the characters on the canvas.
The transcoding example shown in
To add further security, additional randomized characters can be included into the stream of the source file raw data before adding the Filler to make it difficult for third parties to decode these multimedia files. As with the Filler, the example system records the data of where these randomized characters are inserted, and this data is included in the image according to a known protocol and then used by the decoder to ignore or “strip-out” these characters.
For fault tolerance and error correction, in some embodiments, some or all of the initial Values (e.g., the data or mapped data from the data file 1801) can be duplicated and/or repeated across different parts of the canvas or additional canvases. This can be especially useful where the initial Values in parts of an image (or frame in an animation) become difficult to recognize during the decoding process (e.g., as a result of compression, resizing, etc. of the multimedia file). These duplicated Values can be drawn in different ways, for example different font styles, different sizes, different colors, or represented as different shapes/characters, to help with the computer's recognizability of the Values when converted to machine readable data, particularly when the transcoded image is heavily compressed or resized. The system records where these duplicated characters are drawn, and this data is saved into the final transcoded image or animation. During the decoding process, if a Value is difficult to read, the decoder will use this data to find the duplicate Value in the same canvas or another frame and that duplicate Value is used instead.
Another example embodiment can be seen in
Almost all computers and mobile phones have a single or multiple photo gallery or multimedia gallery applications, collectively referred to as “gallery applications.” A gallery application may be a native gallery application that comes with the operating system or a third-party application. These gallery applications store multimedia content in a gallery database or memory, referred to herein as a “gallery.” Since galleries are used to store personal photos or videos and may be synced to online systems for backup or sharing, these galleries tend to be targets for hackers, creating a privacy vulnerability. Users may protect new or existing content, such as photos, videos, and/or animations stored in the gallery by transcoding the content into a transcoded multimedia file (with or without encryption) and then storing the new content or replacing the existing content in the gallery with the transcoded multimedia file. The content that is transcoded for storage in a gallery is not limited to multimedia files. A user can store other types of content in the gallery by transcoding the content into a multimedia file.
After operation 2001, the stored multimedia content may be sent to a data transcoder which transcodes the stored content into a new single transcoded multimedia file, with or without encryption, in operation 2002. In operation 2003, the transcoded multimedia file is sent to the gallery application, the original multimedia content stored in the gallery may be deleted and replaced with the transcoded multimedia file. Alternatively, after operation 2000 in which the user creates or saves multimedia content, in operation 2004, the created or saved content may be sent directly to a data transcoder to be transcoded, with or without encryption, into a new single transcoded multimedia file. This transcoded multimedia file is sent to the gallery application, in operation 2005, for persistent storage in the gallery.
As shown in
Additionally, galleries can be used as a mechanism to transfer transcoded multimedia files from one application to another. When a user receives or creates a transcoded multimedia file or other type of file, the file can be saved in a shared gallery, and another application can then access the transcoded multimedia file from the shared gallery and detranscode it, as described below to recover the original file.
Transcoded multimedia files can be created in a method that can imitate existing multimedia images, animations, or videos as shown in
At operation 2208, the system obtains a multi-frame image template. Each image in the template includes a plurality of pixels arranged as blocks of M×O pixels. Operation 1120 selects a first image of the template and a first symbol of the payload. In operation 2214, the system maps the pixels of the current symbol in the stream of symbols into the next pixel block. When operation 2214 occurs immediately after operation 2210, this pixel block is the first pixel block in the first image of the template. This mapping process determines a first color and a second color for the pixel block and modifies the brightness of the two colors so that there is a contrast difference between them. In example embodiments, this difference may be greater than zero and less than the maximum brightness value of the template image. In one example, the first color is a most-used color in the block and the second color is a next most used color in the block and the difference in brightness between the two colors is about 20%. When the pixel block has only one color, operation 2214 increases or decreases the brightness of this color to provide the second color. Alternatively, operation 2214 may select another color having a brightness difference with respect to the first color. The symbol is mapped to the pixel block by replacing the pixels in the block with an image of the symbol such that an image of the symbol is rendered in the first color against a background rendered in the second color or vice versa. Alternatively, the first color may be a color that is an average of all the pixels in the selected pixel block of the template. The first color should be used to represent whatever the foreground color is in the symbol of the alphabet. For example, if each symbol in the alphabet is an 8×8 image having 64 pixels and a particular symbol has 40 black pixels and 24 white pixels, then the 40 black pixels would become the first template color. The transcoding process would then apply the second color of the pixel block for the 24 white pixels of the symbol.
Operation 2216 then determines if the last symbol in the symbol sequence has been mapped. If it has, the transcoding process is at an end, as shown by operation 2218. If there are more symbols in the sequence, control passes to operation 2220 which determines whether the pixel block that was just mapped is the last block in the current image. If it was, then operation 2222 determines whether the current image is the last image in the multi-frame image template. If the current image is the last image in the template, the transcoding process resets to the first image in the template. Thus, for large files, the transcoded multimedia file may include multiple concatenated instances of the template, each instance mapped with different symbols from the symbol stream.
Returning to operation 2220, when the current block is not the last block in the current image, the transcoding process transfers control to operation 2212 which selects the next symbol from the payload symbol stream and the next pixel block from the current image of the template and transfers control to operation 2214, described above.
When, at operation 2222, it is determined that the current image is not the last image in the template, operation 2224 selects the next image in the template as the current image and transfers control to operation 2212 so that symbols in the symbol stream can be mapped into the pixel blocks of that image to produce the transcoded multimedia file.
As shown in
An example decoding process may have some error correcting properties. For example, the alphabet may be processed to eliminate similar symbols (e.g., symbols that have a high cross-correlation), for example if the alphabet includes Roman characters one of “P” or “R” may be eliminated. In addition, if there are no image artifacts, then each tile should only be represented by 2 colors. When the pixel block contains more than 2 colors, the process determines which color is the first color and which color is the second color. For a pixel having a color other than the first color and second color, the detranscoding process may calculate a color distance of the pixel from each of the first and second colors and reassign the color of the pixel to the closer color. In addition, as described above, there may be error correcting properties in the original bit stream, for example, the bit-stream may include Reed-Solomon coding or be repeated.
Returning to
In an example embodiment, operation 2312 may parse the beginning of the data stream to determine if it represents a header packet, (e.g., a JSON header packet). Since the header is small, this may be a brute force correlation based on the first 1000 or fewer bytes of the sequence. Alternatively, knowing the structure of the header, the detranscoding process may use a syntax-based method to identify components of the header packet. The decoding process may employ multithreading. Since the transcoded multimedia file is a multiframe file, multiple frames can be detranscoded in parallel in respective threads.
As described above, the example transcoding process maps raw data to an alphabet comprising a symbol library in which each symbol is rendered in two-color pixel blocks across canvases representing the frames of an original animation or video. Colors for the background and foreground character are determined using the template. These canvases are then combined into the new transcoded multimedia file. The resulting animation resembles the original animation or video. In these embodiments, the computer vision/OCR system can be combined with other techniques, for example machine learning, to increase the effectiveness of the detection and recognition of difficult to read symbols. The system can be “taught” (or optimized) to recognize a specific font or character set used to draw the symbols on a canvas. Also, when used with the fault tolerance methodology, the decoding algorithm can learn and/or adapt from failed recognition attempts of symbols by adjusting the recognition algorithm to account for the distortions encountered. In embodiments using machine learning, when the recognition of a symbol fails, the decoder may pull the correct data from the duplicated symbol and adapt the recognition algorithm to include the visual alterations encountered from the original failed symbol recognition attempt to improve the computer vision/OCR algorithms for better recognition.
Below is an additional exemplary integration of Arebus for use with existing smart phone, desktop, and/or personal digital assistant devices. Accordingly, an end User (User A) opts into Arebus and signs on using a username and password. Arebus becomes prevalent of the end User's existing social media and/or email services and integrates into the contact list(s) existing on the social media and/or email services. One or more potential recipients (User B(s)) are selected from the contacts list(s) existing on the end User's existing social media and/or email service and acts as the base for permissions management. Once one or more recipient(s) are identified, then User A selects the data to be sent to the receiving user, and Arebus incorporates security/transcoding prior to handing the data off to the existing social media and/or email service. At the time of encryption/transcoding the data, Arebus assigns permissions, based on User A's directives, for use/viewing/restrictions of the data by User B(s). The transcoded message(s) is then sent to User B(s) where they are detranscoded and decrypted to the original Clear Data assuming User B(s) are authorized based on their assigned permissions.
The example methods, systems and applications, described above, support privacy controls for messaging and storage of pictures/videos on desktop and mobile devices. These example methods, systems and applications operate to protect email, text, and business and social media messaging (e.g. Slack, Facebook, Google Hangouts) as well as to allow an end user to take their files (including picture/videos) and store them locally and on online services encrypted in existing “camera roll,” gallery, or native application specific storage of the electronic device. The example methods, systems and applications operate to solve the data at rest issue (people accessing your online accounts or stealing your phone, getting sensitive messages and pictures) and data in transit (messages and pictures being intercepted while being sent through a service).
In addition, for consumers, the example methods, systems and applications operate to protect online content for social media (Facebook wall post, Google+, Tumblr, etc.). The example methods, systems and applications enable the ultimate online content control and kill switch if desired, placing control of data on the electronic device in the user's hands if the device is stolen or misplaced.
Referring now to the hardware/software components supportive of Arebus, below is an exemplary disclosure of a computer operable system to execute the disclosed Arebus architecture. In order to provide additional context for various aspects of the described embodiments, the following discussion are intended to provide a brief, general description of a suitable computing environment in which the various aspects of the described embodiments can be implemented. While the described embodiments have been described above in the general context of computer-executable instructions and algorithms that may run on one or more computers, those skilled in the art will recognize that the described embodiments can also be implemented in combination with other program modules and/or as a combination of hardware (see the hardware in
Generally, program modules include routines, programs, components, data structures, etc., that perform particular tasks or implement particular abstract data types. Moreover, those skilled in the art will appreciate that the example methods can be practiced with other computer system configurations, including single-processor or multiprocessor computer systems, minicomputers, mainframe computers, as well as personal computers, hand-held computing devices, microprocessor-based or programmable consumer electronics, and the like, each of which can be operatively coupled to one or more associated devices.
The described aspects of the embodiments may also be practiced in distributed computing environments where certain tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules can be located in both local and remote memory storage devices.
A computer typically includes a variety of computer-readable media. Computer-readable media can be any available media that can be accessed by the computer and includes both volatile and nonvolatile media, removable and non-removable media. By way of example, and not limitation, computer readable media can comprise computer storage media and communication media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital video disk (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information, and which can be accessed by the computer.
Communication media typically embodies computer-readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media. The term “modulated data signal” means a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal. By way of example, and not limitation, communication media includes wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, RF, infrared and other wireless media. Combinations of the any of the above should also be included within the scope of computer-readable media.
An exemplary environment for implementing various aspects of the embodiments requires a computer, the computer including a processing unit, a system memory and a system bus. The system bus couples system components including, but not limited to, the system memory to the processing unit. The processing unit can be any of various commercially available processors. Dual microprocessors and other multiprocessor architectures may also be employed as the processing unit.
The system bus can be any of several types of bus structure that may further interconnect to a memory bus (with or without a memory controller), a peripheral bus, and a local bus using any of a variety of commercially available bus architectures. The system memory includes read only memory (ROM) and random-access memory (RAM). A basic input/output system (BIOS) is stored in a non-volatile memory such as ROM, EPROM, EEPROM, which BIOS contains the basic routines that help to transfer information between elements within the computer, such as during start-up. The RAM can also include a high-speed RAM such as static RAM for caching data.
The computer further may include an internal hard disk drive (HDD) (e.g., BIDE, SATA), which internal hard disk drive may also be configured for external use in a suitable chassis (not shown), a magnetic floppy disk drive (FDD), (e.g., to read from or write to a removable diskette and an optical disk drive, (e.g., reading a CD-ROM disk or, to read from or write to other high capacity optical media such as the DVD). The hard disk drive, magnetic disk drive and optical disk drive can be connected to the system bus by a hard disk drive interface, a magnetic disk drive interface and an optical drive interface, respectively. The interface for external drive implementations includes at least one or both of Universal Serial Bus (USB) and IEEE interface technologies.
The drives and their associated computer-readable media provide nonvolatile storage of data, data structures, computer-executable instructions, and so forth. For the computer, the drives and media accommodate the storage of any data in a suitable digital format. Although the description of computer-readable media above refers to a HDD, a removable magnetic diskette, and a removable optical media such as a CD or DVD, it should be appreciated by those skilled in the art that other types of media which are readable by a computer, such as zip drives, magnetic cassettes, flash memory cards, cartridges, and the like, may also be used in the exemplary operating environment, and further, that any such media may contain computer-executable instructions for performing the methods of the described embodiments.
A number of program modules can be stored in the drives and RAM, including an operating system, one or more application programs, other program modules and program data. All or portions of the operating system, applications, modules, and/or data can also be cached in the RAM. It is appreciated that the described embodiments can be implemented with various commercially available operating systems or combinations of operating systems.
A user (User A, B, C . . . ∞) can enter commands and information into the computer through one or more wired/wireless input devices, e.g., a keyboard and a pointing device, such as a mouse. Other input devices (not shown) may include a microphone, an IR remote control, a joystick, a game pad, a stylus pen, touch screen, or the like. These and other input devices are often connected to the processing unit through an input device interface that is coupled to the system bus, but can be connected by other interfaces, such as a parallel port, an IEEE 1394 serial port, a game port, a USB port, an IR interface, etc.
A monitor or other type of display device is also connected to the system bus via an interface, such as a video adapter. In addition to the monitor, a computer typically includes other peripheral output devices (not shown), such as speakers, printers, etc. The computer may operate in a networked environment using logical connections via wired and/or wireless communications to one or more remote computers, such as a remote computer(s). The remote computer(s) can be a workstation, a server computer, a router, a personal computer, portable computer, microprocessor-based entertainment appliance, a peer device or other common network node, and typically includes many or all of the elements described relative to the computer, although, for purposes of brevity, only a memory storage device is illustrated. The logical connections depicted include wired/wireless connectivity to a local area network (LAN) and/or larger networks, e.g., a wide area network (WAN). Such LAN and WAN networking environments are commonplace in offices, and companies, and facilitate enterprise-wide computer networks, such as intranets, all of which may connect to a global communication network, e.g., the Internet.
When used in a LAN networking environment, the computer is connected to the local network through a wired and/or wireless communication network interface or adapter. The adaptor may facilitate wired or wireless communication to the LAN, which may also include a wireless access point disposed thereon for communicating with the wireless adaptor. When used in a WAN networking environment, the computer can include a modem, or is connected to a communications server on the WAN, or has other means for establishing communications over the WAN, such as by way of the Internet. The modem, which can be internal or external and a wired or wireless device, is connected to the system bus via the serial port interface. In a networked environment, program modules depicted relative to the computer, or portions thereof, can be stored in the remote memory/storage device. It will be appreciated that the network connections shown are exemplary and other means of establishing a communications link between the computers can be used.
The computer may be operable to communicate with any wireless devices or entities operatively disposed in wireless communication, e.g., a printer, scanner, desktop and/or portable computer, portable data assistant, communications satellite, any piece of equipment or location associated with a wirelessly detectable tag (e.g., a kiosk, news stand, restroom), and telephone. This includes at least Wi-Fi and Bluetooth™ wireless technologies. Thus, the communication can be a predefined structure as with conventional network or simply an ad hoc communication between at least two devices.
Wi-Fi, or Wireless Fidelity, allows connection to the Internet from a couch at home, a bed in a hotel room or a conference room at work, without wires. Wi-fi is a wireless technology like a cell phone that enables such devices, e.g., computers, to send and receive data indoors and out; anywhere within the range of a base station, Wi-Fi networks use radio technologies called IEEE 802.11 (a, b, g, etc.) to provide secure, reliable, fast wireless connectivity. A Wi-Fi network can be used to connect computers to each other, to the Internet, and to wired networks (which use IEEE 802.3 or Ethernet). Wi-Fi networks operate in the unlicensed 2.4 and 5 GHz radio bands, at an 11 Mbps (802.11a) or 54 Mbps (802.11b) data rate, for example, or with products that contain both bands (dual band), so the networks can provide real-world performance similar to the basic 10BaseT wired Ethernet networks used in many offices.
Furthermore, a second exemplary computing environment in accordance with the described embodiments may include one or more client(s). The client(s) can be hardware and/or software (e.g., threads, processes, computing devices). The client(s) can house cookie(s) and/or associated contextual information by employing the described embodiments, for example. The system also includes one or more server(s). The server(s) can also be hardware and/or software (e.g., threads, processes, computing devices). The servers can house threads to perform transformations by employing the described embodiments, for example. One possible communication between a client and a server can be in the form of a data packet adapted to be transmitted between two or more computer processes. The data packet may include a cookie and/or associated contextual information, for example. The system includes a communication framework (e.g., a global communication network such as the Internet) that can be employed to facilitate communications between the client(s) and the server(s).
Communications can be facilitated via a wired (including optical fiber) and/or wireless technology. The client(s) are operatively connected to one or more client data store(s) that can be employed to store information local to the client(s) (e.g., cookie(s) and/or associated contextual information). Similarly, the server(s) are operatively connected to one or more server data store(s) that can be employed to store information local to the servers.
What has been disclosed above includes exemplary uses of the described embodiments. It is, of course, not possible to describe every conceivable combination of elements and/or methodologies for purposes of describing the embodiments, but one of ordinary skill in the art may recognize that many further combinations and permutations of the embodiments are possible. Accordingly, the described embodiments are intended to embrace all such alterations, modifications and variations that fall within the spirit and scope of the disclosure and associated Figures. Furthermore, to the extent that the term “includes” is used in either the detailed description or the specification, such term is intended to be inclusive in a manner similar to the term “comprising” as “comprising” is interpreted when employed as a transitional word.
In addition to those features described above, the embodiments may also have the following features: Business Management UI: A User Interface focused on the needs of the Businesses that use, or integrate with, the system. Most likely this will include some level of API management tools.
Consumer UI: A User Interface focused on the needs of the end consumer that use the system.
Management Bus: Allows all services and UI's to communicate through a universal communication bus. It provides a standardized API for all inter-process communication as well as guaranteed delivery of communications.
ACL: As already described, the ACL contains multiple functions, including identity management as well as rules for object and user interaction, traditional access control list functionality, address book functionality, knowledge of pathing, information that other services need related to what functions are allowed and how they are to be configured or tuned. It may contain any form of rule (business rule) for manipulating, storing, transmitting and transforming data. This is the central database for most of the organizational functions of the system as a whole.
Cron: A service all controls all time-based functions of the system. For example, timing of encryption key expiry, events based on time cadence, timed housekeeping activities, etc.
Analytics and Reporting: A report of the massive web of data and relationships between users.
Identities & Keys: Contains multiple functions, including cryptographic identity and cryptographic key management for the purposes of encryption.
Event Processing: Complex event structures based on system level needs or the needs of the users of the system are processed. Pattern Recognition: The system finds and tracks patterns in users' connections, meta data, transmission patterns, etc.
Transformations: Transforming of data into formats that are almost universally transmittable.
Encryption: Controls all forms of encryption based on input from the Identity and Key management systems including the ACL system. This may include multiple symmetric key cryptography algorithms, as well as multiple asymmetric key cryptography implementations.
While the foregoing has described what are considered to be the best mode and/or other examples, it is understood that various modifications may be made therein and that the subject matter disclosed herein may be implemented in various forms and examples, and that the teachings may be applied in numerous applications, only some of which have been described herein. It is intended by the following claims to claim any and all applications, modifications and variations that fall within the true scope of the present teachings.
Unless otherwise stated, all measurements, values, ratings, positions, magnitudes, sizes, and other specifications that are set forth in this specification, including in the claims that follow, are approximate, not exact. They are intended to have a reasonable range that is consistent with the functions to which they relate and with what is customary in the art to which they pertain.
It will be understood that the terms and expressions used herein have the ordinary meaning as is accorded to such terms and expressions with respect to their corresponding respective areas of inquiry and study except where specific meanings have otherwise been set forth herein. Relational terms such as first and second and the like may be used solely to distinguish one entity or action from another without necessarily requiring or implying any actual such relationship or order between such entities or actions. The terms “comprises,” “comprising,” or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by “a” or “an” does not, without further constraints, preclude the existence of additional identical elements in the process, method, article, or apparatus that comprises the element.
The Abstract of the Disclosure is provided to allow the reader to quickly ascertain the nature of the technical disclosure. It is submitted with the understanding that it will not be used to interpret or limit the scope or meaning of the claims. In addition, in the foregoing Detailed Description, it can be seen that various features are grouped together in various embodiments for the purpose of streamlining the disclosure. This method of disclosure is not to be interpreted as reflecting an intention that the claimed embodiments require more features than are expressly recited in each claim.
This application is a continuation of U.S. patent application Ser. No. 16/263,160, filed Jan. 31, 2019, which is a continuation-in-part of U.S. patent application Ser. No. 15/043,737, filed Feb. 15, 2016, which claims priority to U.S. Provisional Application No. 62/116,682, filed Feb. 16, 2015. U.S. patent application Ser. No. 16/263,160 (referenced above) also claims priority to U.S. Provisional Application No. 62/626,540, filed Feb. 5, 2018, the contents of such applications being incorporated by reference herein.
Number | Date | Country | |
---|---|---|---|
62116682 | Feb 2015 | US | |
62626540 | Feb 2018 | US |
Number | Date | Country | |
---|---|---|---|
Parent | 16263160 | Jan 2019 | US |
Child | 17534442 | US |
Number | Date | Country | |
---|---|---|---|
Parent | 15043737 | Feb 2016 | US |
Child | 16263160 | US |