Patent Grant
11651468
This invention relates to graphical user interfaces and more particularly relates to data security in a graphical user interface.
Most data are displayed to a user in some form of graphical user interface. However, a user may easily take a screenshot of the data in the graphical user interface and use an image editor or other program to alter the data. The altered data may look like the original screenshot of the graphical user interface, and may be used to perpetrate fraud, identity theft, or the like. For example, a user may alter account numbers, transactions, monetary totals, authorizations, or the like.
Apparatuses are presented for watermark security. An apparatus, in one embodiment, includes a watermark module configured to generate a digital watermark to be presented as part of a graphical interface based on data presented on the graphical interface. A digital watermark may verify an authenticity of data to be presented in a graphical interface. An apparatus, in further embodiments, includes a presentation module configured to embed the digital watermark into the graphical interface prior to the data being presented in the graphical interface such that the digital watermark is graphically imperceptible to a user, dynamically update the digital watermark during runtime in response to detecting a change in the at least a portion of the data that is encoded into the digital watermark, and re-embed the digital watermark into the graphical interface in response to the digital watermark being updated.
Methods are presented for watermark security. A method, in one embodiment, includes generating a digital watermark to be presented as part of a graphical interface based on data presented on the graphical interface. A digital watermark may verify an authenticity of data to be presented in a graphical interface. A method, in certain embodiments, includes embedding the digital watermark into the graphical interface prior to the data being presented in the graphical interface such that the digital watermark is graphically imperceptible to a user. A method, in further embodiments, includes dynamically updating the digital watermark during runtime in response to detecting a change in the at least a portion of the data that is encoded into the digital watermark. A method, in various embodiments, includes re-embedding the digital watermark into the graphical interface in response to the digital watermark being updated.
Apparatuses are presented for watermark security. An apparatus, in one embodiment, includes means for generating a digital watermark to be presented as part of a graphical interface based on data presented on the graphical interface. A digital watermark may verify an authenticity of data to be presented in a graphical interface. An apparatus, in certain embodiments, includes means for embedding the digital watermark into the graphical interface prior to the data being presented in the graphical interface such that the digital watermark is graphically imperceptible to a user. In some embodiments, an apparatus includes means for dynamically updating the digital watermark during runtime in response to detecting a change in the at least a portion of the data that is encoded into the digital watermark. In certain embodiments, an apparatus includes means for re-embedding the digital watermark into the graphical interface in response to the digital watermark being updated.
In order that the advantages of the invention will be readily understood, a more particular description of the invention briefly described above will be rendered by reference to specific embodiments that are illustrated in the appended drawings. Understanding that these drawings depict only typical embodiments of the invention and are not therefore to be considered to be limiting of its scope, the invention will be described and explained with additional specificity and detail through the use of the accompanying drawings, in which:
Reference throughout this specification to “one embodiment,” “an embodiment,” or similar language means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment. Thus, appearances of the phrases “in one embodiment,” “in an embodiment,” and similar language throughout this specification may, but do not necessarily, all refer to the same embodiment, but mean “one or more but not all embodiments” unless expressly specified otherwise. The terms “including,” “comprising,” “having,” and variations thereof mean “including but not limited to” unless expressly specified otherwise. An enumerated listing of items does not imply that any or all of the items are mutually exclusive and/or mutually inclusive, unless expressly specified otherwise. The terms “a,” “an,” and “the” also refer to “one or more” unless expressly specified otherwise.
Furthermore, the described features, advantages, and characteristics of the embodiments may be combined in any suitable manner. One skilled in the relevant art will recognize that the embodiments may be practiced without one or more of the specific features or advantages of a particular embodiment. In other instances, additional features and advantages may be recognized in certain embodiments that may not be present in all embodiments.
These features and advantages of the embodiments will become more fully apparent from the following description and appended claims, or may be learned by the practice of embodiments as set forth hereinafter. As will be appreciated by one skilled in the art, aspects of the present invention may be embodied as a system, method, and/or computer program product. Accordingly, aspects of the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, micro-code, etc.) or an embodiment combining software and hardware aspects that may all generally be referred to herein as a “circuit,” “module,” or “system.” Furthermore, aspects of the present invention may take the form of a computer program product embodied in one or more computer readable medium(s) having program code embodied thereon.
Many of the functional units described in this specification have been labeled as modules, in order to more particularly emphasize their implementation independence. For example, a module may be implemented as a hardware circuit comprising custom VLSI circuits or gate arrays, off-the-shelf semiconductors such as logic chips, transistors, or other discrete components. A module may also be implemented in programmable hardware devices such as field programmable gate arrays, programmable array logic, programmable logic devices or the like.
Modules may also be implemented in software for execution by various types of processors. An identified module of program code may, for instance, comprise one or more physical or logical blocks of computer instructions which may, for instance, be organized as an object, procedure, or function. Nevertheless, the executables of an identified module need not be physically located together, but may comprise disparate instructions stored in different locations which, when joined logically together, comprise the module and achieve the stated purpose for the module.
Indeed, a module of program code may be a single instruction, or many instructions, and may even be distributed over several different code segments, among different programs, and across several memory devices. Similarly, operational data may be identified and illustrated herein within modules, and may be embodied in any suitable form and organized within any suitable type of data structure. The operational data may be collected as a single data set, or may be distributed over different locations including over different storage devices, and may exist, at least partially, merely as electronic signals on a system or network. Where a module or portions of a module are implemented in software, the program code may be stored and/or propagated on in one or more computer readable medium(s).
The computer program product may include a computer readable storage medium (or media) having computer readable program instructions thereon for causing a processor to carry out aspects of the present invention.
The computer readable storage medium can be a tangible device that can retain and store instructions for use by an instruction execution device. The computer readable storage medium may be, for example, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of the computer readable storage medium includes the following: a portable computer diskette, a hard disk, a random access memory (“RAM”), a read-only memory (“ROM”), an erasable programmable read-only memory (“EPROM” or Flash memory), a static random access memory (“SRAM”), a portable compact disc read-only memory (“CD-ROM”), a digital versatile disk (“DVD”), a memory stick, a floppy disk, a mechanically encoded device such as punch-cards or raised structures in a groove having instructions recorded thereon, and any suitable combination of the foregoing. A computer readable storage medium, as used herein, is not to be construed as being transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission media (e.g., light pulses passing through a fiber-optic cable), or electrical signals transmitted through a wire.
Computer readable program instructions described herein can be downloaded to respective computing/processing devices from a computer readable storage medium or to an external computer or external storage device via a network, for example, the Internet, a local area network, a wide area network and/or a wireless network. The network may comprise copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers and/or edge servers. A network adapter card or network interface in each computing/processing device receives computer readable program instructions from the network and forwards the computer readable program instructions for storage in a computer readable storage medium within the respective computing/processing device.
Computer readable program instructions for carrying out operations of the present invention may be assembler instructions, instruction-set-architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state-setting data, or either source code or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, C++ or the like, and conventional procedural programming languages, such as the “C” programming language or similar programming languages. The computer readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate arrays (FPGA), or programmable logic arrays (PLA) may execute the computer readable program instructions by utilizing state information of the computer readable program instructions to personalize the electronic circuitry, in order to perform aspects of the present invention.
Aspects of the present invention are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer readable program instructions.
These computer readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks. These computer readable program instructions may also be stored in a computer readable storage medium that can direct a computer, a programmable data processing apparatus, and/or other devices to function in a particular manner, such that the computer readable storage medium having instructions stored therein comprises an article of manufacture including instructions which implement aspects of the function/act specified in the flowchart and/or block diagram block or blocks.
The computer readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process, such that the instructions which execute on the computer, other programmable apparatus, or other device implement the functions/acts specified in the flowchart and/or block diagram block or blocks.
Many of the functional units described in this specification have been labeled as modules, in order to more particularly emphasize their implementation independence. For example, a module may be implemented as a hardware circuit comprising custom VLSI circuits or gate arrays, off-the-shelf semiconductors such as logic chips, transistors, or other discrete components. A module may also be implemented in programmable hardware devices such as field programmable gate arrays, programmable array logic, programmable logic devices or the like.
Modules may also be implemented in software for execution by various types of processors. An identified module of program instructions may, for instance, comprise one or more physical or logical blocks of computer instructions which may, for instance, be organized as an object, procedure, or function. Nevertheless, the executables of an identified module need not be physically located together, but may comprise disparate instructions stored in different locations which, when joined logically together, comprise the module and achieve the stated purpose for the module.
The schematic flowchart diagrams and/or schematic block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of apparatuses, systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the schematic flowchart diagrams and/or schematic block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions of the program code for implementing the specified logical function(s).
It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the Figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. Other steps and methods may be conceived that are equivalent in function, logic, or effect to one or more blocks, or portions thereof, of the illustrated Figures.
Although various arrow types and line types may be employed in the flowchart and/or block diagrams, they are understood not to limit the scope of the corresponding embodiments. Indeed, some arrows or other connectors may be used to indicate only the logical flow of the depicted embodiment. For instance, an arrow may indicate a waiting or monitoring period of unspecified duration between enumerated steps of the depicted embodiment. It will also be noted that each block of the block diagrams and/or flowchart diagrams, and combinations of blocks in the block diagrams and/or flowchart diagrams, can be implemented by special purpose hardware-based systems that perform the specified functions or acts, or combinations of special purpose hardware and program code.
In one embodiment, the system 100 includes one or more hardware devices 102. The hardware devices 102 (e.g., computing devices, information handling devices, or the like) may include one or more of a desktop computer, a laptop computer, a mobile device, a tablet computer, a smart phone, a set-top box, a gaming console, a smart TV, a smart watch, a fitness band, an optical head-mounted display (e.g., a virtual reality headset, smart glasses, or the like), an HDMI or other electronic display dongle, a personal digital assistant, and/or another computing device comprising a processor (e.g., a central processing unit (CPU), a processor core, a field programmable gate array (FPGA) or other programmable logic, an application specific integrated circuit (ASIC), a controller, a microcontroller, and/or another semiconductor integrated circuit device), a volatile memory, and/or a non-volatile storage medium. In certain embodiments, the hardware devices 102 are in communication with one or more servers 108 of one or more third party service providers 108 and/or one or more backend servers 110 via a data network 106, described below. The hardware devices 102, in a further embodiment, are capable of executing various programs, program code, applications, instructions, functions, or the like.
An aggregation module 104, in the depicted embodiment, includes a security module 120, which is described in more detail below with reference to
In one embodiment, a security module 120 injects, encodes, or otherwise places a secure watermark within a dynamically generated screen of a graphical user interface on a hardware device 102 or the like. The secure watermark, in certain embodiments, may be undetectable and/or substantially undetectable to a user viewing the graphical user interface. For example, the secure watermark may be encoded and/or embedded in a background, button or other user interface element, image, or the like of the graphical user interface (e.g., smaller than detectable and/or noticeable by a user, using colors indistinguishable and/or substantially indistinguishable from neighboring colors, using a pattern that's indistinguishable and/or substantially indistinguishable to a user, or the like). The secure watermark, in certain embodiments, is structured and configured such that it continues to be detectable and/or decodable by a security module 120 when duplicated (e.g., in a screenshot or other image file, in a paper printout, converted to a document, or the like).
A security module 120, in certain embodiments, may encode at least a portion of the data displayed in the graphical user interface into a secure watermark (e.g., for later verification, or the like). For example, a security module 120 may encode one or more numbers from a graphical user interface (e.g., transaction amounts, monetary values and/or totals, monetary transfer amounts, financial account balances, dates, social security numbers, account numbers, ID numbers, or the like) into a secure watermark. A security module 120 may encode and/or embed data in a secure watermark and/or embed a secure watermark in a graphical user interface, in a location away from the original data being encoded, to increase a likelihood that the secure watermark and/or the encoded data will not be edited, tampered with, altered, or the like by a user attempting to edit, tamper with, and/or alter the original data. A security module 120 may dynamically update a secure watermark during runtime as data changes, in a live graphical user interface presented to a user.
A security module 120, in one embodiments, provides an interface (e.g., an API, an HTML, or web interface such as an upload form, a submission queue, a monitored file folder, a command line interface, a GUI, or the like) for users to submit screenshots, print outs, copies, or the like taken from a graphical user interface with a secure watermark, for verification. For example, a user at a third party service provider 108 (e.g., an administrator, account manager, loan officer, customer support representative, employee, or the like) may receive a screenshot, printout, or other copy of a graphical user interface (e.g., for income verification, asset verification, identity verification, account verification, or the like) from an end-user of a hardware device 102, with an embedded secure watermark, and may submit the received screenshot, printout, or other copy of the graphical user interface to a security module 120 (e.g., a security module 120 disposed on a backend server 110, a security module 120 of a third party service provider server 108, or the like), and the security module 120 may scan or otherwise process the submission to decode the secure watermark and compare to the actual data in the submission. In certain embodiments, a security module 120 may use optical character recognition (OCR) to determine the actual data (e.g., visual text and/or numbers), to compare to data encoded in a secure watermark. In other embodiments, a user may enter the actual data from a submission for a security module 120.
In this manner, in certain embodiments, a security module 120 may reduce or eliminate verification fraud, may secure dynamic displays of data such as graphical user interfaces, and/or may otherwise use a secure watermark to verify the authenticity of displayed data.
In one embodiment, an aggregation module 104 is configured to determine and/or receive a user's electronic credentials (e.g., username and password, fingerprint scan, retinal scan, digital certificate, personal identification number (PIN), challenge response, security token, hardware token, software token, DNA sequence, signature, facial recognition, voice pattern recognition, bio-electric signals, two-factor authentication credentials, or the like) for one or more third party service providers 108. The aggregation module 104, in certain embodiments, accesses a server 108 of a third party service provider 108 using a user's electronic credentials to download data associated with the user from the server 108, such as a user's photos, a user's social media posts, a user's medical records, a user's financial transaction records or other financial data, and/or other data associated with and/or owned by a user but stored by a server 108 of a third party service provider 108 (e.g., stored by hardware not owned, maintained, and/or controlled by the user). The aggregation module 104, in various embodiments, may provide the downloaded data to the user locally (e.g., displaying the data on an electronic display of a hardware device 102); may provide the downloaded data from the hardware device 102 of the user to and/or package the data for a remote server 110 (e.g., a backend aggregation module 104b) or other remote device (e.g., another hardware device 102 of the user, a hardware device 102 of a different user, or the like) which may be unaffiliated with the third party service provider 108; may provide one or more alerts, messages, advertisements, or other communications to the user (e.g., on a hardware device 102) based on the downloaded data; or the like.
In certain embodiments, the system 100 includes a plurality of aggregation modules 104 disposed/located on hardware devices 102 of a plurality of different users (e.g., comprising hardware of and/or executable code running on one or more hardware devices 102). The plurality of aggregation modules 104 may act as a distributed and/or decentralized system 100, executing across multiple hardware devices 102, which are geographically dispersed and using different IP addresses, each downloading and/or aggregating data (e.g., photos, social media posts, medical records, financial transaction records, other financial data, and/or other user data) separately, in a distributed and/or decentralized manner. While a third party service provider 108 (e.g., a financial institution, bank, credit union, and/or other online banking provider; a social media site; a medical provider; a photo hosting site; or the like) may block a data aggregation service or other entity from accessing data for a plurality of users from a single location (e.g., a single IP address, a single block of IP addresses, or the like), a distributed and/or decentralized swarm of many aggregation modules 104, in certain embodiments, may be much more difficult for a third party service provider 108 to block.
In one embodiment, a hardware device 102 may include and/or execute an internet browser, which a user may use to access a server 108 of a third party service provider 108 (e.g., by loading a webpage of the third party service provider 108 in the internet browser). At least a portion of an aggregation module 104, in certain embodiments, may comprise a plugin to and/or an extension of an internet browser of a user's personal hardware device 102, so that a third party service provider 108 may not block the aggregation module 104 from accessing the server 108 of the third party service provider 108 without also blocking the user's own access to the server 108 using the internet browser. For example, the aggregation module 104 may use the same cookies, IP address, saved credentials, or the like as a user would when accessing a server 108 of a third party service provider 108 through the internet browser. In certain embodiments, the aggregation module 104 may support integration with multiple different types of internet browsers (e.g., on different hardware devices 102).
An aggregation module 104, in certain embodiments, may mimic or copy a user's behavioral pattern in accessing a server 108 of a third party service provider 108, to reduce a likelihood that the third party service provider 108 may distinguish access to the server 108 by an aggregation module 104 from access to the server 108 by a user. For example, an aggregation module 104 may visit one or more locations (e.g., webpages) of a server 108 of a third party service provider 108, even if the aggregation module 104 does not intend to download data from each of the one or more locations, may wait for a certain delay time between accessing different locations, may use a certain scroll pattern, or the like, to mask the aggregation module 104's downloading and/or aggregating of a user's data, to reduce the chances of being detected and/or blocked by the third party service provider 108.
In one embodiment, at least a portion of an aggregation module 104 may be integrated with or otherwise part of another application executing on a hardware device 102, such as a personal financial management application (e.g., computer executable code for displaying a user's financial transactions from multiple financial institutions, determining and/or displaying a user's financial budgets and/or financial goals, determining and/or displaying a user's account balances, determining and/or displaying a user's net worth, or the like), a photo viewer, a medical application, an insurance application, an accounting application, a social media application, or the like, which may use data the aggregation module 104 downloads from a server 108 of a third party service provider 108.
In one embodiment, the aggregation modules 104a comprise a distributed system 100, with the aggregation modules 104a and/or the associated hardware devices 102 downloading and/or aggregating data substantially independently (e.g., downloading data concurrently or non-concurrently, without a global clock, with independent success and/or failure of components). Distributed aggregation modules 104a may pass messages to each other and/or to a backend aggregation module 104b, to coordinate their distributed aggregation of data for users. In one embodiment, the aggregation modules 104a are decentralized (e.g., hardware devices 102 associated with users perform one or more aggregation functions such as downloading data), rather than relying exclusively on a centralized server or other device to perform one or more aggregation functions.
In a distributed and/or decentralized system 100, a central entity, such as a backend aggregation module 104b and/or a backend server 110, in certain embodiments, may still provide, to one or more aggregation modules 104a, one or more messages comprising instructions for accessing a server 108 of a third party service provider 108 using a user's credentials, or the like. For example, a backend aggregation module 104b may provide one or more aggregation modules 104a of one or more hardware devices 102 with one or more sets of instructions for accessing a server 108 of a third party service 108, such as a location for entering a user's electronic credentials (e.g., a text box, a field, a label, a coordinate, or the like), an instruction for submitting a user's electronic credentials (e.g., a button to press, a link to click, or the like), one or more locations of data associated with a user (e.g., a row in a table or chart, a column in a table or chart, a uniform resource locator (URL) or other address, a coordinate, a label, or the like), and/or other instructions or information, using which the aggregation modules 104a may access and download a user's data.
In a further embodiment, one or more aggregation modules 104a may pass messages to each other, such as instructions for accessing a server 108 of a third party service provider 108 using a user's credentials, or the like, in a peer-to-peer manner. In another embodiment, a central entity, such as a backend aggregation module 104b, may initially seed one or more sets of instructions for accessing a server 108 of a third party service provider 108 using a user's credentials to one or more aggregation modules 104a, and the one or more aggregation modules 104a may send the one or more sets of instructions to other aggregation modules 104a.
Instructions for accessing a user's data, however, in certain embodiments, may change over time, may vary for different users of a third party service provider 108, or the like (e.g., due to upgrades, different service levels or servers 108 for different users, acquisitions and/or consolidation of different third party service providers 108, or the like), causing certain instructions to fail over time and/or for certain users, preventing an aggregation module 104 from accessing and downloading a user's data. A backend aggregation module 104b, in one embodiment, may provide one or more aggregation modules 104a with a hierarchical list of multiple sets of instructions, known to have enabled access to a user's data from a server 108 of a third party service provider 108. An aggregation module 104a on a hardware device 102 may try different sets of instructions in hierarchical order, until the aggregation module 104a is able to access a user's data.
An aggregation module 104, in certain embodiments, may provide an interface to a user allowing the user to repair or fix failed instructions for accessing the user's data, by graphically identify an input location for the user's electronic credentials, an instruction for submitting a user's electronic credentials, a location of data associated with the user, or the like. An aggregation module 104, in one embodiment, may highlight or otherwise suggest (e.g., bold, color, depict a visual comment or label, or the like) an estimate which the aggregation module 104 has determined of an input location for the user's electronic credentials, an instruction for submitting a user's electronic credentials, a location of data associated with the user, or the like. For example, an aggregation module 104 may process a web page of a server 108 of a third party service provider 108 (e.g., parse and/or search a hypertext markup language (HTML) file) to estimate an input location for the user's electronic credentials, an instruction for submitting a user's electronic credentials, a location of data associated with the user, or the like.
An aggregation module 104, in certain embodiments, may provide an advanced interface for a user to graphically repair broken and/or failed instructions for accessing a user's data from a server 108 of a third party service provider 108, which allows a user to view code of a webpage (e.g., HTML or the like) and to identify an input location for the user's electronic credentials, an instruction for submitting a user's electronic credentials, a location of data associated with the user, or the like within the code of the webpage. In one embodiment, an aggregation module 104 may provide a basic interface for a user to graphically repair broken and/or failed instructions for accessing a user's data from a server 108 of a third party service provider 108 by overlaying a basic interface over a web page or other location of the server 108 wherein the user may graphically identify an input location for the user's electronic credentials, an instruction for submitting a user's electronic credentials, a location of data associated with the user, or the like (e.g., without requiring the user to view HTML or other code of the web page). An aggregation module 104, in certain embodiments, may provide an interface that includes a selectable list of broken and/or missing instructions, locations, or the like, and may highlight and/or display suggestions graphically in response to a user selecting an item from the list.
An aggregation module 104, in one embodiment, may test instructions provided by users (e.g., using a test set) before allowing each of the aggregation modules 104a to use the provided instructions (e.g., to prevent an abusive user from providing false or incorrect instructions). An aggregation module 104 may score or rate users based on a success rate of the users' provided instructions, and may expedite (e.g., provide to a greater number of aggregation modules 104a and/or users) the use of instructions from users with a higher score or rating. The distributed network of aggregation modules 104, in certain embodiments, may thereby be self-healing and/or self-testing, allowing continued access to and/or aggregation of users' data from one or more third party service providers 108, even if access instructions change or become broken.
The one or more aggregation modules 104, in certain embodiments, may provide an interface (e.g., an application programming interface (API)) to provide downloaded and/or aggregated user data from servers 108 of one or more third party service providers 108 to one or more other entities (e.g., a remote server 110 or other hardware device 102 unaffiliated with the third party service provider 108, a backend aggregation module 104b, or the like). The interface, in one embodiment, comprises a private interface between aggregation modules 104a of users' hardware devices 102 and one or more backend aggregation modules 104b. For example, this may enable a backend aggregation module 104b to provide a user with access to downloaded and/or aggregated user data at multiple locations, on multiple hardware devices 102, through multiple channels, or the like, even if the user's hardware device 102 which downloaded the data is turned off, out of battery, not connected to the data network 106, or the like. In another embodiment, the interface comprises a public and/or open interface, which may be secured, allowing a user to share the user's downloaded data from an aggregation module 104 to one or more other tools, services, and/or other entities to store, process, and/or otherwise use the data.
In various embodiments, an aggregation module 104 may be embodied as hardware, software, or some combination of hardware and software. In one embodiment, an aggregation module 104 may comprise executable program code stored on a non-transitory computer readable storage medium for execution on a processor of a hardware device 102, a backend server 110, or the like. For example, an aggregation module 104 may be embodied as executable program code executing on one or more of a hardware device 102, a backend server 110, a combination of one or more of the foregoing, or the like. In such an embodiment, the various modules that perform the operations of an aggregation module 104, as described below, may be located on a hardware device 102, a backend server 110, a combination of the two, and/or the like.
In various embodiments, an aggregation module 104 may be embodied as a hardware appliance that can be installed or deployed on a backend server 110, on a user's hardware device 102 (e.g., a dongle, a protective case for a phone 102 or tablet 102 that includes one or more semiconductor integrated circuit devices within the case in communication with the phone 102 or tablet 102 wirelessly and/or over a data port such as USB or a proprietary communications port, or another peripheral device), or elsewhere on the data network 106 and/or collocated with a user's hardware device 102. In certain embodiments, an aggregation module 104 may comprise a hardware device such as a secure hardware dongle or other hardware appliance device (e.g., a set-top box, a network appliance, or the like) that attaches to another hardware device 102, such as a laptop computer, a server, a tablet computer, a smart phone, or the like, either by a wired connection (e.g., a USB connection) or a wireless connection (e.g., Bluetooth®, near-field communication (NFC), or the like); that attaches to an electronic display device (e.g., a television or monitor using an HDMI port, a DisplayPort port, a Mini DisplayPort port, VGA port, DVI port, or the like); that operates substantially independently on a data network 106; or the like. A hardware appliance of an aggregation module 104 may comprise a power interface, a wired and/or wireless network interface, a graphical interface (e.g., a graphics card and/or GPU with one or more display ports) that outputs to a display device, and/or a semiconductor integrated circuit device as described below, configured to perform the functions described herein with regard to an aggregation module 104.
An aggregation module 104, in such an embodiment, may comprise a semiconductor integrated circuit device (e.g., one or more chips, die, or other discrete logic hardware), or the like, such as a field-programmable gate array (FPGA) or other programmable logic, firmware for an FPGA or other programmable logic, microcode for execution on a microcontroller, an application-specific integrated circuit (ASIC), a processor, a processor core, or the like. In one embodiment, an aggregation module 104 may be mounted on a printed circuit board with one or more electrical lines or connections (e.g., to volatile memory, a non-volatile storage medium, a network interface, a peripheral device, a graphical/display interface. The hardware appliance may include one or more pins, pads, or other electrical connections configured to send and receive data (e.g., in communication with one or more electrical lines of a printed circuit board or the like), and one or more hardware circuits and/or other electrical circuits configured to perform various functions of an aggregation module 104.
The semiconductor integrated circuit device or other hardware appliance of an aggregation module 104, in certain embodiments, comprises and/or is communicatively coupled to one or more volatile memory media, which may include but is not limited to: random access memory (RAM), dynamic RAM (DRAM), cache, or the like. In one embodiment, the semiconductor integrated circuit device or other hardware appliance of an aggregation module 104 comprises and/or is communicatively coupled to one or more non-volatile memory media, which may include but is not limited to: NAND flash memory, NOR flash memory, nano random access memory (nano RAM or NRAM), nanocrystal wire-based memory, silicon-oxide based sub-10 nanometer process memory, graphene memory, Silicon-Oxide-Nitride-Oxide-Silicon (SONOS), resistive RAM (RRAM), programmable metallization cell (PMC), conductive-bridging RAM (CBRAM), magneto-resistive RAM (MRAM), dynamic RAM (DRAM), phase change RAM (PRAM or PCM), magnetic storage media (e.g., hard disk, tape), optical storage media, or the like.
The data network 106, in one embodiment, includes a digital communication network that transmits digital communications. The data network 106 may include a wireless network, such as a wireless cellular network, a local wireless network, such as a Wi-Fi network, a Bluetooth® network, a near-field communication (NFC) network, an ad hoc network, and/or the like. The data network 106 may include a wide area network (WAN), a storage area network (SAN), a local area network (LAN), an optical fiber network, the internet, or other digital communication network. The data network 106 may include two or more networks. The data network 106 may include one or more servers, routers, switches, and/or other networking equipment. The data network 106 may also include one or more computer readable storage media, such as a hard disk drive, an optical drive, non-volatile memory, RAM, or the like.
The one or more third party service providers 108, in one embodiment, may include one or more network accessible computing systems such as one or more web servers hosting one or more web sites, an enterprise intranet system, an application server, an application programming interface (API) server, an authentication server, or the like. The one or more third party service providers 108 may include systems related to various institutions or organizations. For example, a third party service provider 108 may include a system providing electronic access to a financial institution, a university, a government agency, a utility company, an email provider, a social media site, a photo sharing site, a video sharing site, a data storage site, a medical provider, or another entity that stores data associated with a user. A third party service provider 108 may allow users to create user accounts to upload, view, create, and/or modify data associated with the user. Accordingly, a third party service provider 108 may include an authorization system, such as a login element or page of a web site, application, or similar front-end, where a user can provide credentials, such as a username/password combination, to access the user's data.
In one embodiment, the one or more backend servers 110 and/or one or more backend aggregation modules 104b provide central management of the networked swarm of aggregation modules 104a. For example, the one or more backend aggregation modules 104b and/or a backend server 110 may store downloaded user data from the aggregation modules 104a centrally, may provide instructions for the aggregation modules 104a to access user data from one or more third party service providers 108 using user credentials, or the like. A backend server 110 may include one or more servers located remotely from the hardware devices 102 and/or the one or more third party service providers 108. A backend server 110 may include at least a portion of the modules or sub-modules described below with regard to the aggregation modules 104 of
In one embodiment, the authentication module 202 receives a user's electronic credentials for a third party service provider 108 from the user on a hardware device 102 of the user. In a further embodiment, the authentication module 202 may receive electronic credentials for a different user (e.g., from a different hardware device 102, from a backend aggregation module 104, or the like), which may be encrypted and/or otherwise secured, so that the direct access module 204 may download data for the different user (e.g., downloading data for multiple users from a single user's hardware device 102).
For example, in the distributed/decentralized system 100, if one user's hardware device 102 is turned off, asleep, out of battery, blocked by a third party service provider 108, or the like, in certain embodiments, an aggregation module 202 on a different user's hardware device 102 and/or on a backend server 110 may download data for the one user, using the one user's electronic credentials, and may send the data to the one user's hardware device 102, may send an alert and/or push notification to the one user's hardware device 102, or the like. In this manner, in one embodiment, a user may continue to aggregate data, receive alerts and/or push notifications, or the like, even if the user's own hardware device 102 is blocked, unavailable, or the like. In cooperation with one or more authentication modules 202, the aggregation modules 104a, 104b, in certain embodiments, may communicate with each other using a secure and/or encrypted protocol, and/or may store electronic credentials in a secure and/or encrypted manner, so that a user may not see and/or access another user's electronic credentials, downloaded data, or other private and/or sensitive data.
In embodiments where an aggregation module 104 comprises hardware (e.g., a semiconductor integrated circuit device such as an FPGA, an ASIC, or the like), the authentication module 202 may comprise dedicated security hardware for storing and/or processing electronic credentials, downloaded data, and/or other sensitive and/or private data, such as a secure cryptoprocessor (e.g., a dedicated computer on a chip or microprocessor embedded in a packaging with one or more physical security measures) which does not output decrypted data to an unsecure bus or storage, which stores cryptographic keys, a secure storage device; a trusted platform module (TPM) such as a TPM chip and/or TPM security device; a secure boot ROM or other type of ROM; an authentication chip; or the like. In another embodiment, the authentication module 202 may store and/or process electronic credentials, downloaded data, and/or other sensitive data in a secure and/or encrypted way using software and/or hardware of a user's existing hardware device 102 (e.g., encrypting data in RAM, NAND, and/or other general purpose storage) with or without dedicated security hardware. In certain embodiments, the authentication module 202 may encrypt and/or secure data (e.g., electronic credentials, downloaded data) associated with a first user that is received by, processed by, and/or stored by a second (e.g., different) user's hardware device 102 (e.g., from the first user's hardware device 102 over the data network 106 or the like), preventing the second user from accessing the first user's data while still allowing the first user's data to be downloaded and/or aggregated from a different user's hardware device 102.
In one embodiment, as described above, electronic credentials may comprise one or more of a username and password, fingerprint scan, retinal scan, digital certificate, personal identification number (PIN), challenge response, security token, hardware token, software token, DNA sequence, signature, facial recognition, voice pattern recognition, bio-electric signals, two-factor authentication credentials, or other information whereby the authentication module 202 may authenticate and/or validate an identity of and/or an authorization of a user.
The authentication module 202, in certain embodiments, may receive different credentials from a user for different accounts of the user with different third party service providers 108 (e.g., different social networks, different photo sharing sites, different financial institutions) so that the aggregation module 104 may download, aggregate, and/or combine the user's data from the multiple different third party service providers 108. In one embodiment, as described below with regard to the password manager module 306 of
In one embodiment, the direct access module 204 accesses one or more servers 108 of one or more third party service providers 108, from a hardware device 102 of a user and/or from a backend server 110, using a user's electronic credentials from the authentication module 202 (e.g., for the user associated with the hardware device 102, for a different user, or the like). The direct access module 204, in certain embodiments, downloads data associated with a user (e.g., a user's social media posts, a user's photos, a user's financial transactions, or the like) from one or more servers 108 of one or more third party service providers 108 to a hardware device 102 of a user (e.g., of the user associated with the downloaded data, of a different user for processing and/or for transfer to the hardware device 102 of the user associated with the downloaded data, or the like) and/or to a backend server 110 associated with the direct access module 204, instead of or in addition to downloading the data directly to a hardware device 102 of the user (e.g., based on an availability of the hardware device 102 of the user, to backup the data in a second location, or the like).
The direct access module 204, in certain embodiments, may use a webpage interface of a server 108 of a third party service provider 108 to access the server 108 using a user's electronic credentials and/or to download data associated with the user. For example, in certain embodiments, the direct access module 204 may download/load a webpage from a server 108 of a third party service provider 108, enter a username and password or other electronic credentials for a user into textboxes in a form on the webpage, submit the username and password or other electronic credentials using a submit button or other interface element of the webpage, and/or otherwise submit electronic credentials using a website to gain authorized access to data on the server 108 associated with the user. As described below, the pattern module 308 may receive and/or provide instructions enabling the direct access module 204 to access a server 108 (e.g., a location or method for submitting electronic credentials, or the like).
In response to successfully authenticating with and accessing a server 108 of a third party service provider 108 with a user's electronic credentials, the direct access module 204 may download data associated with the user (e.g., from a user's account or the like) from the server 108, to a hardware device 102 associated with the user, to a backend server 110, to a hardware device 102 of another user downloading the data in proxy for the user, or the like. As described below, in certain embodiments, the pattern module 308 may receive and/or provide instructions enabling the direct access module 204 to download data associated with a user from a server 108 of a third party service provider 108 (e.g., a URL or other link to a location for the data, a label or other identifier for locating the data within one or more webpages or other data structures, or the like). The direct access module 204, in certain embodiments, may follow instructions from a pattern module 308 to authenticate and/or access data from one or more webpages from a server 108 in a screen scraping manner, parsing one or more webpages to locate an entry location and/or submit electronic credentials; to locate, download, and/or extract data associated with a user; or the like.
In one embodiment, the direct access module 204 sends or otherwise submits electronic credentials and/or receives or otherwise downloads data using an API or other access protocol of a server 108 of a third party service provider 108. For example, the direct access module 204 may send a request in a format specified by and/or compatible with a server 108 (e.g., an API server 108) of a third party service provider 108. The sent request may comprise electronic credentials for a user or a portion thereof (e.g., a username and/or a password), a subsequent request may comprise electronic credentials for a user or a portion thereof (e.g., in response to receiving an acknowledgment from the server 108 for the first request, or the like), and/or the direct access module 204 may use a different access protocol of a server 108.
In response to a request for data from the direct access module 204 (e.g., in response to the direct access module 204 authenticating a user using an access protocol of a server 108), a server 108 of a third party service provider 108 may send and/or return data associated with a user (e.g., in one or more messages, packets, payloads, as a URL or other pointer to a location from where the direct access module 204 may retrieve the data, or the like). The direct access module 204, in various embodiments, may receive data associated with a user directly from a server 108 of a third party service provider 108 over a data network 106; may receive a pointer, URL or other link to a location of data associated with a user from a server 108 of a third party service provider 108; may receive data associated with a user from another entity on a data network 106 (e.g., in response to a request from the server 108 of the third party service provider 108 to the other entity or the like); or may otherwise receive data associated with a user according to an access protocol of a third party service provider 108.
In one embodiment, a third party service provider 108 provides a direct access module 204 with an API or other access protocol. In a further embodiment, a direct access module 204 may act as a wrapper for and/or a plugin or extension of, an application of a third party service provider 108 (e.g., a mobile application), and the application may have access to an API or other access protocol of the third party service provider 108. In another embodiment, a direct access module 204 may be configured to use an API or other access protocol in a same manner as an application of a third party service provider 108 (e.g., a mobile application), through observation of the application of the third party service provider 108 or the like. In certain embodiments, a direct access module 204 may cooperate with an application of a third party service provider 108, a web browser through which a user accesses services of a third party service provider 108, or the like to access data associated with a user (e.g., accessing data already downloaded by an application and/or user, accessing a database or other data store of an application and/or web browser, scanning and/or screen scraping a web page of a third party service provider 108 as a user accesses the web page, or the like).
The direct access module 204, in certain embodiments, may access different third party service providers 108 in different manners. For example, a first third party service provider 108 may grant the direct access module 204 with access to an API or other access protocol, while the direct access module 204 may use a web page interface (e.g., screen scraping) to access and download data from a second third party service provider 108, or the like. In one embodiment, a remote backend server 110 may be associated with a first party service provider 110 (e.g., a vendor and/or provider of an aggregation module 104) and the direct access module 204 may download data associated with a user from both the first party service provider 110 and from one or more third party service providers 108, aggregating the data together so that the user may access the data in a single interface and/or application. For example, as described below with regard to the interface module 206, the interface module 206 may provide a user access to the user's photos from multiple third party cloud storage providers 108 within a single photo application, may provide a user with access to the user's personal financial information within a single personal financial management application and/or online banking application, may provide a user with access to posts from multiple social networks within a single social networking application, or the like.
The direct access module 204, in certain embodiments, may store downloaded and/or aggregated data independently from the one or more third party service providers 108. For example, the direct access module 204 may store a user's downloaded and/or aggregated data on a hardware device 102 of the user, on a backend server 110 accessible by the user, or the like. In this manner, in certain embodiments, a user may control and/or access the user's data, even if a third-party service provider 108 closes down or is not available, may use the user's data in any manner desired by the user even if the use is not supported by a third-party service provider 108, or the like.
The direct access module 204, in one embodiment, in addition to and/or instead of downloading data from one or more third party service providers 108, may upload data to and/or change one or more settings of one or more third party service providers 108, in response to user input or the like. For example, in embodiments where the data comprises photos, the direct access module 204 may upload a photo from a hardware device 102 of the user to one or more third party service providers 110 (e.g., a downloaded photo that the user has edited on the hardware device 102 or the like). In embodiments where the data comprises social media posts or other content, the direct access module 204 may receive input from a user (e.g., a photo, a textual post, one or more emoji, a video, a document or other file, or the like) and upload the received input to one or more third party service providers 108 (e.g., social media sites or the like). In embodiments where the data comprises financial transactions or other financial data, the direct access module 204 may schedule a bill pay or other payment or funds transfer, remotely deposit a check (e.g., by uploading photos of the front and/or back of the check, or the like), and/or perform another action.
The direct access module 204 may update or change a user's account information with a third party service provider 108, such as an account type or plan, credit card or other payment information associated with an account, a phone number or address or other contact information associated with an account, a password or other electronic credentials for an account, and/or other account information of a user for a third party service provider 108. The direct access module 204 may update and/or upload data in a substantially similar manner to that described herein for downloading data (e.g., determining a user's electronic credentials for a third party service provider 108, accessing a server 108 of the third party service provider 108, uploading and/or providing data to the third party service provider 108, or the like).
In one embodiment, the interface module 206 provides a user's data downloaded by the direct access module 204, from a hardware device 102 of a user (e.g., of the user associated with the downloaded data, of a different user) to another entity, such as a hardware device 102 of a user associated with the downloaded data (e.g., in response to the data being downloaded by a hardware device 102 of a different user, from one hardware device 102 of a user to another hardware device 102 of the same user), a remote server 110 or other remote device 102 unaffiliated with (e.g., not owned by, operated by, controlled by, or the like) the third party service provider 108 from which the data was downloaded, or the like. For example, the interface module 206 may provide an API or other interface to provide a user's downloaded and/or aggregated data to a hardware device 102 of the user, to a backend aggregation module 104b, to a backend server 110, to a different third-party service provider 108, to a different/second hardware device 102 of the user, or the like.
In certain embodiments, it may be transparent and/or substantially transparent to a user (e.g., not apparent) which hardware device 102, 110 has downloaded data associated with the user. For example, the interface module 206 may provide downloaded data associated with a user from one hardware device 102 of the user to another hardware device 102 of the user, from a hardware device 102 of the user to a backend server 110 (e.g., from which the user may access the data using a web browser, an application, or the like), from a backend server 110 to a hardware device 102 of the user, or the like, allowing the user to access the data from a different location than the location to which the data was downloaded.
In certain embodiments, the interface module 206 provides a graphical user interface (GUI) on a hardware device 102 of a user, and provides downloaded data associated with the user to the user through the GUI (e.g., allowing the user to view the data directly, providing one or more notifications and/or recommendations to the user based on the data, providing one or more tables or charts to the user based on the data, providing a summary of or one or more statistics related to the data, or the like). The interface module 206, in various embodiments, may provide a GUI to the user from the same hardware device 102 to which the data was downloaded, on a different hardware device 102 than the hardware device 102, 110 to which the data was downloaded, or the like.
For example, in one embodiment, where the data associated with a user comprises photos, the interface module 206 may provide a photo management interface, a photo editing interface, or the like wherein the user may view and/or otherwise access the user's downloaded and/or aggregated photos. In a further embodiment, where the data associated with a user comprises the user's financial transaction history (e.g., purchases and/or other financial transactions downloaded from one or more financial institutions 108 such as banks, credit unions, lenders, or the like), the interface module 206 may provide a personal financial management interface, with a list of transactions, one or more budgets, one or more financial goals, a debt management interface, a net worth interface, and/or another personal financial management interface wherein the user may view the user's downloaded and/or aggregated financial transaction history, and/or alerts or recommendations based thereon. In another embodiment, where the data associated with a user comprises social media posts, the interface module 206 may provide a GUI comprising a stream, feed, and/or wall of social media posts for the user to view (e.g., downloaded and/or aggregated social media posts from multiple social networks 108, from different contacts or friends of the user, or the like).
The interface module 206, in certain embodiments, may provide one or more access controls to a user, allowing the user to define which devices 102, users, third party service providers 110, or the like may access which data. For example, the interface module 206 may provide an interface for a user to allow and/or restrict certain mobile applications, certain APIs for third party services, certain plugins or extensions, certain users, certain hardware devices 102, and/or one or more other entities to access data downloaded for the user from one or more third party service providers 108 (e.g., with access controls by third party service provider 108 or other data source, by data type, by entity requesting access, and/or at another granularity). In this manner, the aggregation module 104, in certain embodiments, may comprise a local repository of aggregated data, which one or more other devices 102 and/or services may access and use, with a user's permission.
In one embodiment, the local authentication module 302 secures and/or authenticates the user's access to downloaded data, to stored passwords, and/or other data on a user's hardware device 102, transferred to and/or from a user's hardware device 102, or the like. For example, the local authentication module 302 may cooperate with one or more security and/or authentication systems of the user's hardware device 102, such as a PIN, password, fingerprint authentication, facial recognition, or other electronic credentials used by the user to gain access to the hardware device 102. In a further embodiment, the local authentication module 302 may authenticate a user before allowing the interface module 206 to provide the user access to downloaded/aggregated data and/or alerts or other messages. For example, the local authentication module 302 may manage and/or access electronic credentials associated with the aggregation module 104, for a user, and may authenticate the user in response to the user accessing an application and/or service of the aggregation module 104.
In certain embodiments, the local authentication module 302 may encrypt and/or otherwise secure, on a user's hardware device 102, electronic credentials and/or downloaded data associated with a different user, so that the user may not access data associated with the different user, but the different user may access the data once it is transmitted to a hardware device 102 of the different user, to a backend server 110, or the like. Local authentication modules 302 of different hardware devices 102, 110 may cooperate to securely transfer data (e.g., one or more electronic credentials, downloaded data, or the like) over the data network 106, from one hardware device 102, 110 to another hardware device 102, 110. In a further embodiment, the local authentication module 302 may ensure that a user's electronic credentials and/or downloaded data remain on a single hardware device 102 (e.g., are not transmitted on a data network 106), in a secure repository or the like, and are not stored on and/or accessible to a backend server 110, a hardware device 102 of another user, or the like.
In one embodiment, the network authentication module 304 receives and/or stores a user's electronic credentials for one or more third party service providers 108 on a hardware device 102 of the user, on a backend server 110, or the like. The network authentication module 304, in various embodiments, may receive a user's electronic credentials from the user, from a hardware device 102 of the user, from a backend server 110, or the like. The network authentication module 304 may cooperate with the direct access module 204 to provide a user's electronic credentials to a server 108 of a third party service provider 108 (e.g., the network authentication module 304 may provide electronic credentials to the direct access module 204 to provide to a server 108, the network authentication module 304 may provide electronic credentials directly to a server 108, or the like).
The network authentication module 304, in certain embodiments, may cooperate with the local authentication module 302 to encrypt and/or otherwise secure a user's electronic credentials for one or more third party service providers 108, on a hardware device 102 of a user, on a data network 106, on a hardware device 102 of a different user, on a backend server 110, while being provided to a server 108 of a third party service provider 108, or the like. In a further embodiment, the network authentication module 304 ensures that a user's electronic credentials are only stored on a user's hardware device 102 and sent from the user's hardware device 102 to a server 108 of a third-party service provider 108 and does not store a user's electronic credentials on a backend server 110, on a different user's hardware device 102, or the like. In another embodiment, the network authentication module 304 may securely store (e.g., using secure encryption) a user's electronic credentials for a third party service provider 108 on a backend server 110, on a different user's hardware device 102, or the like, so that a direct access module 204 may access and/or download data associated with the user, even if the hardware device 102 of the user is unavailable, blocked, or the like, as described below with regard to the route module 314. In certain embodiments, whether the network authentication module 304 and/or the local authentication module 302 allow electronic credentials to be sent to and/or stored by a different user's hardware device 102, a backend server 110, or the like may be based on a setting defined based on user input, so that the user may decide a level of security, or the like.
In one embodiment, the password manager module 306 may manage and/or store electronic credentials of a user for a plurality of third party service providers 108, so that the direct access module 204 may access and/or download data associated with the user from each of the plurality of third party service providers 108. The password manager module 306, in certain embodiments, may generate and/or otherwise manage different, secure, credentials for each of a plurality of third party service providers 108.
The password manager module 306, in one embodiment, may securely store generated credentials for a user on a hardware device 102 of the user, so that the user does not have to remember and enter the generated electronic credentials. For example, in addition to allowing a direct access module 204 to access a third party service provider 108 using generated electronic credentials, the password manager module 306 may automatically populate one or more interface elements of a form on a webpage with electronic credentials (e.g., a username, a password) of the user, in response to the user visiting the web page in a web browser, or the like, without the user manually entering the electronic credentials. The password manager module 306, in certain embodiments, may periodically update (e.g., regenerate different credentials, such as a different password, and update the user's account with the third party service provider 108 with the regenerated different credentials) electronic credentials for a user, such as every week, every month, every two months, every three months, every four months, every five months, every six months, every year, every two years, in response to a user request, in response to a request from a third party service provider 108, and/or over another time period or in response to another periodic trigger.
The password manager module 306, in one embodiment, may synchronize a user's electronic credentials (e.g., provided by the user, generated by the password manager module 306, or the like) across different hardware devices 102, web browsers, or the like of a user. For example, in response to a password manager module 306 and/or the user updating or otherwise changing electronic credentials, the password manager module 306 may propagate the update/change to one or more other password manager modules 306, on different hardware devices 102 of the user, or the like.
In one embodiment, the pattern module 308 determines an ordered list (e.g., a pattern, a script, or the like) of multiple locations on one or more servers 108 of a third party service provider 108 for the direct access module 204 to access the server (e.g., which may include locations other than where the data of the user is stored and/or accessible), one or more delays for the direct access module 204 to wait between accessing locations on the server 108, and/or other components of an access pattern for accessing data of a server. Locations, in certain embodiments, comprise independently addressable and/or accessible content and/or assets provided by one or more servers of a third party service provider 108, or the like, such as webpages, portions of a webpage, images or other data files, databases or other data stores, pages or sections of a mobile application, or the like. The pattern module 308, in one embodiment, determines a pattern/ordered list that contains one or more locations and/or delays that are not necessary for the direct access module 204 to access or use in order to download desired data, but instead, the pattern/ordered list may make it difficult or impossible for the third party service provider 108 to distinguish between the direct access module 204 accessing a server of the third party service provider 108 and a user accessing the server of the third party service provider.
The pattern module 308, in one embodiment, may determine and/or select the multiple locations and/or the one or more delays (e.g., a pattern/ordered list) based on an average pattern or a combined pattern identified in or based on behavior of multiple users accessing a third party service provider 108 using a web browser, a mobile application, or the like. The pattern module 308, in one embodiment, may monitor one or more users (e.g., for a predetermined period of time or the like) as they access a server of a third party service provider 108, tracking which links, data, webpages, and/or other locations the one or more users access, how long the one or more users access different locations, an order in which the one or more users access locations, or the like. In certain embodiments, the one or more monitored users may be volunteers, who have provided the pattern module 308 with authorization to temporarily or permanently monitor the users' access, in order to provide a more realistic access pattern for the direct access module 204 to use to access a server of a third-party service provider 108.
In a further embodiment, the pattern module 308 determines and/or selects multiple locations and/or one or more delays between accessing different locations based on a pattern identified in behavior of the user associated with the hardware device 102 on which the pattern module 308 is disposed, accessing the third party service using a web browser, a mobile or desktop application, or other interface of the user's hardware device 102. For example, the pattern module 308 may comprise network hardware of the user's hardware device 102 (e.g., a network access card and/or chip, a processor, an FPGA, an ASIC, or the like in communication with the data network 106 to monitor data and/or interactions with a server of a third party service provider 108), a web browser plugin or extension, a mobile and/or desktop application executing on a processor of the user's hardware device 102, or the like. The pattern module 308 may request and receive authorization from the user to monitor the user's activity with regard to one or more servers of one or more third party service providers 108 from the user's hardware device 102.
The pattern module 308, in certain embodiments, may update a pattern/ordered list over time, based on detected changes in access patterns of one or more users or the like. In one embodiment, the pattern module 308 may coordinate and/or cooperate with the access repair module 310, described below, to update a pattern/ordered list in response to a server 108 of a third party service provider 108 and/or data associated with a user becoming broken and/or inaccessible.
In one embodiment, the access repair module 310 detects that access to a server 108 of a third party service 108 and/or data associated with a user is broken and/or becomes inaccessible. The access repair module 310, in certain embodiments, provides an interface to a user allowing the user to graphically identify an input location for the user's electronic credentials, a location of data associated with the user, or the like. For example, the access repair module 310 may provide a GUI, a command line interface (CLI), an API, and/or another interface allowing an end user to identify an input location for electronic credentials, an action for submitting electronic credentials, a location of data, or the like. The access repair module 310, in one embodiment, provides an interface to a user on a hardware device 102 of the user.
In certain embodiments, for example, the access repair module 310 may overlay an interface over one or more pages of a website of a third-party service provider 108 on an electronic display screen of a user's hardware device 102, as described in greater detail below with regard to
The access repair module 310, in certain embodiments, may determine and/or display one or more suggestions 504 and/or recommendations 504 for the user, which the user may either confirm or change/correct (e.g., in a basic interface, a standard interface, a beginning user interface, or the like). For example, the access repair module 310 may display one or more interface elements with a suggested location for a user to enter a user name, a suggested location for a user to enter a password, a suggested credential submit action, a suggested location of data associated with the user, and/or one or more other interface elements allowing a user to graphically identify one or more locations within a website of a third party service provider 108.
The access repair module 310, in certain embodiments, processes one or more pages of and/or other locations on a server 108 (e.g., one or more websites, web apps, or the like) to determine an estimate and/or prediction of an input location for a user's electronic credentials, an action for submitting a user's electronic credentials, a location of data associated with a user, or the like. In one embodiment, the access repair module 310 may estimate one or more locations and/or actions (e.g., by scanning and/or parsing one or more pages of a web site, based on input from other users accessing one or more pages of a web site, based on previous interactions of the user with one or more pages of a web site, a prediction made using a machine learning and/or artificial intelligence analysis of a website, based on a statistical analysis of historical changes to one or more pages of a website and/or of one or more similar websites, or the like). The access repair module 310 may display to a user in an interface an estimate and/or prediction of an input location for the user's electronic credentials, a location of data associated with the user, or the like so that the user may confirm whether or not the estimate and/or prediction is correct using the interface.
The access repair module 310 may indicate one or more estimated locations and/or actions with an arrow or other pointer to a location; a link or other identifier of a location; a box or other highlighting around a location; by altering text labeling for a location to make the text bold, italic, and/or underlined; or the like. A user, in certain embodiments, may click, select, or otherwise identify a location to either confirm or change/correct a location suggested by the access repair module 310. For example, a user may click or otherwise select an interface element associated with a location and/or action and may click or otherwise select the location and/or perform the action, which the access repair module 310 may record (e.g., automatically populating a text field identifying the location and/or action, recording a macro allowing the action to be automatically repeated without the user, for a different user, or the like).
In certain embodiments, instead of or in addition to a standard, basic, or beginning user interface, the access repair module 310 may provide an advanced interface, for experienced users or the like, with source code of a website and/or other details of the website. For example, in one embodiment, an advanced access repair interface may allow one or more advanced users to identify one or more locations and/or actions within source code of a website, which may not be visible and/or readily apparent in the website itself. In certain embodiments, the access repair module 310 may provide a user interface element allowing a user to select and/or toggle between a standard user interface or view and an advanced user interface or view.
In one embodiment, the test module 318 cooperates with the access repair module 310 to verify whether or not one or more received locations and/or instructions from a user are accurate (e.g., usable to access data from a server of a third-party service provider 108). The test module 318, in certain embodiments, attempts to access a server 108 of a third party service provider 108 for a plurality of different users (e.g., a sample group or test set), based on an identification the access repair module 310 received from a single user, using electronic credentials of the different users or the like.
The test module 318, in certain embodiments, determines whether data associated with the different users (e.g., a sample group or test set) is accessible using the identification from the single user. The test module 318 may repeatedly attempt to access data from a third party service provider 108 using identifications which the access repair module 310 received from different users (e.g., on different hardware devices 102 and sent to the test module 318 on a single hardware device 102 over the data network 106, sent to multiple test modules 318 on different hardware devices 102 over the data network 106, sent to a test module 318 on a central backend server 110, or the like).
The test module 318, in one embodiment, provides one or more identifications from a user to other instances of the direct access module 204 (e.g., other test modules 318) for accessing a server 108 of a third party service provider 108 in response to an amount of the different users (e.g., a sample group or test set) for which data is accessible using the identification from the single user satisfying a threshold. For example, if the identification from the single user successfully allows a predefined number of other test users (e.g., 2 users, 10 users, 100 users, 1000 users, 50% of test users, 75% of test users, and/or another predefined threshold number of test users) to access their data from a third party service provider 108, the test module 318 may provide instructions based on the identification to more users (e.g., all or substantially all users, or the like).
In certain embodiments, the test module 318 may successively increase a test size comprising a number of users to which the test module 318 provides instructions for accessing their data from a third party service provider 108 using an identification from a single user (e.g., starting with one or more test users, increasing to two or more, three or more, four or more, five or more, ten or more, twenty or more, thirty or more, forty or more, fifty or more, one hundred or more, five hundred or more, one thousand or more, five thousand or more, ten thousand or more, one hundred thousand or more, a million or more, and/or other successively increasing numbers of test users). The test module 318, in one embodiment, includes instructions based on an identification from a single user in an ordered list of multiple different sets of instructions for accessing a server 108 of a third-party service provider 108, as described in greater detail below with regard to the hierarchy module 312.
The test module 318, in certain embodiments, is configured to prioritize identifications from one or more users based on one or more trust factors for the one or more users (e.g., scores or the like). A trust factor, in one embodiment, may comprise a score or other metadata indicating a likelihood that a user's identification is correct. For example, in various embodiments, a trust factor may include and/or be based on one or more of a history of a user's previous identifications (e.g., correct or incorrect), a user's affiliation with a provider (e.g., a creator, a vendor, an owner, a seller, a reseller, a manufacturer, the backend server 110, or the like) of the one or more aggregation modules 104, positive and/or negative indicators (e.g., votes, likes, uses, feedback, stars, endorsements, or the like) from other users, and/or other indicators of whether or not a user's identification is likely to be correct. The test module 318 may determine how many other users to provide a user's identification based on one or more trust factors associated with the user (e.g., accelerating a rate at which a user's identification is provided to other users in response to a higher trust factor, decreasing a rate at which a user's identification is provided to other users in response to a lower trust factor, or the like).
The test module 318 may provide an override interface, allowing an administrator, moderator user, or the like to remove an identification, adjust and/or override an identification, adjust and/or override a trust factor for a user, ban a user from providing identifications, and/or otherwise override a user or a user's identification. In various embodiments, the test module 318 may provide an override interface to an administrator and/or moderator as a GUI, an API, a CLI, or the like.
In certain embodiments, the test module 318 causes the one or more aggregation modules 104 and their aggregation services to be self-healing, self-testing, and/or self incrementally deploying, as it tests and uses the most effective solutions, or the like (e.g., sets of instructions based on indications from one or more users).
In one embodiment, the hierarchy module 312 provides the direct access module 204 with an ordered list of multiple different sets of instructions for accessing a server 108 of a third party service provider 108 using a user's electronic credentials, for downloading data associated with the user, or the like. Each different set of instructions, in certain embodiments, comprises a location for entering a user's electronic credentials, an instruction for submitting the user's electronic credentials, one or more locations of the data associated with the user, or the like.
The hierarchy module 312, in one embodiment, may receive one or more sets of instructions from a backend server 110 (e.g., a backend aggregation module 104b of a backend server 110), from another user hardware device 102 in a peer-to-peer manner (e.g., an aggregation module 104a of a user hardware device 102), from a test module 318, or the like. The hierarchy module 312, in certain embodiments, may receive multiple different sets of instructions already in an ordered list (e.g., a global hierarchical order) based on a history of successful and/or unsuccessful uses of the different sets of instructions by different user hardware devices 102 and/or users, or the like. In one embodiment, the hierarchy module 312 may determine a hierarchy for and/or create an ordered list from multiple different sets of instructions for a single user (e.g., a custom or individualized hierarchy) based on a history of successful and/or unsuccessful uses of the different sets of instructions by the user (e.g., from one or more hardware devices 102 of the user).
The direct access module 104, in one embodiment, may iterate through an ordered list of multiple sets of instructions for accessing a server 108 of a third party service provider 108, in the order of the list, until one of the sets of instructions is successful and the direct access module 104 is able to access and/or download data from the third party service provider 108. The hierarchy module 312, in one embodiment, may place a most recent successfully used set of instructions at the top (e.g., as the first set to try). For example, the hierarchy module 312 for a user's hardware device 102 may place a set of instructions for accessing a third party service provider 108 at the top of a list (e.g., adjusting an order of the list over time) in response to the direct access module 204 successfully accessing and/or downloading data from the third party service provider 108 using the set of instructions. In certain embodiments, the hierarchy module 312 may receive an ordered list of multiple different sets of instructions for accessing a server 108 of a third party service provider 108 in a first order (e.g., a global order) and may dynamically adjust and/or rearrange the different sets of instructions over time based on a single user's/hardware device 102's use (e.g., moving a set of instructions up in the list if access using the set of instructions is successful for the user/hardware device 102, moving a set of instructions down in the list if access using the set of instructions is unsuccessful for the user/hardware device 102, or the like).
The hierarchy module 312, in certain embodiments, may be configured to share one or more sets of instructions, an ordered list of multiple sets of instructions, or the like with a hierarchy module 312 of another user's hardware device 102 over a data network 106 (e.g., directly to the other user's hardware device 102 in a peer-to-peer manner, indirectly by way of a backend aggregation module 104b of a backend server 110, or the like). Different sets of instructions may be successful or unsuccessful for different users, in various embodiments, due to different account types, different account settings, different originating systems (e.g., due to a corporate acquisition or the like, different users of the same third party service provider 108 may have one or more different settings, different access methods, or the like), system changes or upgrades, and/or another difference in accounts, services, or the like for different users of the same third party service provider 108.
In one embodiment, the route module 314 determines whether a hardware device 102 of a user is available for the direct access module 204 to download data associated with the user from a server 108 of a third party service provider 108. The route module 314, in certain embodiments, may access a server 108 of a third party service provider 108, from a remote backend server 110, using the user's electronic credentials, to download data associated with the user from the server 108 to the remote backend server 110 in response to the route module 314 determining that the hardware device 102 of the user is unavailable. The route module 314, in one embodiment, provides a user one or more alerts (e.g., downloaded data from a third party service provider 108, a recommendation or suggestion determined based on data from a third party service provider 108, a notification or other alert based on an event or other trigger detected in data from a third party service provider 108, or the like) on a hardware device 102 of the user based on the data associated with the user downloaded to the remote backend server 110.
In certain embodiments, the route module 314 maintains and/or stores a list of multiple hardware devices 102 associated with a single user and/or account. In response to determining that one hardware device 102 associated with a user and/or account is unavailable (e.g., powered down, in airplane mode, not connected to the data network 106, or the like), the route module 314 may access a server 108 of a third party service provider 108 from a different, available hardware device 102 of the user and/or account, may provide one or more notifications or other alerts on a different, available hardware device 102, or the like. The route module 314, in various embodiments as described below with regard to
The route module 314, in one embodiment, may alternate or rotate between multiple hardware devices 102, 110 (e.g., of the same user, of different users, or the like) for downloading data for the same user from a third party service provider 108 periodically. For example, rotating and/or alternating devices 102, 110 from which data is downloaded, may decrease a likelihood that the downloading will be misinterpreted as fraudulent or improper. In another embodiment, the route module 314 may download data from the same device 102, 110 (e.g., a primary hardware device 102 of a user, a backend server 110, or the like), which may be authorized and/or identified by the third party service provider 108 as a trusted device, or the like.
In one embodiment, the frequency module 316 sets a frequency with which the direct access module 204 accesses the server 108 of a third party service provider 108. The frequency module 316, in certain embodiments, determines a frequency based on input from a remote backend server 110, which may be unaffiliated with the third party service provider 108 being accessed, so that the remote backend server 110 (e.g., the frequency module 316 executing on the remote backend server 110) determines frequencies for a plurality of direct access modules 204 for different users and/or different hardware devices 102. For example, the frequency module 316 may limit a single user and/or hardware device 102 from accessing the same third-party service provider 108 more than an allowed threshold number of times within a time period (e.g., once every ten minutes, once every half an hour, once every hour, twice a day, three times a day, four times a day, or the like). The frequency module 316, in certain embodiments, limits an access frequency to prevent inadvertent denial of service by a third party service provider 108, or the like.
The frequency module 316, in certain embodiments, may dynamically adjust a frequency with which a user and/or hardware device 102 may access a third party service provider 108 over time. For example, the frequency module 316 may monitor access and/or downloads by multiple users (e.g., all users, available users, active users, or the like) to cap or limit a total access and/or download bandwidth for each of the different third party service providers 108 (e.g., so as not to overwhelm any single third party service provider 108, or the like). In this manner, in one embodiment, a user and/or hardware device 102 may access and/or download data with a higher frequency when fewer other users and/or hardware devices 102 are accessing and/or downloading data (e.g., low peak times), but may be limited to a lower cap or access frequency when more other users and/or hardware devices 102 are accessing and/or downloading data (e.g., high peak times).
In a further embodiment, the frequency module 316 determines a frequency based on input from a user, allowing the user to set the access frequency independently of other users and/or of a backend server 110. The frequency module 316 may provide a user interface (e.g., a GUI, CLI, API, or the like) allowing a user to set and/or adjust an access frequency for downloading data from one or more third party service providers 108 using one or more hardware devices 102 (e.g., providing different settings allowing the user to set different access frequencies for different third party service providers 108, different hardware devices 102 of the user, or the like).
For example, the second user's hardware device 102b may download data for the first user in response to the first user's hardware device 102a being powered off, being asleep, being blocked from accessing one or more third party service providers 108, or the like, as determined by a route module 314, or the like. The interface module 206 of the second aggregation module 104a may provide one or more alerts/messages to the first user based on the downloaded data and/or may provide the downloaded data to the first user (e.g., in response to the first user's hardware device 102a becoming available, to a different hardware device 102 associated with the first user, to a backend server 110 to which the first user has access, or the like). As described above, in certain embodiments, the authentication module 202, the direct access module 204, the interface module 206, and/or the route module 314 may encrypt and/or otherwise secure data for the first user (e.g., the first user's electronic credentials, downloaded data associated with the first user, alerts/messages for the first user), so that it is difficult or impossible for the second user to access the data for the first user, thereby preventing and/or minimizing unauthorized access to the first user's data while providing greater flexibility in devices 102 and/or locations from which data for the first user may be downloaded.
For example, the backend server 110 may download data for the user in response to the user's hardware device 102a being powered off, being asleep, being blocked from accessing one or more third party service providers 108, or the like, as determined by a route module 314, or the like. The interface module 206 of the backend aggregation module 104b may provide one or more alerts/messages to the user based on the downloaded data and/or may provide the downloaded data to the user (e.g., in response to the user's hardware device 102a becoming available, to a different hardware device 102 associated with the first user, directly from the backend server 110 as a web page and/or through a dedicated application, or the like).
In the depicted embodiment, the access repair module 310 overlays an interface 502 over one or more pages of a website 506 of a third party service provider 108 on an electronic display screen of a user's hardware device 102. As described above, in various embodiments, the access repair module 310 may comprise a browser plugin and/or extension which provides an interface 502 within an internet browser, may comprise an embedded browser within an application of the access repair module 310, or may otherwise be integrated with and/or in communication with an internet browser.
The access repair module 310, in the depicted embodiments, determines and/or displays one or more suggestions 504 and/or recommendations 504 for the user, which the user may either confirm or change/correct. For example, the access repair module 310 may display an interface element 504a with a suggested location for the user to enter a user name, an interface element 504b with a suggested location for the user to enter a password, an interface element 504c with a suggested credential submit action, an interface element 504d with a suggested location of data associated with the user, and/or one or more other interface elements allowing a user to graphically identify one or more locations within a website 506 of a third party service provider 108.
In one embodiment, an interface element 504 may include one or more identifiers of an estimated location and/or action which the access repair module 310 has determined (e.g., by scanning and/or parsing one or more pages of a website 506, based on input from other users accessing one or more pages of a website 506, based on previous interactions of the user with one or more pages of a website 506, a prediction made using a machine learning and/or artificial intelligence analysis of a website 506, based on a statistical analysis of historical changes to one or more pages of a website 506 and/or of one or more similar websites, or the like), such as an arrow or other pointer to a location; a link or other identifier of a location; a box or other highlighting around a location; altering text labeling for a location to make the text bold, italic, and/or underlined; or the like. A user, in certain embodiments, may click, select, or otherwise identify a location to either confirm or change/correct a location suggested by the access repair module 310. For example, a user may click or otherwise select an interface element 504 associated with a location and/or action (e.g., to activate the selected interface element 504) and may click or otherwise select the location and/or perform the action, which the access repair module 310 may record (e.g., automatically populating a text field identifying the location and/or action, recording a macro allowing the action to be automatically repeated without the user, or the like).
The user, in one embodiment, may interact with the website 506 in order to locate and/or identify one or more locations, perform one or more actions, or the like. For example, in certain embodiments, the user may navigate to one or more different pages within the website 506, may login to the website 506 using the user's electronic credentials for the website 506, may navigate to a different website 506, may navigate to and/or download data associated with the user from the website 506, may use the website 506 in a usual manner, or the like. As described above with regard to the pattern module 308, the pattern module 308, in one embodiment, may monitor the user's access pattern for the website 506, allowing the direct access module 204 to at least partially emulate the user's access pattern in accessing the website 506, downloading data associated with the user from the website 506, or the like. In the depicted embodiment, the access repair module 310 (and/or an associated browser) displays a browser view of the website 506, with text, images, and/or other elements displayed substantially how an internet browser would display the web site 506, with the addition of the interface 502 displayed over the web site 506, to one side of the website 506, or the like.
In the depicted embodiment, the access repair module 310 displays a user interface 512 over and/or adjacent to the displayed source code 516, with one or more interface elements 514a-d allowing a user to identify one or more locations, actions, or the like substantially as described above. The access repair module 310, in the depicted embodiment, displays one or more suggestions and/or estimates of locations and/or actions, which the user may confirm and/or change/correct. In various embodiments, a user may identify a location and/or an action in the source code 516 by selecting and/or activating an interface element 514 and selecting a portion of the source code 516, by dragging a portion of the source code 516 and dropping the portion onto an interface element 514, by cutting and pasting a portion of the source code 516 into an interface element 514, and/or otherwise identifying a location and/or an action based on the source code 516.
In response to a user identifying one or more locations and/or actions (e.g., for entering, submitting, and/or sending electronic credentials; for locating and/or downloading data; or the like), in certain embodiments, the access repair module 310 may cooperate with the test module 318 to perform a live and/or real-time test of the identified one or more locations and/or actions, to determine the validity and/or effectiveness of the identified one or more locations and/or actions while the interface 500, 510 is visible to and/or in use by the user, allowing the user to change and/or correct provided information during the same session. For example, the access repair module 310 may display a test button or other user interface element to a user, which the user may select and/or activate to initiate a test. In another embodiment, the access repair module 310 may automatically perform a test in response to a user providing a location and/or action, without the user selecting and/or activating a test button or other user interface element. In a further embodiment, the test module 318 may perform one or more tests independent of the access repair module 310, with or without testing functionality of the access repair module 310.
A direct access module 204 aggregates 708 the downloaded 706 data from the plurality of different third party service providers 108. An interface module 206 provides 710 the aggregated 708 data to the user (e.g., displaying the data on a hardware device 102 of the user, sending an alert or other message to a hardware device 102 of the user, sending the data to a remote backend server 110 unaffiliated with the third party service providers 108 which the user may access using a web interface and/or API, or the like) and the method 700 ends.
An access repair module 310 determines 806 whether or not there is a change in access for the one or more third party service providers 108 (e.g., whether access is broken or unavailable, whether access is partial or incomplete, whether access bandwidth is slower than previously determined, and/or whether another change in access has occurred). If the access repair module 310 determines 806 that access for a third party service provider 108 has changed, the access repair module 310 provides 808 a graphical user interface 500, 510 to the user. The access repair module 310 receives 810, through the provided 808 graphical user interface 500, 510, an identification of one or more locations and/or actions for authenticating the user and/or downloading data from the third party service provider 108. The test module 318 tests 812 access to the third party service provider 108 using the received 810 identification of one or more locations and/or actions. In response to successful testing 812 by the test module 318, the test module 318 and/or the pattern module 308 provide 814 instructions for accessing and/or downloading data from the third party service provider 108 based on the received 810 identification of one or more locations and/or actions to one or more direct access modules 204 associated with one or more different users.
A route module 314 determines 816 whether a hardware device 102 associated with the user is available. In response to the route module 314 determining 816 that a hardware device 102 associated with the user is available, a direct access module 204 downloads 818 data associated with the user from one or more third party service providers 108 from the available hardware device 102 associated with the user.
In response to the route module 314 determining 816 that a hardware device 102 associated with the user is not available, a direct access module 204 of a different device (e.g., a hardware device 102 of a different user, a backend server 110, or the like) downloads 820 data associated with the user from one or more third party service providers 108 from the different device. A route module 314 (e.g., on a different device 102, 110) determines 822 whether an alert or other message is available for the user based on the downloaded 820 data and pushes 824 and/or otherwise sends the alert or other message to a device 102 associated with the user (e.g., an unavailable device 102) in response to determining 822 that the alert or other message is available. For example, in one embodiment, a hardware device 102 of a user may be unavailable for downloading data (e.g., powered down, offline, asleep, using mobile data instead of Wi-Fi, or the like), but may receive a pushed 824 alert or other message anyway (e.g., over a different channel, such as a text message, a voicemail, an email, a push notification, or the like) and/or may receive a pushed 824 alert or other message in response to becoming available at a later time.
An interface module 206 provides 826 the downloaded 818, 820 data and/or the pushed 824 alert to the user (e.g., displaying the data on a hardware device 102 of the user, displaying a pushed/sent 824 alert or other message on a hardware device 102 of the user, sending the data to a remote backend server 110 unaffiliated with the third party service provider 108 which the user may access using a web interface and/or API, or the like). The method 800, in certain embodiments, continues, periodically determining 806 whether there is a change in access for a third party service provider 108, determining 816 whether a hardware device 102 of the user is available, downloading 818, 820 data associated with the user, and/or providing 826 downloaded data and/or a pushed 824 alert or other message to the user, or the like.
In one embodiment, the content module 902 is configured to identify data to be presented in a graphical interface. In certain embodiments, the data may comprise text, images, video, documents, audio, and/or the like. For instance, the data may comprise personal identification data, financial data, social media data, proprietary data, and/or the like. For example, the data may include transaction amounts, monetary values and/or totals, monetary transfer amounts, financial account balances, dates, social security numbers, account numbers, identification numbers, and/or the like. In such an embodiment, the data may be received, downloaded, retrieved, or the like from an aggregation server, such as a backend server 110, that aggregates user data from a plurality of different third-party servers 108.
The content module 902 may determine the data that will be presented in the graphical interface prior to the graphical interface being displayed. For instance, the content module 902 may be part of or in communication with a graphics or video card of a device 102 and may intercept the graphical interface prior to the graphical interface being presented on a graphical display to determine the data that will be displayed on the graphical display.
In one embodiment, the watermark module 904 is configured to generate a digital watermark to be presented in the graphical interface based on the data that is identified to be presented in the graphical interface. As used herein, a digital watermark comprises a pattern, an image, text, a series of bits, or the like that is embedded in a digital image, audio, video, or the like and verifies the authenticity of the data that will be presented in the graphical interface. In certain embodiments, the digital watermark is embedded in the graphical interface so that it is indistinguishable, hidden, or invisible to a user, e.g., so that the user cannot notice the watermark using the naked eye but is detectable and extractable using image processing methods on a computing device.
In one embodiment, the watermark module 904 generates the digital watermark by encoding at least a portion of the data to be presented in the graphical interface into the digital watermark. For example, if the data that will be presented in the graphical interface comprises financial data such as transaction amounts, the watermark module 904 may encode at least a portion of the transaction amounts into the digital watermark so that when a screenshot, printout, or other copy of the graphical interface is made, the digital watermark can be used to verify that the transaction amounts that are presented in the graphical interface have not been changed, altered, or otherwise tampered with by comparing the presented transaction amounts with the transaction amounts that are encoded in the digital watermark.
The watermark module 904, for instance, may generate sets of bits or bytes of the data, may encrypt the encoded data, and/or may otherwise manipulate the data to generate an encoding scheme that can be used as a digital watermark in the graphical interface. In certain embodiments, the watermark module 904 may determine a particular order of the bits/bytes, may generate a random order of the bits/bytes, may arrange the bits/bytes in a pattern, and/or the like for the digital watermark. The watermark module 904 may further encode information used to decode the digital watermark such as the encryption scheme used, the order of the bits/bytes, a location within the graphical interface where the digital watermark is located, where portions of the digital watermark are located, and/or the like.
In one embodiment, the presentation module 906 is configured to embed the digital watermark into the graphical interface prior to the data being presented in the graphical interface such that the digital watermark is graphically indistinguishable to a user in the graphical interface. In certain embodiments, the presentation module 906 embeds the digital watermark into the graphical interface using various image processing techniques. For instance, additional data may be added into, or existing data may be changed for one or more channels (e.g., red, green, blue, opacity, or the like channels) of an image's pixels. For example, a least significant bit (“LSB”) method may be used where the color channels of each pixel of an image, e.g., the red, green, and blue channels, are allocated three bytes with values ranging from 0 to 255. So, if a pixel is purple, the channels would look like (R=255, G=0, B=255). Digital watermark data may be encoded into this pixel by slightly changing the LSB of the blue channel so that the pixel channel values are (R=255, G=0, B=254). This slight change is not noticeable to the human eye and may be indistinguishable from other pixels that are adjacent to this pixel. Thus, in this example, for every pixel of three bytes, three bits of watermarking information can be hidden in the LSBs of each channel. In the foregoing example, to decode the digital watermark information, the data in the LSBs for each pixel may be combined to arrive at the data that is encoded in the digital watermark.
Other methods, algorithms, techniques, or the like for embedding the digital watermark may include using a secret key to choose a random set of bits to replace with the encoded data of the digital watermark, converting the pixel data to the frequency domain and selecting pixels based on the frequency to change or alter with the encoded data of the digital watermark (e.g., using a Fast Fourier Transform and applying the digital watermark to the lower frequency pixels), or the like. One of skill in the art, in light of this disclosure, will recognize other image processing techniques for embedding the digital watermark into the graphical interface.
In one embodiment, the presentation module 906 is further configured to embed the digital watermark into the graphical interface at a location of the graphical interface that is a predetermined distance from the at least a portion of the data that is encoded into the digital watermark. For example, if the data includes a column of transaction amounts, the digital watermark that includes the encoded transactional amounts may be placed at a location that is a predetermined number of pixels away from the column, that is a percentage of the viewing area of the graphical display away from the column, and/or the like. In this manner, the chances of the digital watermark being changed or altered are reduced if the column of transactional data is changed or altered.
In one embodiment, the presentation module 906 embeds the digital watermark into an interface element that is presented as part of the graphical interface. For instance, the digital watermark may be embedded in a button, image, background, menu, or the like of the graphical interface so that the digital watermark is indistinguishable from the interface element. For example, as described above, various values of the pixels of the digital watermark may be slightly altered to store the encoded data without making the digital watermark visible or noticeable to a user. In certain embodiments, portions of the digital watermark may be stored in a plurality of interface elements of the graphical display, and then reassembled or combined when decoded.
In certain embodiments, the watermark module 904 is further configured to dynamically update the digital watermark in response to detecting a change in the at least a portion of the data that is encoded into the digital watermark. For instance, if the watermark module 904 detects changes in transaction amounts, personal information, account information, or the like that is currently presented in the graphical interface, the watermark module 904 may update a current digital watermark embedded in the graphical interface, may generate a new digital watermark to embed in the graphical interface, and/or the like.
In such an embodiment, the presentation module 906 is further configured to re-embed the digital watermark into the graphical interface in response to the digital watermark being updated. The presentation module 906, for instance, may determine a location for the digital watermark (which may be the same or different from the location of a previously-used watermark); may determine one or more interface elements to embed the digital watermark in (which may be the same or different than previously-used interface elements); and/or the like prior to the updated data being displayed in the graphical interface on a graphical display.
In one embodiment, the presentation module 906 makes the digital watermark graphically indistinguishable to the user by sizing the digital watermark in the graphical interface so that the user cannot see the digital watermark (e.g., using pixels that are adjacent to one another; limiting the digital watermark to a predefined region of the graphical interface; or the like); by coloring the digital watermark so that the digital watermark blends into the graphical interface where the digital watermark is located, as described above; and by using a pattern that is indistinguishable, to the user, from the graphical interface (e.g., embedding the digital watermark in a border, in text, and/or in other interface graphics such that the digital watermark appears to be part of the graphical interface).
In one embodiment, the verification module 908 is configured to verify an authenticity of the data presented in the graphical interface using the digital watermark embedded into the graphical interface. For instance, the verification module 908 may be configured to receive a copy of the graphical interface with the embedded digital watermark (e.g., a screenshot of the graphical interface, an image of a printout of the graphical interface, a PDF or other version of the graphical interface, or the like), identify the data presented on the copy of the graphical interface (e.g., using optical character recognition, or the like), decode the digital watermark embedded into the copy of the graphical interface (e.g., depending on the embedding algorithm that was used such as the LSB method, a frequency transform of the graphical interface, or the like) to get the original data that was encoded into the digital watermark, and verify the authenticity of the data presented on the copy of the graphical interface based on the decoded data from the digital watermark. If the decoded data does not match the data presented in the graphical interface, then the verification module 908 may determine that the data presented in the graphical interface has been tampered with and is fraudulent.
For example, if transactional data that is presented in the graphical interface was encoded into the digital watermark, and the verification module 908 determines that the transactional data presented in the copy of the graphical interface does not match the transactional data that is encoded into the digital watermark when it is decoded, then the verification module 908 may take an action such as notify a user, may send an alert, may lock a user out of the system, or the like because the data mismatch may indicate that the transactional data in the copy of the graphical interface has been tampered with.
In such an embodiment, the digital watermark remains graphically indistinguishable to the user in a duplicate version of the graphical interface. For instance, a duplicate version of the graphical interface may include a screenshot of the graphical interface, a printout of the graphical interface, and a version of the graphical interface that is converted to a document. For example, if a user captures a screenshot of the graphical interface, the digital watermark information will still be extractable from the screenshot while remaining indistinguishable to the user.
Similarly, if a user captures an image of a printout or other copy of the graphical interface, the digital watermark information may still be extractable from the capture. In such an embodiment, the verification module 908 is further configured to adjust one or more image settings such as an alignment, a brightness, and a white balance of a received copy of graphical interface in order to identify, extract, and decode the digital watermark.
In certain embodiments, the verification module 908 is further configured to present an interface for submitting the copy of the graphical interface. For example, a user may submit a screenshot of a graphical interface that contains financial and personal information to a loan officer, a mortgage broker, a bank, and/or the like using a web interface, a mobile application interface, a shared storage location, an API, a command line interface, or the like that the verification module 908 provides. The verification module 908 may then analyze the graphical interface to extract the digital watermark and compare (e.g., a bit-by-bit compare) data decoded from the digital watermark to data presented in the graphical interface to verify the authenticity of the data and provide a recommendation, notification, alert, or the like to the loan office, mortgage broker, bank, or the like based on whether the data was authenticated or not.
In one embodiment, the presentation module 906 embeds 1108 the digital watermark into the graphical interface prior to the data being presented in the graphical interface such that the digital watermark is graphically indistinguishable to a user in the graphical interface. The watermark module 904, in certain embodiments, determines 1110 whether the data that is presented in the graphical interface has changed (e.g., financial data is changed, added, or removed). If not, the watermark module 904 continues to monitor the graphical interface for changes in the presented data. Otherwise, the watermark module 904 generates 1104 a digital watermark based on the changed data, and the method 1100 ends.
In certain embodiments, the verification module 908 identifies 1206 the data presented on the copy of the graphical interface (e.g., using OCR or other character or object recognition method). In further embodiments, the verification module 908 decodes 1208 the digital watermark embedded into the copy of the graphical interface (e.g., by locating the digital watermark in the graphical interface and decoding the watermark (using an LSB algorithm, a frequency transformation algorithm, or the like) to determine the data that was encoded in the digital watermark). The verification module 908, in certain embodiments, verifies 1210 the authenticity of the data presented on the copy of the graphical interface based on the decoded digital watermark (e.g., by comparing the decoded data to the data presented on the copy of the graphical interface to determine if there is a mismatch, and if so, that the data presented on the copy of the graphical interface is not authentic or has been altered), and the method 1200 ends.
A means for determining a user's electronic credentials for a third party service provider 108 on a hardware device 102 of the user, in various embodiments, may include one or more of a hardware device 102, a backend server 110, an authentication module 202, a local authentication module 302, a network authentication module 304, a password manager module 306, an aggregation module 104, a processor (e.g., a central processing unit (CPU), a processor core, a field programmable gate array (FPGA) or other programmable logic, an application specific integrated circuit (ASIC), a controller, a microcontroller, and/or another semiconductor integrated circuit device), an HDMI or other electronic display dongle, a hardware appliance or other hardware device, other logic hardware, and/or other executable code stored on a computer readable storage medium. Other embodiments may include similar or equivalent means for determining a user's electronic credentials for a third party service provider 108 on a hardware device 102 of the user.
A means for accessing a server 108 of a third party service provider 108, from a hardware device 102 of a user, using the user's electronic credentials, in various embodiments, may include one or more of a hardware device 102, a backend server 110, a direct access module 204, a pattern module 308, an access repair module 310, a hierarchy module 312, an aggregation module 104, a network interface, a processor (e.g., a central processing unit (CPU), a processor core, a field programmable gate array (FPGA) or other programmable logic, an application specific integrated circuit (ASIC), a controller, a microcontroller, and/or another semiconductor integrated circuit device), an HDMI or other electronic display dongle, a hardware appliance or other hardware device, other logic hardware, and/or other executable code stored on a computer readable storage medium. Other embodiments may include similar or equivalent means for accessing a server 108 of a third party service provider 108, from a hardware device 102 of a user, using the user's electronic credentials.
A means for downloading data associated with a user from a server 108 of a third party service provider 108 to a hardware device 102 of the user, in various embodiments, may include one or more of a hardware device 102, a backend server 110, a direct access module 204, a pattern module 308, an access repair module 310, a hierarchy module 312, an aggregation module 104, a network interface, a processor (e.g., a central processing unit (CPU), a processor core, a field programmable gate array (FPGA) or other programmable logic, an application specific integrated circuit (ASIC), a controller, a microcontroller, and/or another semiconductor integrated circuit device), an HDMI or other electronic display dongle, a hardware appliance or other hardware device, other logic hardware, and/or other executable code stored on a computer readable storage medium. Other embodiments may include similar or equivalent means for downloading data associated with a user from a server 108 of a third party service provider 108 to a hardware device 102 of the user.
A means for packaging downloaded data from a hardware device 102 of a user for a remote device 110, 102 unaffiliated with a third party service provider 108 from which the data was downloaded, in various embodiments, may include one or more of a hardware device 102, a backend server 110, an interface module 206, an aggregation module 104, a processor (e.g., a central processing unit (CPU), a processor core, a field programmable gate array (FPGA) or other programmable logic, an application specific integrated circuit (ASIC), a controller, a microcontroller, and/or another semiconductor integrated circuit device), an HDMI or other electronic display dongle, a hardware appliance or other hardware device, other logic hardware, and/or other executable code stored on a computer readable storage medium. Other embodiments may include similar or equivalent means for packaging downloaded data from a hardware device 102 of a user for a remote device 110, 102 unaffiliated with a third party service provider 108 from which the data was downloaded.
A means for providing downloaded data from a hardware device 102 of a user to a remote device 110, 102 unaffiliated with a third party service provider 108 from which the data was downloaded, in various embodiments, may include one or more of a hardware device 102, a backend server 110, an interface module 206, an aggregation module 104, a processor (e.g., a central processing unit (CPU), a processor core, a field programmable gate array (FPGA) or other programmable logic, an application specific integrated circuit (ASIC), a controller, a microcontroller, and/or another semiconductor integrated circuit device), an HDMI or other electronic display dongle, a hardware appliance or other hardware device, other logic hardware, and/or other executable code stored on a computer readable storage medium. Other embodiments may include similar or equivalent means for providing downloaded data from a hardware device 102 of a user to a remote device 110, 102 unaffiliated with a third party service provider 108 from which the data was downloaded.
A means for identifying data to be presented in a graphical interface, in various embodiments, may include one or more of a hardware device 102, a backend server 110, a content module 902, a security module 120, an aggregation module 104, a processor (e.g., a central processing unit (CPU), a processor core, a field programmable gate array (FPGA) or other programmable logic, an application specific integrated circuit (ASIC), a controller, a microcontroller, and/or another semiconductor integrated circuit device), an HDMI or other electronic display dongle, a hardware appliance or other hardware device, other logic hardware, and/or other executable code stored on a computer readable storage medium. Other embodiments may include similar or equivalent means for identifying data to be presented in a graphical interface.
A means for generating a digital watermark to be presented in the graphical interface based on the identified data, in various embodiments, may include one or more of a hardware device 102, a backend server 110, a watermark module 904, a security module 120, an aggregation module 104, a processor (e.g., a central processing unit (CPU), a processor core, a field programmable gate array (FPGA) or other programmable logic, an application specific integrated circuit (ASIC), a controller, a microcontroller, and/or another semiconductor integrated circuit device), an HDMI or other electronic display dongle, a hardware appliance or other hardware device, other logic hardware, and/or other executable code stored on a computer readable storage medium. Other embodiments may include similar or equivalent means for generating a digital watermark to be presented in the graphical interface based on the identified data.
A means for embedding the digital watermark into the graphical interface prior to the data being presented in the graphical interface such that the digital watermark is graphically indistinguishable to a user in the graphical interface, in various embodiments, may include one or more of a hardware device 102, a backend server 110, a presentation module 906, a security module 120, an aggregation module 104, a processor (e.g., a central processing unit (CPU), a processor core, a field programmable gate array (FPGA) or other programmable logic, an application specific integrated circuit (ASIC), a controller, a microcontroller, and/or another semiconductor integrated circuit device), an HDMI or other electronic display dongle, a hardware appliance or other hardware device, other logic hardware, and/or other executable code stored on a computer readable storage medium. Other embodiments may include similar or equivalent means for embedding the digital watermark into the graphical interface prior to the data being presented in the graphical interface such that the digital watermark is graphically indistinguishable to a user in the graphical interface.
Means for performing the other method steps described herein, in various embodiments, may include one or more of a hardware device 102, a backend server 110, an authentication module 202, a local authentication module 302, a network authentication module 304, a password manager module 306, a direct access module 204, a pattern module 308, an access repair module 310, a hierarchy module 312, an interface module 206, a route module 314, a frequency module 316, a test module 318, an aggregation module 104, a network interface, a processor (e.g., a central processing unit (CPU), a processor core, a field programmable gate array (FPGA) or other programmable logic, an application specific integrated circuit (ASIC), a controller, a microcontroller, and/or another semiconductor integrated circuit device), an HDMI or other electronic display dongle, a hardware appliance or other hardware device, other logic hardware, and/or other executable code stored on a computer readable storage medium. Other embodiments may include similar or equivalent means for performing one or more of the method steps described herein.
The present invention may be embodied in other specific forms without departing from its spirit or essential characteristics. The described embodiments are to be considered in all respects only as illustrative and not restrictive. The scope of the invention is, therefore, indicated by the appended claims rather than by the foregoing description. All changes which come within the meaning and range of equivalency of the claims are to be embraced within their scope.
This application is a continuation of U.S. patent application Ser. No. 17/035,350 entitled “WATERMARK SECURITY” and filed on Sep. 28, 2020, for Brandon Dewitt, et al., which is a continuation of U.S. patent application Ser. No. 16/137,517 entitled “WATERMARK SECURITY” and filed on Sep. 20, 2018, for Brandon Dewitt, et al., which claims the benefit of U.S. Provisional Patent Application No. 62/561,201 entitled “WATERMARK SECURITY” and filed on Sep. 20, 2017, for Brandon Dewitt, et al., which is incorporated herein by reference.
| Number | Name | Date | Kind |
|---|---|---|---|
| 10026138 | Sinclair | Jul 2018 | B2 |
| 10789666 | Dewitt | Sep 2020 | B2 |
| 11216903 | Dewitt | Jan 2022 | B2 |
| 20030147549 | Choi | Aug 2003 | A1 |
| 20040131184 | Wu | Jul 2004 | A1 |
| 20060077419 | Sugiura | Apr 2006 | A1 |
| 20070165261 | Someya | Jul 2007 | A1 |
| 20070180148 | Yadidian | Aug 2007 | A1 |
| 20070194103 | Manheim | Aug 2007 | A1 |
| 20140169616 | Homme | Jun 2014 | A1 |
| 20140325367 | Fear | Oct 2014 | A1 |
| 20150281325 | Takaichi | Oct 2015 | A1 |
| 20160253772 | Kofod | Sep 2016 | A1 |
| 20160358298 | Martin | Dec 2016 | A1 |
| 20190035046 | Tung | Jan 2019 | A1 |
| 20190045078 | Tung | Feb 2019 | A1 |
| Number | Date | Country | |
|---|---|---|---|
| 20220058764 A1 | Feb 2022 | US |
| Number | Date | Country | |
|---|---|---|---|
| 62561201 | Sep 2017 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | 17035350 | Sep 2020 | US |
| Child | 17515291 | US | |
| Parent | 16137517 | Sep 2018 | US |
| Child | 17035350 | US |
