Patent Grant
10091312
Internet based technology traditionally has relied on the use of cookies as one mechanism for tracking user preferences and other information. As one example, cookie identifiers generated and used by web browsers, mobile applications, and web service providers present a convenient approach to help identify devices and users for various purposes.
Consumers are increasingly being represented in the digital world by their devices. This identification of a consumer by their device allows companies to tie together historical activities of digital consumers' from multiple device, and subsequently manage and optimize the consumer experience via better, more informed decisions. As devices and Internet access channels proliferate, the ability to identify consumer devices has significantly fragmented, making fraud detection, authentication processing, data measurement, and data collection substantially more difficult. While companies have relied on the HTTP browser cookie as the primary identification technique, cookies have been losing their efficacy. Techniques beyond the cookie, such as device Hardware IDs, Operating System IDs, and Statistical IDs, have all evolved over the past 5 years to make device identification very fragmented, complex, and expensive.
The challenges of resolving a user to a particular device and/or identifying the various relationships illustrated in
In one embodiment, a computing system comprises one or more hardware computer processors and a network interface providing data communication with (1) a first electronic data store configured to store a plurality of matching algorithms associated with a corresponding plurality of online requesting entities, said matching algorithms each including one or more deterministic matching rule and one or more probabilistic matching rule, and (2) a second electronic data store configured to store a plurality of device profiles. In one embodiment, each device profile is associated with a different electronic communication device and includes at least: a unique device identifier; and one or more identifiers associated with the electronic communication device, each of the identifiers associated with a respective identifier type of a plurality of identifier types. In one embodiment, the computing system further comprises a non-transitory storage device storing executable software instructions that when executed: receive online event data including a plurality of device identifiers associated with a first device that has interacted with a first online requesting entity; access, from the first electronic data store, a first matching algorithm associated with the first online requesting entity, the first matching algorithm including a plurality of matching rules; and until either a determination that a particular accessed device profile of the plurality of device profiles matches a selected matching rule of the matching algorithm, or until all of the matching rules of the matching algorithm are evaluated with reference to the online event data, in real-time: select a highest priority matching rule of the matching algorithm that has not yet been evaluated with reference to the online event data; determine a selected identifier type indicated in the selected matching rule; determining an identifier of the selected identifier type from the one or more device identifiers of the online event data; for each of the plurality of device profiles: accessing the device profile identifier of the selected first identifier type in the device profile; determine whether the device profile identifier matches the identifier of the selected identifier type from the online event data; in response to determination that the profile device identifier matches the identifier of the selected identifier type from the online event data, provide, to the requesting entity, a unique device identifier associated with the matched device profile in the second electronic data store; in response to determination that none of the matching rules of the matching algorithm match a device profile of the plurality of device profiles: generate a new device profile associated with a unique device identifier and at least some of the online event data; and provide, to the requesting entity, the unique device identifier associated with the new device profile, wherein the first matching algorithm is executed in real-time in response to receiving the online event data.
Digital Identification Resolution
Disclosed herein are digital identification resolution processes, which are processes of recognizing and linking disparate identification data elements from multiple sources and reconciling those data elements to correspond to a single device or group of devices. An ID Resolution System described herein performs a digital identification resolution process through the use of unique IDs (DPIDs) for each device and/or group of devices that can be reliably recognized when only anonymous data related to a device (e.g., from a browser request from the device) is known. In addition, the ID Resolution System may support client supplied attributes, such as by mapping DPIDs to client-specific one way hashed identifiers that are representative of personally identifiable information (“PII”).
The ID Resolution System can use any available “identifiers,” which is used throughout this specification to generally describe any attribute associated with a device, such as a device type identifier, a cookie identifier, a particular device ID (such as Apple™'s Identifier for Advertising (“IDFA”) and Google™'s AndroidId or Advertising ID), first or third party identifiers, client supplied identifiers, contextual data (e.g., location data and/or network information), such as IP addresses, timestamps, time-differential linking (“TDL”) data, Latitude/Longitude coordinates (either provided directly or indirectly via an IP address), and/or any other attribute associated with a device, even if later developed. Such identifiers may be received by the ID Resolution System described herein and used to determine, based on a combination of deterministic and/or probabilistic matching rules, a device profile identifier (a “DPID”) and/or a group profile identifier (a “GPID”). Furthermore, these identifiers, the DPIDs, and/or the GPIDs may be translated by the ID Resolution System in various manners to provide the requesting entity with various information associated with the provided input data that have would not otherwise be available.
In various embodiments, the ID Resolution System provides numerous technical benefits in addition to the other benefits described herein. For example, the ID Resolution System may be configured to be provide high-performance and scalability to provide highly responsive, real-time, identifier-to-DPID mappings (e.g., 10 ms or less) for very high volume of device identifiers (e.g., for hundreds, thousands, or millions of devices). The ID Resolution System can also be configured to protect security and privacy, both of the requesting entity data that is shared with the ID Resolution System and of the underlying individual users of the devices, using one or more encryption techniques and by preserving user privacy choices (e.g., opt-out and/or opt-in choices) at the device profile level. For example, requesting entities may provide, to the ID Resolution System, privacy choice information with a set of device identifiers. The ID Resolution System can then store these privacy preferences with the DPID profile and return information to the requesting entity in accordance with these privacy preferences. The ID Resolution System may be configured to comply with any applicable privacy laws, rules, and regulations, including but not limited to COPPA, EU Data Privacy Directives, EU Cookie Laws, eDAA Principles, EU US Safe Harbor, DAA Code of Conduct (2013), and/or NAI Code of Conduct (2013).
An ID Resolution System such as the one described herein may provide a standalone, digital identity management platform focused to improve & enable identifying protection, fraud detection, marketing, and other technologies to better identify devices, related devices and users, and/or identifiers associates with those devices. As an example, the ID Resolution System may be used by or in conjunction with, among others, data management platforms, demand-side platforms, supply-side platforms, attribution solutions, re-targeters, measurement and attribution tools, ad servers, exchanges, ad networks, agency trading desks, and risk-based user authentication solutions.
The ID Resolution System provides a variety of benefits to help requesting entities improve targeting efforts. For example, using the unique DPIDs and/or other services provided by the ID Resolution System, requesting entities can retarget users or customers from mobile device applications (e.g., on a tablet, smartphone, or similar device) which do not rely on or implement third party cookies, and for which traditionally no connection between mobile device applications and third party cookies does not exist. In addition, cookies may not be persistent and are prone to being reset or deleted, which leads to “cookie churn” and/or potentially “losing” the customer due to the disappearance of the cookie. Cookie churn can lead to inaccurate conversion metrics and, ultimately, reduced customer spending. There is also an inability to compute accurate conversion metrics from in-application advertising to mobile web conversion. The unique DPID provided by the ID Resolution System may help overcome the issues of cookie churn and/or inaccurate conversion metrics by, for example, providing a DPID that persists whether the associated cookie is lost.
Another potential benefit provided by the ID Resolution System is an increased confidence level of uniquely identifying trusted devices (and/or, similarly, untrusted devices). Such identification may be used in many security applications, such as to improve authentication of users and/or devices (e.g., to reduce requirements of login processes from known trusted devices), provide privacy persistent assurances for a user, perhaps across multiple devices and applications of the particular user, provide fraud risk assessments (e.g., a fraud risk score associated with one or more received identifiers), and any other applications wherein device identification, group identification, and/or identity translation is advantageous.
In some embodiments, the ID Resolution System may help requesting entities improve attribution efforts. Attribution is the process of identifying a set of user actions (“events”) that contribute in some manner to a desired outcome, and then assigning a value to each of these events. Attribution provides a level of understanding of what combination of events influence individuals to engage in a desired behavior, typically referred to as a conversion. The ID Resolution System improves attribution tracking by enabling improved tracking of user behavior across applications and/or devices. For example, the unique DPID and/or GPID may enable or improve the ability to track conversion from applications (e.g., a mobile app) to the web (e.g., a mobile web browser). Attribution windows may also be increased by extending the lifetime of sessions through the continued use of a unique DPID that persists even though underlying device identifiers may change or evolve over time (e.g., new device identifiers may replace old ones but still be mapped to the same unique DPID, new device identifier types may replace old types but still be mapped to the same unique DPID, etc.). The ID Resolution System also provides the ability to analyze “view-through” attribution without relying on third party cookies. In some embodiments, the ID Resolution System provides the ability to utilize device groupings or “households” and multi-screen, multi-device associations in order to enable cross-device attribution.
The ID Resolution System provides a variety of benefits that may help requesting entities improve audience building and maintenance efforts. For example, ID Resolution System can enable requesting entities to build or identify segments in the mobile web context that do not rely on third party cookies. Segments may also be built across multiple devices using the cross-device visibility provides by the GPIDs and related GPID profiles provided by the ID Resolution System. The ability to increase the overall lifetime and persistence of unique DPIDs also can contribute to improved maintenance of active profiles and segment quality. This can also further enable the ability to track customers across multiple websites in both mobile web and in applications to build segments based on multiple websites owned by a publisher or service provider.
Another potential benefit provided by the ID Resolution System may include enabling media buying platforms to increase reach to targetable audiences across multiple digital touch points with greater confidence and persistency for improved campaign performance (for example, by enabling targeting in instances where cookies are not available or may not work, such as in non-web browser applications that do not implement cookies; by enabling targeting across mobile applications and web browsers; by enhancing cross-screen and/or cross-device targeting; and so on).
A fragmented, complex and expensive identification landscape adversely impacts companies' ability to deliver relevant, personalized and safe experiences to their customers, degrading not only ROI, but also brand reputation. As example, marketers are inefficient in their advertising spend, since they are not able to reliably target individuals due to the proliferation of web access channels (e.g. multiple browsers, native apps) and devices. These inefficiencies can be due to factors such as poor conversion analytics, too-broadly targeted media, or not reaching a customer during their decision journey. Marketers often cannot identify their customers reliably for many reasons: identifiers (“IDs”) typically exist in their own context (for example, each web browser has its own cookie space, mobile ID's may not be accessible in mobile web environments, etc.); there is no concept of an individual or device address; IDs can be transient and can be erased easily; IDs cannot be generated in all cases (for example, a third party may block cookie reading or writing); and in general, sharing IDs among partners (universality) can be complex and require significant resource overhead.
Another potential benefit provided by the ID Resolution System may include enabling brands and agencies to better track the performance of marketing programs holistically across multiple digital touch points, which can help contribute to a more meaningful, accurate measurement of marketing spend (for example, by improving conversion tracking from mobile applications to web browsers; by increasing attribution windows; by enabling or improving view through attribution and/or cross-device attribution; and so on).
Another potential benefit provided by the ID Resolution System may include enabling publishers and/or media selling partners to better define and extend targetable audiences across multiple digital touch points to maximize yield (for example, by enabling or improving tracking and segmentation in mobile applications; by enabling or improving tracking and segmentation across multiple devices and/or domains; by increasing the overall persistency and “lifetime” of users and associated profiles; and so on).
Another potential benefit provided by the ID Resolution System may include enabling data owners and data management platforms to better capture and rationalize audience data across multiple digital touch points to maximize the impact of audience insights (for example, by enabling or improving the ability to connect data across multiple devices and/or environments; by increasing the lifetime of a profile; by enabling or providing seamless data distribution that is not reliant on cookie synchronization; and so on).
The ID Resolution System described herein may be used by customers to augment other methods of identification (e.g. browser cookies, native device identifies, etc.), specifically by filling in identification gaps in environments/use cases where these other identification methods do not work or perform poorly, and to connect the activity of “unknown users.” The systems and methods described herein may be used, directly and indirectly, by brands, web publishers, mobile app developers and marketing technology platforms, including ad tech, data management and analytics platforms. These companies can leverage the ID Resolution System to improving the efficacy and return on investment (“ROI”) of their and/or their customers' marketing programs, by filling in identification gaps relating to their ad operations, audience data management and analytics, in a privacy and data secure manner. The ID Resolution System and features described herein may also be applicable to a variety of other online contexts, including but not limited to, ecommerce, session management, fraud detection, user management, web authentication, to name a few.
Embodiments of the disclosure will now be described with reference to the accompanying figures, wherein like numerals refer to like elements throughout. The terminology used in the description presented herein is not intended to be interpreted in any limited or restrictive manner, simply because it is being utilized in conjunction with a detailed description of certain specific embodiments of the disclosure. Furthermore, embodiments of the disclosure may include several novel features, no single one of which is solely responsible for its desirable attributes or which is essential to practicing the embodiments of the disclosure herein described.
For purposes of this disclosure, certain aspects, advantages, and novel features of various embodiments are described herein. It is to be understood that not necessarily all such advantages may be achieved in accordance with any particular embodiment of the invention. Thus, for example, those skilled in the art will recognize that one embodiment may be carried out in a manner that achieves one advantage or group of advantages as taught herein without necessarily achieving other advantages as may be taught or suggested herein.
Overview
As discussed in further detail below, the ID Resolution System may leverage deterministic and/or probabilistic identifiers to provide a consistent and coherent view of a particular device. By using both types of identifiers, the system may better associate an identifier (or multiple identifiers) with a DPID and/or GPID. For example, in some embodiments use of multiple probabilistic identifiers may result in a near deterministic match to a DPID. As used herein, a deterministic ID of a device is unique to the device. If such a deterministic identifier is available, and it matches one that has been seen previously, the ID Resolution System can be confident that the event (from which the deterministic identifier was identified) originated from the same device. For example, a cookie that has been seen more than once can be deterministically associated with that same cookie that was previously seen (and the corresponding DPID of the browser), except perhaps in fraud use cases where, for example, cookie values may be hijacked by fraudsters (see further discussion below regarding such fraud use cases). When only probabilistic IDs are available for an online event, the ability to do this matching becomes more challenging. Probabilistic IDs, such as a device insight identifier (e.g., provided by 41st Parameter) are more prone to collisions and divisions due to the probabilistic nature of the identifiers. Thus, general, device identifier mapping systems that analyze only probabilistic identifiers have problems with collisions, such as where multiple probabilistic identifiers are resolved to a same device, while deterministic systems have problems with device separation, such as when a device identifier changes and then has a new device identifier and also an old device identifier each associated with a same physical device.
The ID Resolution System disclosed herein improves this technical challenge of resolving device identifiers with better accuracy to particular devices. In some embodiments, the ID Resolution System measures performance of probabilistic IDs when used for real-time matching so that an understanding of statistical error inherent in probabilistic matching related to respective identifiers may be determined. Outcomes of this analysis can then be used to generate models for using probabilistic identifiers alongside deterministic identifiers so that a statistical device identification technology can be provided to a requesting entity with an expected and controlled error rate.
The systems and methods described below make use of both probabilistic and deterministic identifiers in order to provide a more coherent view of the device. Advantageously, the requesting entity can control the extent to which probabilistic and/or deterministic identifiers are used in determining a matching DPID and/or GPID, such as to set a confidence level that the requesting entity has in a set of identifiers.
The ID Resolution System may include a device profile data store wherein DPIDs and GPIDs are maintained and updated over time. The device profiles (e.g., associated with respective DPIDs) may include the unique DPID and a history of all device identifiers which have been associated or mapped to the unique DPID over time, including frequency of use and other statistical information which may be useful for auditing and improving resolution rules over time. For example, a device profile may also include any matching pattern used to link or map certain device identifiers and/or sessions to the DPID. In some embodiments, auditing information and reports may be generated by the ID Resolution System to provide a summary of the frequency of each match pattern being applied and corresponding mapping results. Such reports may be useful to the service provider who may use the auditing data to modify or further customize identifier mapping or matching rules to improve the efficacy of device identifier mappings. The device profile may further include any information that may be necessary to identify the device, including contextual data such as IP addresses, timestamps, TDL data, Latitude/Longitude coordinates, and/or privacy choice flags in order to preserve privacy choices associated with user(s) of the device. In some embodiments, the ID Resolution System can provide device profile merging capability (for example, identifying when two profiles are really the same). Thus, through the observation of IDs over time, device profiles can be merged through common attributes.
Example ID Resolution System Processes
In one embodiment, the ID Resolution System performs multiple functions that may be requested separately (or in combination) from requesting entities. For example, in one embodiment the ID Resolution System may perform a device matching process, a group matching process, and an identity translation process. In this embodiment, the device matching process receives one or more identifiers of an event associated with a device (e.g., information regarding an access request to a requesting entities website or database) and attempts to identify a DPID associated with the requesting device. The group matching process receives a DPID, or some other device identifier that can be used to resolve a DPID, and determines one or more GPIDs to which the device is associated. The identity translation process provides translations of various types of identifiers to other resolved data items (e.g., any data in an associated DPID and/or GPID). Each of these exemplary processes are described in further detail below. While described separately, the processes may be performed as part of a single data request from a requesting entity. For example, a cookie identifier may be provided by a request entity with a request to provide all available information, which may result in return of a DPID, one or more GPIDs, and various data from the device and/or group profiles associated with the identified IDs.
In order to facilitate an understanding of the systems and methods discussed herein, a number of terms are defined below. The terms defined below, as well as other terms used herein, should be construed broadly to include the provided definitions, the ordinary and customary meaning of the terms, and/or any other implied meaning for the respective terms. Thus, the definitions below do not limit the meaning of these terms, but only provide exemplary definitions.
Requesting Entity, Service Provider, or Online Service Provider: an entity that requests data from the ID Resolution System, such as via an API through which the requesting entity can transmit online event data to the ID Resolution System. The requesting entity may be an entity that provides services directly to end users, e.g. via devices that communicate with a server (e.g., website, application, etc.) of the requesting entity or an entity that provides analysis of information regarding user devices (e.g., a fraud detection entity). Requesting entities may be clients of an entity that operates the ID Resolution System or may receive information from the ID Resolution System based on some other arrangement.
Online Event or Event: Any interaction by a user device with a server, website, database, and/or other online data owned by or under control of the requesting entity. Online events include access of webpages, submission of information via webpages, accessing a server via a standalone application (e.g., an application on a mobile device or desktop computer), or any other access protocols.
Event data: Attributes or any other information associated with one or more online events, such as an attribute that may be used to identify, possibly in combination with other identifiers, a particular electronic device.
Device Identifiers or Identifiers: Any attribute associated with a device or any activity of the device. Device identifiers may include event data, such as may be gathered in response to an online event initiated by a device. Device identifiers may include attributes of a device that are usable to identify the device with varying degrees of uniqueness—some identifiers may be deterministic (e.g., an identifier that is unique for a particular device), while some identifiers may be probabilistic (e.g., an identifier may not be unique for a particular device, but may in some cases be used to identify the particular device with a degree of probability, especially in association with other identifiers).
Without limiting the scope of this term, in various embodiments device identifiers may include, for example, one or more of a cookie, a device-specific identifier, an operating system generated identifier, an identifier generated and/or used in association with a particular application installed on the device (e.g., an “app ID”), an identifier generated and/or used in association with a particular web service accessed using the device (e.g., a “login ID”), an IP address (or portion thereof) used by the device to communicate with one or more networks, a MAC address, authentication credentials, browser information, time of day, week, year information, and/or any other identifier which may be associated with a device.
In some implementations, some device identifiers may be derived (“derived device identifiers”) based on one or more device-related attributes and/or identifiers. For example, the ID Resolution System may deploy client side JavaScript code (also referred to as a JavaScript Collector, which may be included in a client's website or App, for example), for collecting various attributes and/or identifiers from a device, and provide the data collected to a server side identification system that executes an algorithm(s) on those identifiers to compute a derived device identifier, which may be used in the same manner as any other device identifier discussed herein, such as in application of device matching rules. In some embodiments, different algorithms (or “recipes”), e.g., applied to different device identifiers with different weightings, may be applied to the same event data in order to provide multiple derived device identifiers, each potentially usable with different levels of accuracy and longevity in uniquely identifying the device. In some embodiments, derived device identifiers do not use any personally identifiable information. In some embodiments, derived device identifiers do not rely on cookies and/or other deterministic identifiers. In some implementations, device identifiers may include one or more expanded identifiers, which are discuss further below.
Device Profile Identifier or DPID: An identifier that is unique to a device, such as to a particular mobile phone, smartphone, tablet, laptop, desktop, smart watch, smart glasses, gaming system, server computing system, or any other computing device. A DPID is associated with a device profile, which includes various information regarding, for example, the device, online events of the device, one or more users of the device, one or more groups associated with the device, and any other information about the device, such as any of the information discussed below.
Group Profile Identifier of GPID: An identifier that may be associated with more than one DPID, such as multiple DPIDs each determined to be part of a common household or multiple DPIDs that are each associated with a particular user. A GPID may be created initially with only a single DPID and then other DPIDs associated with the corresponding group may be later added. A GPID is associated with a group profile, which includes various information regarding, for example, the devices associated with the group, such as links to the DPIDs of those devices.
Matching Algorithm: logic that compares one or more identifiers, such as received from a requesting entity in association with an online event, in an attempt to determine a particular device associated with the one or more identifiers. A matching algorithm may be particular to a requesting entity, such as to optimize use of particular identifiers that are typically available to the requesting entity and/or to meet the desires match confidence level of the particular requesting entity. A matching algorithm may have multiple components, such as a deterministic matching algorithm and/or a probabilistic matching, which are discussed further below. A matching algorithm may include a series of rules that are executed in a prioritized order until a device is matched with a requisite confidence level or all of the rules have been executed. A rule may include one or more of the following components:
As noted above, in cases where a deterministic identifier is known, a matching algorithm may determine whether there is a one to one match of the input identifier to any of the identifiers associated with DPIDs in the profile data store. In the case of a user-level identifier, such as email addresses or account IDs, the matching algorithm may apply analyze one or more additional identifiers to deterministically resolve the event to a DPIID. For example, a user id identifier (that deterministically identifies a user) may be analyzed in combination with a User-Agent string to match a particular device (DPID) and its corresponding device profile. For example, a matching algorithm for deterministic identifiers may include several rules each related to one type of deterministic ID using an “exact match” comparison logic, such as an exact match on cookie, exact match on email and User Agent, exact match on IDFA, and other related combinations.
When probabilistic IDs are used by a requesting entity, the ID Resolution System may initially examine behavior of these identifiers alongside deterministic identifiers to evaluate the extent to which statistical error of the probabilistic identifier causes wrong matches that adversely impact the quality of device resolution. In one embodiment, the ID Resolution System performs testing of a probabilistic matching algorithm, such as including multiple probabilistic matching rules, in order to train the matching algorithm. In one embodiment, the ID Resolution system executes an initial probabilistic matching algorithm alongside a deterministic matching algorithm on sets of event data including both deterministic and probabilistic identifiers, in order to benchmark how well rules and/or sets of rules, based on probabilistic identifiers match to DPIDs. The probabilistic matching algorithm may then be updated to test various combinations of probabilistic rules that may increase confidence levels in matching to unique devices.
In one embodiment, a benchmark matching algorithm is applied to incoming event data from a requesting entity while a test matching algorithm is also applied to the same incoming event data. The benchmark matching algorithm analyzes one or more deterministic identifiers so that deterministic matches are output, while the test matching algorithm includes rules associated with one or more probabilistic identifiers. In general, the outcome of the two matching algorithms is compared (for each set of event data that is analyzed) in order to determine which probabilistic rules improve confidence level of a match and which probabilistic rules do not. For example, if a set of identifiers associated with 1000 events are processed over a time period, the outcome from each of the benchmark and test matching algorithms may be compared to generate a score indicative of how well the test matching algorithm (and/or each individual test rule of the test matching algorithm) works in identifying corresponding devices. For example, if the benchmark matching algorithm (e.g., applying deterministic rules) matches a first device profile and a first test rule also matches that same first device profile, the score for the first test rule may increase. Correspondingly, if the benchmark matching algorithm matches a first device profile, but the first test rule matches a second device profile, the score for the first test rule may decrease. In one embodiment, if neither matching algorithm matches an existing device profile (e.g., such that a new device profile is created), the score for the test rule may not be impacted (e.g., this result does not necessarily indicate any increased or decreased matching capabilities of the test rule). In the example above where 1000 events are processed by both matching algorithms, if 700 events resulted in the same device profile match from the benchmark and first test rule, while 200 events resulted in different device profile matches and 100 events resulted in no profile matches, a score for that test rule may be calculated as 700/(700+200)=78%. Thus, this percentage indicates a confidence level that the first test rule would accurately match subsequent event data to devices, assuming the event data is of a similar type and quality. Similar analysis may be performed of other test rules of the test matching algorithm, either on the same set of event data (e.g., the example 1000 events discussed above) or other sets of event data (e.g., test rules may be analyzed sequentially two different sets of data) to determine confidence levels for additional test rules.
In one embodiment, an overall score for the test matching algorithm (e.g., comprising three different test rules each applied to 1000 set up event data) may be calculated based on the individual test rule accuracy scores weighted based on quantity of events that triggered the respective rule (e.g., number of events that outputted a “matching” device profile from the rule, whether or not the device profile matched the benchmark matching algorithm output). For example, if the first test rule at a 78% accuracy for a total of 900 triggered matches (700 of which were correct and 200 of which were incorrect), while a second test rule had a 95% accuracy for a total of 200 triggered matches and a third test rule had a 50% accuracy for a total of 600 triggered matches, the overall test matching algorithm accuracy may be calculated using the formula:
Where A is the matching accuracy, n is the number of rules, RA is rule accuracy for a particular rule, and T is a quantity of events for which the particular rule triggered. Using this formula and the example benchmark matching algorithm output discussed above results in: A=(0.78*900+0.95*200+0.5*600)/(900+200+600)=70%. Similarly, accuracy of a subset of test rules may be calculated, such as to increase an overall matching accuracy of a test matching algorithm. For example, the test rules having the highest matching accuracy in the example above would result in an overall matching accuracy of 81% (0.78*900+0.95*200)/(900+200)). Thus, these various matching accuracies (e.g., the rule specific and/or overall matching algorithm accuracies) may be provided to the requesting entity in order to determine the optimal test rules for production use in its probabilistic matching algorithm, perhaps in combination with a deterministic matching algorithm.
In other embodiments, accuracy of a test matching algorithm may be assessed and/or calculated in different manners. Advantageously, with a matching accuracy developed for the test matching algorithm, if the confidence level is not at the requisite level (a required matching level set by the requesting entity), parameters of the test matching algorithm may be adjusted (e.g., adjusting value ranges of match tiers for one or more identifiers and/or removing or adding additional attributes to the test matching rules) and retested in an attempt to increase the confidence level.
With a test matching algorithm accuracy of such as using the methodology discussed above, accuracy of an overall matching algorithm (e.g., that also includes deterministic matching algorithm) may be tested and, typically, expected to obtain an even higher matching accuracy in view of the addition of the deterministic matching algorithm that normally would be applied first on incoming event data. For example, matches output from each of one or more deterministic rules of the benchmark matching algorithm, as applied to the same event data (e.g., the set of 1000 event data sets discussed above) may be used in combination with the probabilistic accuracy levels of particular probabilistic rules in order to determine an overall Matching Algorithm Accuracy. In one embodiment, the proposed matching algorithm comprising one or more deterministic rules and one or more probabilistic roles may be reapplied to the set of event data and an overall accuracy calculated using the probabilistic rule accuracy levels determined previously. Because some events will be matched using deterministic rules, not giving probabilistic rules a chance to match, the overall ruleset accuracy is expected to increase. Below is an example matching algorithm including two deterministic rules (rules 1-2) and the two highest accuracy probabilistic rules from above (rules 3-4 below) that are applied in priority number until one rule matches, such that if rule one matches, 2-4 are not evaluated.
Thus, the overall matching algorithm accuracy has been increased to 97.4% ([(1*500)+(1*300)+(0.78*100)+(0.95*50)]/(500+300+100+50)) with an increase of 150 more events (out of 1000 total events) being matched to devices than would be possible using only the deterministic rules.
Example Device Matching
In one embodiment, an ID Resolution System may provide a unique and persistent DPID in response to receipt of one or more of various identifiers received from a requesting entity. Furthermore, the ID Resolution System may provide various other information associated with the provided identifiers, such as an indication of a particular user, other devices associated with the user, a household (or other group) of which the user or device is a member, and other related information. DPID resolution may be performed for various requesting entities, such as entities that analyze data associated with users and/or provide data to users. For example, a requesting entity may be a fraud detection entity, an authentication entity, a marketer, a website provider, an application provider, and/or a media buying and/or selling entity. Depending on the embodiment, the requesting entity (also referred to herein as a “service provider”) may provide multiple services for users in conjunction with the device profile and/or other related resolution information associated with a DPID (e.g., discussed below with reference to translation processed) that may be determined.
Advantageously, the ID Resolution System provides a flexible and highly customizable and configurable architecture that enables requesting entities to configure matching algorithms to satisfy their own levels of tolerance for statistical error that may arise when resolving various identifiers associated with a device with varying degrees of uniqueness to a device profile. For example, some device identifiers may be unique and/or highly deterministic (e.g., unique for a particular device or user), while some device identifiers may be less unique and/or more generally probabilistic (e.g., the identifier may be reused for multiple devices, as may sometimes be the case for an IP address).
Below is an example “Match Device” request that may be received from a requesting entity.
POST: http://localhost:1111/service/matchdevice
{
“inputIds”: {
“tpcookie”: “bb8a2551-4eca-4006-8a55-a4”,
“IP”: “59.232.19.9”
},
“time”: “1442252559”, “verbose”: “false”,
“clientId”: “client1”,
“privacy”:[
}
In response to the above request, the ID Resolution System may return, such as by applying a matching algorithm developed for and/or by the requesting entity that includes both probabilistic and/or deterministic components, for example, the determined DPID in one of the below example formats:
As discussed further below, the ID Resolution System implements a device matching technology that analyzes probabilistic and/or deterministic identifiers of a device. Such identifiers currently number in the hundreds or thousands, and are expected to change and increase over time. Advantageously, the ID Resolution System analyzes such identifiers from Internet-connected devices to match a DPID that can be used for tracking and targeting, similarly to other identifiers (e.g., cookies or native device IDs), but whereas other identifiers only work in certain environments/platforms/devices, a DPID may be determined for a device based on various different identifiers and/or combinations of identifiers that may be available from online events.
In various embodiments, the ID Resolution System creates and maintains profiles for each DPID that may be integrated, analyzed, linked, managed, translated and/or distributed in a wide array of different online and offline identifiers and other information, from different device/application/browser contexts. Such device profiles may further be associated with device groups created at the household level (e.g., association of multiple devices to one or more living unit or location) and/at the user level (e.g., association of multiple devices to a single user), as discussed below.
Example Group Matching
In this example, the device 162B is also associated with multiple identifiers that may be transmitted by the device 162B to various requesting entities (e.g., websites, application providers, databases, storage devices, etc.). In this particular embodiment, the identifiers illustrated (which may be provided individually or in some combination to the ID Resolution System) include a login data element 106A, a cookie data element 106B, an IDFA data element 106C, a DI ID data element 106D, and another cookie data element 106E. Many other identifiers, which may correspond to a particular browser, application, operating system, device hardware, device type, or various other properties of a device, may also be associated with the device 162B (and similarly with the other devices 162 associated with the user 102). Some identifiers are constant (e.g., never change for a particular device) while others are transient (e.g., may change each time the data element is transmitted). Some identifiers are more accurate than others, and some are available everywhere, regardless of platform or access point while others are platform, device, or channel specific. Accordingly, identity fragmentation results from this wide variety of identifiers from various devices in different contexts.
The ID Resolution System may provide the ability to identify or generate device groupings (e.g., “householding”) by implementing one or more matching rules based on IP address, device types, and various patterns displayed by each. A GPID matching engine may also provide the capability to associate multiple devices with a single user to enable multi-screen marketing and targeted content efforts. For example, one-way hashed identifiers provided by a first party (for example, an encrypted or hashed login ID) may be utilized in conjunction with the unique device ID and/or device profile in order to link or associate multiple unique devices together as being likely associated with a single user. Thus, a GPID may be associated with a group of devices that are used by multiple users (e.g., a “household” GPID including multiple DPIDs of devices used within a household) and/or multiple devices used by a particular user (e.g., a “multiscreen” GPID).
In the example of
Furthermore, device 162B, as wells as the five other devices 163A-163E that are associated with the household 104 may be associated with a household GPID. This household grouping may provide greater visibility into connections between individuals associated with the user 102. For example, multiple members of a household may use one or more shared computing devices, such as device 162B, but have different and/or unique login IDs and other identifiers associated with their respective usage. In certain embodiments, device groupings implementing unique matching rules based on IP address, device types, and the patterns displayed by each may be implemented to generate a device-based view of a household. In this embodiment, a household GPID for the household 104 may be created and associated with multiple DPIDSs of computing devices, including device 162B and devices 163A-163E. In some implementations, certain devices may be included in multiple group profiles, including one or more multiscreen and/or household groups.
One example of householding is based primarily on an IP address of matching requests received from various service providers. For example, the ID Resolution System may receive an IP address (e.g., an IPv4 address) from a first requesting entity and determine a connection type of the IP address. In one embodiment, the connection type is determined by a third-party service, such as by using a geo-IP database to determine (or predict in some circumstances) whether the connection type is residential, commercial, and/or another type. If the received IP address is determined to be residential, the determined DPID (e.g., using the device matching process discussed herein) may be associated with a GPID that is representative of a household. When other DPIDs are later associated with the same IP address (e.g., through subsequent device matching processes performed for the same or other service providers), those other DPIDs are also associated with the GPID such that the devices of that household are then linked within the GPID.
Depending on the embodiment, the householding rules may include limitations on combining groups of DPIDs from an IP address into a GPID. For example, in one embodiment if more than a predetermined number of DPIDs are associated with the same IP address, those DPIDs are not associated with a GPID (or at least not a household GPID) since the large number of DPIDs from that IP address may indicate some larger organization than a household.
Below is an example “Group” request that may be received from a requesting entity.
GET
In response to the above request, the ID Resolution System may first perform a device matching process (e.g., such as by using a matching algorithm including probabilistic and deterministic matching rules) to identify a DPID and then identify a GPID associated with the DPID in the device profile. In other embodiments, the GPID profile information may be requested with a request that includes the DPID and/or the GPID. The identified GPID (or multiple GPIDs) may then be parsed, filtered, and/or organized in various manners (such as may be defined by the requesting entity) for return to the requesting entity. Below is an example response to the a GPID request in one particular example format:
Example Identity Translation Processes
In one embodiment, a requesting entity may provide one or more identifiers to the ID Resolution System (e.g., a device identifier, such as a DPID, device type identifier, etc.), and/or the identifiers may be extracted from the device by the ID Resolution System, with a request for specific resolution data items, wherein “resolution data” and “resolution data items” each generally refer to any information associated with received device identifier(s) that may be returned by the ID Resolution System, either in a raw, combined, or derivative format, such as any information included in a device profile and/or related group profiles. In other embodiments, the requesting entity may request all available resolution data (e.g., that the ID Resolution System can determine based on the provided identifiers). The ID Resolution System may then use the provided identifier(s) to determine and/or generate resolution data. For example, a requesting entity (e.g., a client of the ID Resolution System) may send into the service a browser cookie identifier {abc} and ask the service to return all other cookie values associated with the DPID or with a GPID the DPID is a member of. Another requesting entity (or the same requesting entity) may send the browser cookie identifier {abc} and ask the service to return all native device identifier values (e.g. IDFA) associated with the DPID or with a GPID the DPID is a member of.
Listed below are example resolution data items that may be returned in response to a request from a requesting entity. This resolution data may be returned in response to first determining a particular DPID based on one or more received identifiers. In other applications, the requesting entity provides a DPID in the initial request for resolution information, and that DPID is used to determine and provide one or more of the following resolution data items to the requesting entity.
Similarly, the ID Resolution System may provide resolution data in response to receiving a GPID from a requesting entity, or in response to determining a GPID based on other identifiers provided by a requesting entity. Such resolution data items may include:
Thus, various identifiers may be used to resolve and determine various resolution data, such as identifiers of particular devices, groups of devices, and any other profile information (e.g., profile data items associated with the DPID and/or GPIDs). The requesting entity may then use the resolution data to replace, supplement, or complement any other device, group, or user information the service provider may already have.
Provided below are three sample translation requests that might be received by the ID Resolution System, along with sample responses that may be returned. As noted elsewhere, these translations are only exemplary since there are a vast quantity of identifiers that may be received and resolved data items that may be returned in various embodiments.
Translation Between Service Providers
The ID Resolution System may further include an identifier translator to provide the ability for participating service providers to create segments within their own data systems and environments and seamlessly translate DPIDs into common or shared IDs which other third party partner entities may use. The identifier translator may also improve synchronization across multiple requesting entities by helping to ensure that each requesting entity stores a set of device attributes necessary to effectively bridge or link identifiers across platforms. In addition, service providers and partners may share common IDs, which can be device identifiers or traditional shared cookie IDs, which can be translated into DPIDs via the ID Resolution System. Further, in some embodiments where the ID Resolution System maintains a history of device identifiers mapped to a unique DPID, profiles between partners which have different sets of device identifiers may be matched to further enable linking among requesting entities. Thus, in some embodiments, the ID Resolution System may be configured to provide certain profile data to requesting entities that is obtained from profile data for the same device in other requesting entities profile data stores. For example, a sharing agreement may be developed by multiple requesting entities wherein each requesting entity maintains its own profile data store, but under certain conditions that profile data store may be accessed and information provided to other of the requesting entities.
Example Device Identifier Mapping Architecture
The ID Resolution System 100 may implement a resolution engine 122 that may perform the various process discussed herein, including device matching, group matching, identifier translation, etc. In the example of
In the example of
As discussed above, matching algorithms may be customized for or by the requesting entity 168 depending on the requesting entity 168's preferences. Device profiles may be generated, stored, and updated over time as new device identifiers and identifier types are provided to the ID Resolution System 100. Thus, a DPID for computing device 162A may over time be mapped to or associated with any type of device identifier (such as a later developed device identifier that did not exist when the DPID was assigned to computing device 162A) that the computing device 162A may use or be associated with. The requesting entity 168 may use the DPID instead of, or in conjunction with, pre-existing device identifiers ID1 . . . IDN, without the need to manage these different device identifiers and related identification techniques. Thus the ID Resolution System 100 provides a high degree of flexibility and customization for the requesting entity 168.
As introduced above, the ID Resolution System may apply a matching algorithm to determine whether device identifiers can be resolved to an existing device profile and, if not, create a new device profile. For example, an existing device (or “returning device”) refers to one that the system determines it has already seen before and for which it already created a device profile with a DPID. A new device is one that the system has not seen before (e.g., the system does not identify a match to a particular DPID with the requisite confidence level) and for which it may create a new profile and corresponding unique DPID in the profile data source 172. As discussed above, the matching algorithm used with reference to a particular requesting entity may be customized to a high degree of complexity based on, among other things, a combination of probabilistic and/or deterministic matching rules, such as a quantity of identifiers to be evaluated by a matching rule, algorithms and comparison functions used to compare identifiers, and weightings to determine when identifiers match or do not match. In various embodiments, matching rules can be audited to determine why a particular device matched at any point in time.
At action (1) of
At action (2), the requesting entity 168 provides the device identifiers to the ID Resolution System 100. The device identifiers may be provided as part of a request for a DPID unique to the requesting device, a GPID, and/or any other resolution information that may be associated with the device. In some embodiments, the device identifiers may be provided as part of a request for additional consumer or market segment data that may be associated with the computing device 162, which the requesting entity 168 may use to generate custom or targeted content for the user. In some embodiments, the identifiers are not received by the requesting entity 168, but instead are transmitted from the computing device 162 to the ID resolution system 100.
At actions (3)-(5), the ID Resolution System 100 performs a device matching process. In one embodiment, the ID Resolution System 100 receives the device identifiers and applies a matching algorithm associated with the requesting entity (and/or a default matching algorithm, in some instances). The matching algorithm may be stored and accessed from the matching algorithm data source(s) 170 and may include prioritized deterministic and/or probabilistic matching rules that were previously configured by the requesting entity 168 (e.g., via the user interfaces described further herein).
At action (4), the resolution engine 122 applies the matching algorithm by accessing device profiles of the requesting entity. Example processes of applying the matching algorithm and determining whether any device profiles match is described in more detail with reference to
At action (5), the resolution engine 122 identifies any matching device profiles found after applying at least one of the matching rules of the matching algorithm (e.g., the first matching rule that was satisfied). In some embodiments, actions (4) and (5) described above may be repeated or performed in parallel depending on the particular configuration for the matching algorithm being applied until a matching device profile is found or all matching rules are executed.
If no matches were identified after application of all applicable matching rules, the ID Resolution System may generate a new device profile associated with a new unique DPID. The ID Resolution System 100 can then store the device profile in the device profile data source(s) 172 for later access and subsequent device profile match requests.
If a matching device profile is found, or if a new device profile is created, at action (6) the ID Resolution System 100 provides the matching device profile (and/or other requested information) to the requesting entity 168. In some embodiments, the ID Resolution System 100 may access and provide additional data which may be associated with a device profile, such as consumer or other market segment data accessed from a consumer data store 174 (illustrated and described with reference to
In another embodiment, the unique DPID and/or associated device profile may be utilized to identify, determine, and/or cross-reference many other types of user data, such as resolution information discussed above and/or other information associated with the DPID and or one or more of various identifiers in the DPID profile from multiple disparate data sources that otherwise may not be linked or associated. Thus, for example, a requesting entity may use a DPID and/or associated device profile information to cross reference to an internal data source of devices and/or consumer activity, and determine that a particular device has been used to access both a mobile application and a website (either in the same session or in different sessions). In some embodiments this information may also be determined by the ID Resolution System 100 and provided as part of the device profile. In either scenario the requesting entity gains the knowledge that a same particular device is used to interact with the requesting entity in multiple, often disjoint ways, which can help reduce the device identity fragmentation and improve the customer relationship, as discussed above.
At action (7), the requesting entity can use the received device profile and/or additional consumer or market segment to authenticate a user login, approve a transaction, gather statistical data for analytics, generate a fraud risk score, and/or generate and/or provide targeted content, such as targeted advertising, marketing, special offers, and the like, to the computing device 162.
Examples Device Matching Processes
Determining a DPID
The process 500 begins at block 505, where the ID Resolution System 100 (for example, via the matching algorithm analysis module 122 of
In some embodiments, expansion of one or more device identifiers is performed as part of the matching process. Example expansion processes are described further with reference to
At block 510, the ID Resolution System 100 accesses a matching algorithm, which may comprise a rule set including multiple prioritized matching rules to be applied to the received device identifiers. The device profile matching rules may be accessed, for example, from the matching algorithm data source 170. As noted above, in one embodiment the device matching algorithm may be customized for, or by, the client requesting entity. For example, the user interfaces illustrated and described herein with respect to
In some embodiments, a matching algorithm may include one or more default matching rules, such as may be provided by the ID Resolution System 100. For example, default matching rules may be predetermined or generated over time by the ID Resolution System 100 based on observed device identifier patterns, common or recurring matching scenarios, best practices, or other criteria. In some embodiments, default matching rules may be provided (e.g., displayed via the user interfaces described herein) for the client requesting entity to select and/or customize.
In some embodiments, deterministic identifiers may be considered after probabilistic identifiers, or not at all. For example, in a fraud use case, the system may first analyze probabilistic identifiers, since a fraudster could spoof the deterministic identifiers of a device. Thus, matching on the deterministic identifiers would incorrectly indicate that the fraudster's device is the user's device. By looking at probabilistic identifiers in this use case, the system may not associate the receive identifiers with the user's device and may even associated the received identifiers with a DPID that is associated with other fraudulent activities.
The example matching algorithm of
Next, at block 515, the ID Resolution System 100 applies each respective matching rule to the received device identifiers. In embodiments where matching rules are applied in a priority order (discussed further below), the ID Resolution System 100 may only apply matching rules in order until a match is found (e.g., subsequent matching rules in the priority order need not be applied or evaluated). For example, device profiles that match the received device identifiers according to the matching rule may be accessed from the device profile data source(s) 172.
Matching rules may be applied in a particular order of priority, for example as specified by the client requesting entity or as specified by a default matching order. For example, matching rules may be applied in an order to first check for matches on deterministic identifiers, and then sequentially check for matches on probabilistic identifiers in gradually decreasing degrees of probability that a matching device profile may be identified. In another embodiment, the matching rules may be applied in any order (equal priority) or in parallel. In another embodiment, the ordering of the matching rules may be based on a likelihood that the particular ordering will help improve or optimize performance, such as based on machine learning that is applied to deterministic matches in order to identify probabilistic identifiers that are most likely to result in matches in future matching processes.
In the example of
Additionally, some of the rules in the example of
The velocity filter used in certain rules of the matching algorithm in
For those event data not excluded by a precondition, key requirement, or velocity requirement, one or more attribute qualifications of the rules are then applied. In general, each identifier attribute is associated with a corresponding match tier. If each attributes of a particular rule match the corresponding match tier for the attribute, the rule is matched (or “fired”). The description below with reference to
At block 520, the ID Resolution System 100 determines whether a matching device profile was found at block 515. In response to determining that no matching device profile was found, the process 500 may proceed to block 525. In response to determining that a matching device profile was found, the process 500 may proceed to block 530. In determining whether a matching device profile was found, a score indicating a degree, probability, likelihood, or other indication of a confidence level of the match may be generated and evaluated. For example, the match confidence level for a probabilistic rule that fired on a particular event data may be provided to the requesting entity to indicate the related degree of confidence that the returned device profile is associated with the device that initiated the online event. In other embodiments, the score may be a binary value (for example, yes or no, true or false), a letter score, a range or range of values, a percentage, and so on. The score may also be determined based in part on weightings associated with a degree of match for one or more device identifiers (e.g., in a fire and continue matching algorithm).
In a matching profile is not identified by the matching rule, the process 500 proceeds to block 525 to determine whether additional matching rules are available, such as a next lower priority matching rule. In response to determining that are no additional matching rules to apply, the process 500 may proceed to block 535. In response to determining there is at least one additional matching rule to apply, the process 500 may return to block 515 to apply the next matching rule. Thus, the process 500 may repeat blocks 515 through 525 an indeterminate number of times until a matching device profile is identified, or until all rules are evaluated in a fire and continue matching algorithm.
At block 530, in response identifying a matching device profile, the ID Resolution System 100 provides the DPID associated with the matching device profile (and/or other information associated with the device), to the client requesting entity. For example, some or all of the matching device profile may be provided along with or in association with the DPID. The client requesting entity may then use the DPID and/or device profile for various purposes, such as those discuss herein (e.g., user/device authentication, determining targeted content to provide to the device, etc.).
In some embodiments, additional data may be associated with a device profile, such as consumer or other market segment data accessed from the consumer data store 174. For example, a DPID may be associated with a particular individual or market segment (e.g., the DPID may be associated with an IP address, a country code, or other identifier which may be associated with or used to access market segment data for that IP address, country code, or other identifier). Thus, in addition to providing a DPID and/or device profile to the client requesting entity, the ID Resolution System 100 may access and provide additional consumer or segment data which the client requesting entity can use to further refine its authentication, security, targeted advertising, marketing, audience building, and/or other uses of the requested device information.
At block 535, in response to the matching algorithm not identifying any matching profiles, ID Resolution System 100 generates a new device profile and DPID to be associated with the received event data (e.g., multiple device identifiers). The new device profile and DPID mapping may then be stored, for example, in the device profile data source(s) 172 for later access, retrieval, and subsequent device profile matching in response to further requests from the requesting entities 168. The process 500 may then proceed to block 530 where the ID Resolution System 100 provides the DPID associated with the new device profile to the client requesting entity, as described above.
Example Device Identifiers
Without limiting the scope of the possible format and content of a device profile, below are provided two examples of device profiles that may be stored by the ID Resolution System 100 and/or one or more requesting entities that interact with the associated device and/or user.
Applying Matching Rules and Match Tiers
Beginning at block 610, the ID Resolution System 100 evaluates one or more tier conditions, each including an identifier and an associated match tier, associated with the matching rule. Each tier condition may indicate a comparison function (e.g., “true”, less than, greater than, within range, etc.) that must be met by the associated attribute for the condition to be matched. If a rule includes multiple tier conditions, all of the attributes must meet the corresponding match tiers for the rule to be matched.
At block 615, the ID Resolution System 100 applies or evaluates a tier condition based on the received event data. In the example of
A non-limiting list of example identifier comparison functions which may be implemented are further described below. The identifier comparison function used may vary depending on the type of identifier being evaluated for a potential match. In one instance, integer-based device identifiers may be evaluated using, for example, an EXACTMATCH function or a DIFFERENCE function, among others. In another instance, a string-based device identifier may be evaluated using, for example, an EXACTMATCH function or an EXISTS function, among others. The identifier comparison function used for a particular identifier may be selected or customized by the requesting entity, for example via the identifier configuration user interface(s) illustrated and described with reference to
At block 620, the ID Resolution System 100 determines whether the respective match tier was satisfied at block 615. In response to determining that the respective tier condition was not satisfied, the process 600 may proceed to block 630. In response to determining that the respective tier condition was satisfied, the process 600 may proceed to block 625.
At block 625, the ID Resolution System 100 determines whether there are any additional tier conditions to evaluate with reference to the match rule. In response to determining that no additional match tiers remain to evaluate, the process 600 may proceed to block 635. In response to determining there is at least one additional tier condition to evaluate, the process 600 may return to block 615 to apply the next tier condition. Thus, the process 600 may repeat blocks 615 through 625 an indeterminate number of times until all tier conditions are matched with reference to a particular profile and the method continues to block 630, or until one of the tier conditions is not matched and the method continues to block 635.
At block 630, the ID Resolution System 100 provides an indication that the device profile matching rule is not satisfied.
At block 635, in response to determining that each tier condition was satisfied, the ID Resolution System 100 provides an indication that the matching rule is satisfied.
Example Identifier Comparison Functions
The following provides a non-exhaustive sample listing of some identifier comparison functions which may be associated with particular matching rules and/or match tiers and implemented by the ID Resolution System 100 in one or more embodiments. In some embodiments, more or less additional identifier comparison functions may be implemented.
In one embodiment, one or more dashboard user interfaces may be provided by the ID Resolution System in order to provide requesting entities, such as fraud analysists, content providers, online marketers, and/or other requesting entities, with the ability to manage various aspects of device matching, group matching, ID translations, partner translation, etc., that might be performed by the ID Resolution System. The dashboard user interfaces may enable clients to, for example: manage and configure identifiers and identifier mapping rules; perform testing of matching rules to assess their effectiveness and accuracy; perform device profile lookups; access analytic reports related to the efficacy of device level IDs and matching rules; view or receive monitoring status updates, notifications, and/or alarms; and perform other administrative functions.
The sample list of Identifiers show in user interface 700 may represent identifiers which are available for use in various matching rules and rule sets as further discussed below. Thus, in one embodiment, at least one Identifier must first be created, for example via the user interface 700, before a match rule can be configured based on that Identifier.
The user interface 800 may also provide a list of configurable tier conditions and settings associated with the Identifier. The user may add as many match tiers as desired by selecting the Add Match Tier option, which may present additional user interface options for the user to specify match settings for the new match tier. Match tier settings may include, for example, a minimum range integer value, a maximum range integer value, a minimum range double precision value, a maximum range double precision value, a minimum range string value, a maximum range string value, and/or a Boolean value. Depending on the type of the Identifier, some of the match tier settings may not be available. For example, max and min range integer values may not be applicable for string-based types of identifiers. Each match tier may include user-selectable options to Edit or Delete the respective match tier.
Once one or more tier conditions have been configured and saved for the Identifier, the tier conditions will be available for use in configuring respective matching rules, further discussed with reference to
Using one or more expansion techniques, even if only the Identifier of a first type is received in a set of device identifiers for evaluation, the Identifier may be translated and evaluated for further matching with respect to the one or more additional types. As an example, an IP address may be eligible for translation into one, two, or three octets of the IP address. This may be of benefit to a requesting entity which may prefer to configure a matching rule based only on a portion of the IP address (e.g., one or more octets) and not the IP address as a whole (e.g., a matching rule configured to match only on a single octet may produce a higher number of match results than a matching rule configured to match on the entire IP address). In another example, an IP address may be eligible to be translated to a country code. Depending on the particular Identifier, other translations may be available, such as for other identifiers which may represent a combination of information which may be parsed and evaluated separately.
User interface 900 illustrates two example expansions available for translation; more or less expansions may be available depending on the type of Identifier. For example, one expansion is for an “IP Octet” type and a second expansion is for an “IP to Country” type, each with a corresponding list of child Identifiers to be used for the respective expansion. Child Identifiers may be selected from the list of previously configured Identifiers (e.g., the list of identifiers illustrated in
Example System Implementation and Architecture
In the embodiment of
The resolution engine 122 provides capabilities related to device identifier mapping and resolution determination processes described herein, including the processes described for example, with reference to
The interface module 124 provides capabilities related to interfacing between the ID Resolution System 100, the requesting entity computing systems 168, and various data sources 170, 172, and 174. For example, the interface module 124 may be configured to provide, to the requesting entity, various client-side scripts configured to collect device identifier information, which may in turn be installed as part of a web service provided by the requesting entity. The interface module 124 may further be configured to receive data via the client-side scripts for further processing by the various other modules described herein. The interface module 124 may further be configured to generate and provide various console configuration options and user interfaces, including but not limited to the user interfaces illustrated and described with reference to
The data partition and security module 126 provides capabilities related to ensuring that device identifier accessed or received from various client/requesting entity systems 168 and/or data sources 170, 172 and 174 are strictly separated or partitioned to maintain data privacy for each respective requesting entity.
The analytics and reporting module 128 provides capabilities related to generating and providing various analytics and reports related to the device identifier mapping processes described herein. For example, matching rules may be analyzed to assess their efficacy, calculate statistical data related to the number and frequency by which device identifiers are mapped to unique device IDs, and other information which may be useful to requesting entities (e.g., for auditing and making improvements identifier mapping rules over time). Some or all of the data to be analyzed and reported on may be stored in any of the data sources 170, 172, and/or 174 described further below.
The ID Resolution System 100 includes, for example, a server, workstation, or other computing device. In one embodiment, the exemplary ID Resolution System 100 includes one or more central processing units (“CPU”) 150, which may each include a conventional or proprietary microprocessor. The ID Resolution System 100 further includes one or more memories 130, such as random access memory (“RAM”) for temporary storage of information, one or more read only memories (“ROM”) for permanent storage of information, and one or more mass storage device 120, such as a hard drive, diskette, solid state drive, or optical media storage device. Typically, the modules of the ID Resolution System 100 are connected to the computer using a standard based bus system. In different embodiments, the standard based bus system could be implemented in Peripheral Component Interconnect (“PCI”), Microchannel, Small Computer System Interface (“SCSI”), Industrial Standard Architecture (“ISA”) and Extended ISA (“EISA”) architectures, for example. In addition, the functionality provided for in the components and modules of ID Resolution System 100 may be combined into fewer components and modules or further separated into additional components and modules. Each of the devices discussed herein, such as the user computing devices 168 may include any combination of the components discussed herein with reference to the ID Resolution System 100.
The ID Resolution System 100 is generally controlled and coordinated by operating system software, such as Windows XP, Windows Vista, Windows 7, Windows 8, Windows Server, Unix, Linux, SunOS, Solaris, iOS, Blackberry OS, or other compatible operating systems. In Macintosh systems, the operating system may be any available operating system, such as MAC OS X. In other embodiments, the ID Resolution System 100 may be controlled by a proprietary operating system. Conventional operating systems control and schedule computer processes for execution, perform memory management, provide file system, networking, I/O services, and provide a user interface, such as a graphical user interface (“GUI”), among other things.
The exemplary ID Resolution System 100 may include one or more commonly available input/output (I/O) devices and interfaces 110, such as a keyboard, mouse, touchpad, and printer. In one embodiment, the I/O devices and interfaces 110 include one or more display devices, such as a monitor, that allows the visual presentation of data to a user. More particularly, a display device provides for the presentation of GUIs, application software data, and multimedia analytics, for example. The ID Resolution System 100 may also include one or more multimedia devices 140, such as speakers, video cards, graphics accelerators, and microphones, for example.
Network
In the embodiment of
According to
Matching Algorithm Data Sources
The matching algorithm data source(s) 170 may store, for example, identifier and/or device profile matching rules. Matching rules may be customized by the client requesting entity according to, for example, the client requesting entity's business, marketing preferences, and/or tolerances for allowing matching of probabilistic identifiers that are more or less unique. One example rule configuration user interface for configuring a rule set and one or more matching rules is illustrated and described with reference to
Device Profile Data Sources
The device profile data source(s) 172 may store, for example, device profiles associated with multiple devices, as well as group profiles in some implementations. The device profiles stored in device profile data source(s) 172 may, for example, include the unique device identifier and a history of all device identifiers which have been associated or mapped to the unique device identifier over time, including frequency of use and other statistical information which may be useful for auditing and improvements identifier mapping rules over time. For example, the device profile may also include any matching pattern used to link or map device identifiers and/or sessions to the unique device identifier. In some embodiments, auditing information and reports may be generated by the ID Resolution System to provide a summary of the frequency of each match pattern being applied and corresponding mapping results. Device profiles may also include privacy choices (e.g., opt-out and/or opt-in choices) associated with respective devices and/or user(s) of the respective devices.
The device profiles may be stored in any type of suitable data structure or data format.
Consumer Data Sources
The consumer data sources 174 may store, for example, credit bureau data (for example, credit bureau data from File One℠) and/or other consumer data. Consumer data sources 166B may also store demographic and consumer segment data that include one or more models, such as models that identify lifestyle and/or socio-economic attributes (e.g., MOSAIC® segmentation and/or codes) and/or behavioral/attitudinal/psychographic attributes (e.g., TrueTouch℠ Touch Points segmentation), some or all of which may be associated with a geographic location, IP address information or ranges, and other types of attributes.
Additional Benefits and Advantages
In addition to the benefits and advantages described above, additional advantages of various embodiments described herein may be achieved, including but not limited to:
Each of the processes, methods, and algorithms described in the preceding sections may be embodied in, and fully or partially automated by, code modules executed by one or more computer systems or computer processors comprising computer hardware. The code modules may be stored on any type of non-transitory computer-readable medium or computer storage device, such as hard drives, solid state memory, optical disc, and/or the like. The systems and modules may also be transmitted as generated data signals (for example, as part of a carrier wave or other analog or digital propagated signal) on a variety of computer-readable transmission mediums, including wireless-based and wired/cable-based mediums, and may take a variety of forms (for example, as part of a single or multiplexed analog signal, or as multiple discrete digital packets or frames). The processes and algorithms may be implemented partially or wholly in application-specific circuitry. The results of the disclosed processes and process steps may be stored, persistently or otherwise, in any type of non-transitory computer storage such as, for example, volatile or non-volatile storage.
In general, the word “module,” as used herein, refers to logic embodied in hardware or firmware, or to a collection of software instructions, possibly having entry and exit points, written in a programming language, such as, for example, Java, Lua, C or C++. A software module may be compiled and linked into an executable program, installed in a dynamic link library, or may be written in an interpreted programming language such as, for example, BASIC, Perl, or Python. It will be appreciated that software modules may be callable from other modules or from themselves, and/or may be invoked in response to detected events or interrupts. Software modules configured for execution on computing devices may be provided on a computer readable medium, such as a compact disc, digital video disc, flash drive, or any other tangible medium. Such software code may be stored, partially or fully, on a memory device of the executing computing device, such as the ID Resolution System 100, for execution by the computing device. Software instructions may be embedded in firmware, such as an EPROM. It will be further appreciated that hardware modules may be comprised of connected logic units, such as gates and flip-flops, and/or may be comprised of programmable units, such as programmable gate arrays or processors. The modules described herein are preferably implemented as software modules, but may be represented in hardware or firmware. Generally, the modules described herein refer to logical modules that may be combined with other modules or divided into sub-modules despite their physical organization or storage.
The various features and processes described above may be used independently of one another, or may be combined in various ways. All possible combinations and subcombinations are intended to fall within the scope of this disclosure. In addition, certain method or process blocks may be omitted in some implementations. The methods and processes described herein are also not limited to any particular sequence, and the blocks or states relating thereto can be performed in other sequences that are appropriate. For example, described blocks or states may be performed in an order other than that specifically disclosed, or multiple blocks or states may be combined in a single block or state. The example blocks or states may be performed in serial, in parallel, or in some other manner. Blocks or states may be added to or removed from the disclosed example embodiments. The example systems and components described herein may be configured differently than described. For example, elements may be added to, removed from, or rearranged compared to the disclosed example embodiments.
Conditional language used herein, such as, among others, “can,” “could,” “might,” “may,” “for example,” and the like, unless specifically stated otherwise, or otherwise understood within the context as used, is generally intended to convey that certain embodiments include, while other embodiments do not include, certain features, elements and/or steps. Thus, such conditional language is not generally intended to imply that features, elements and/or steps are in any way required for one or more embodiments or that one or more embodiments necessarily include logic for deciding, with or without author input or prompting, whether these features, elements and/or steps are included or are to be performed in any particular embodiment. The terms “comprising,” “including,” “having,” and the like are synonymous and are used inclusively, in an open-ended fashion, and do not exclude additional elements, features, acts, operations, and so forth. Also, the term “or” is used in its inclusive sense (and not in its exclusive sense) so that when used, for example, to connect a list of elements, the term “or” means one, some, or all of the elements in the list. Conjunctive language such as the phrase “at least one of X, Y and Z,” unless specifically stated otherwise, is otherwise understood with the context as used in general to convey that an item, term, etc. may be either X, Y or Z. Thus, such conjunctive language is not generally intended to imply that certain embodiments require at least one of X, at least one of Y and at least one of Z to each be present.
While certain example embodiments have been described, these embodiments have been presented by way of example only, and are not intended to limit the scope of the disclosure. Thus, nothing in the foregoing description is intended to imply that any particular element, feature, characteristic, step, module, or block is necessary or indispensable. Indeed, the novel methods and systems described herein may be embodied in a variety of other forms; furthermore, various omissions, substitutions and changes in the form of the methods and systems described herein may be made without departing from the spirit of the inventions disclosed herein. The accompanying claims and their equivalents are intended to cover such forms or modifications as would fall within the scope and spirit of certain of the inventions disclosed herein.
Any process descriptions, elements, or blocks in the flow diagrams described herein and/or depicted in the attached figures should be understood as potentially representing modules, segments, or portions of code which include one or more executable instructions for implementing specific logical functions or steps in the process. Alternate implementations are included within the scope of the embodiments described herein in which elements or functions may be deleted, executed out of order from that shown or discussed, including substantially concurrently or in reverse order, depending on the functionality involved, as would be understood by those skilled in the art.
All of the methods and processes described above may be embodied in, and partially or fully automated via, software code modules executed by one or more general purpose computers. For example, the methods described herein may be performed by the ID Resolution System 100 and/or any other suitable computing device. The methods may be executed on the computing devices in response to execution of software instructions or other executable code read from a tangible computer readable medium. A tangible computer readable medium is a data storage device that can store data that is readable by a computer system. Examples of computer readable mediums include read-only memory, random-access memory, other volatile or non-volatile memory devices, CD-ROMs, magnetic tape, flash drives, and optical data storage devices.
It should be emphasized that many variations and modifications may be made to the above-described embodiments, the elements of which are to be understood as being among other acceptable examples. All such modifications and variations are intended to be included herein within the scope of this disclosure. The foregoing description details certain embodiments of the invention. It will be appreciated, however, that no matter how detailed the foregoing appears in text, the invention can be practiced in many ways. As is also stated above, it should be noted that the use of particular terminology when describing certain features or aspects of the invention should not be taken to imply that the terminology is being re-defined herein to be restricted to including any specific characteristics of the features or aspects of the invention with which that terminology is associated.
| Number | Name | Date | Kind |
|---|---|---|---|
| 4805222 | Young et al. | Feb 1989 | A |
| 4912761 | Tan et al. | Mar 1990 | A |
| 4924387 | Jeppesen | May 1990 | A |
| 5184849 | Taylor | Feb 1993 | A |
| 5491735 | Hsieh | Feb 1996 | A |
| 5519827 | Mizushima | May 1996 | A |
| 5521907 | Ennis, Jr. | May 1996 | A |
| 5557686 | Brown et al. | Sep 1996 | A |
| 5627886 | Bowman | May 1997 | A |
| 5679940 | Templeton et al. | Oct 1997 | A |
| 5721765 | Smith | Feb 1998 | A |
| 5724424 | Giffor | Mar 1998 | A |
| 5748740 | Curry et al. | May 1998 | A |
| 5748780 | Stolfo et al. | May 1998 | A |
| 5764275 | Lappington et al. | Jun 1998 | A |
| 5802156 | Felger | Sep 1998 | A |
| 5819226 | Gopinathan et al. | Oct 1998 | A |
| 5864620 | Pettitt | Jan 1999 | A |
| 5884289 | Anderson et al. | Mar 1999 | A |
| 5886334 | D'Entremont | Mar 1999 | A |
| 5892900 | Ginter et al. | Apr 1999 | A |
| 5894510 | Felger | Apr 1999 | A |
| 5899980 | Wilf et al. | May 1999 | A |
| 5903721 | Sixtus | May 1999 | A |
| 5933480 | Felger | Aug 1999 | A |
| 5960069 | Felger | Sep 1999 | A |
| 6009523 | Owaki et al. | Dec 1999 | A |
| 6029154 | Pettitt | Feb 2000 | A |
| 6029159 | Zorba et al. | Feb 2000 | A |
| 6062474 | Kroll | May 2000 | A |
| 6078907 | Lamm | Jun 2000 | A |
| 6092053 | Boesch et al. | Jul 2000 | A |
| 6094643 | Anderson et al. | Jul 2000 | A |
| 6105012 | Chang et al. | Aug 2000 | A |
| 6112240 | Pogue et al. | Aug 2000 | A |
| 6148407 | Aucsmith | Nov 2000 | A |
| 6151593 | Cho et al. | Nov 2000 | A |
| 6163604 | Baulier et al. | Dec 2000 | A |
| 6163771 | Walker et al. | Dec 2000 | A |
| 6164528 | Hills et al. | Dec 2000 | A |
| 6205436 | Rosenberg et al. | Mar 2001 | B1 |
| 6209104 | Jalili | Mar 2001 | B1 |
| 6216153 | Vortriede | Apr 2001 | B1 |
| 6223289 | Wall et al. | Apr 2001 | B1 |
| 6282276 | Felger | Aug 2001 | B1 |
| 6295605 | Dockter et al. | Sep 2001 | B1 |
| 6327384 | Hirao et al. | Dec 2001 | B1 |
| 6330546 | Gopinathan et al. | Dec 2001 | B1 |
| 6405922 | Kroll | Jun 2002 | B1 |
| 6442529 | Krishan et al. | Aug 2002 | B1 |
| 6442692 | Zilberman | Aug 2002 | B1 |
| 6457021 | Berkowitz et al. | Sep 2002 | B1 |
| 6480710 | Laybourn et al. | Nov 2002 | B1 |
| 6509847 | Anderson | Jan 2003 | B1 |
| 6523019 | Borthwick | Feb 2003 | B1 |
| 6546493 | Magdych et al. | Apr 2003 | B1 |
| 6553108 | Felger | Apr 2003 | B1 |
| 6560455 | Amin et al. | May 2003 | B2 |
| 6567099 | Dawson | May 2003 | B1 |
| 6597775 | Lawyer et al. | Jul 2003 | B2 |
| 6646765 | Barker et al. | Nov 2003 | B1 |
| 6678666 | Boulware | Jan 2004 | B1 |
| 6687390 | Avni et al. | Feb 2004 | B2 |
| 6689055 | Mullen et al. | Feb 2004 | B1 |
| 6718363 | Ponte | Apr 2004 | B1 |
| 6745333 | Thomsen | Jun 2004 | B1 |
| 6803920 | Gossett et al. | Oct 2004 | B2 |
| 6804624 | Silverman | Oct 2004 | B2 |
| 6850606 | Lawyer et al. | Feb 2005 | B2 |
| 6892307 | Wood et al. | May 2005 | B1 |
| 6895507 | Tepler | May 2005 | B1 |
| 6895514 | Kermani | May 2005 | B1 |
| 6898709 | Teppler | May 2005 | B1 |
| 6908030 | Rajasekaran et al. | Jun 2005 | B2 |
| 6937569 | Sarkar et al. | Aug 2005 | B1 |
| 6947978 | Huffman | Sep 2005 | B2 |
| 6954532 | Handley et al. | Oct 2005 | B1 |
| 6957185 | Labaton | Oct 2005 | B1 |
| 6957339 | Shinzaki | Oct 2005 | B2 |
| 7002712 | Barker et al. | Feb 2006 | B2 |
| 7003670 | Heaven et al. | Feb 2006 | B2 |
| 7007174 | Wheeler et al. | Feb 2006 | B2 |
| 7013001 | Felger | Mar 2006 | B1 |
| 7027800 | Haumont et al. | Apr 2006 | B2 |
| 7039505 | Southard et al. | May 2006 | B1 |
| 7039699 | Narin et al. | May 2006 | B1 |
| 7043640 | Pritchard et al. | May 2006 | B2 |
| 7089310 | Ellerman et al. | Aug 2006 | B1 |
| 7089585 | Dharmarajan | Aug 2006 | B1 |
| 7096192 | Pettitt | Aug 2006 | B1 |
| 7100049 | Gasparini et al. | Aug 2006 | B2 |
| 7103570 | Morea et al. | Sep 2006 | B1 |
| 7120590 | Eisen et al. | Oct 2006 | B1 |
| 7130858 | Ciaramitaro et al. | Oct 2006 | B2 |
| 7143095 | Barrett et al. | Nov 2006 | B2 |
| 7158622 | Lawyer et al. | Jan 2007 | B2 |
| 7165051 | Ronning et al. | Jan 2007 | B2 |
| 7174454 | Roskind | Feb 2007 | B2 |
| 7191467 | Dujari et al. | Mar 2007 | B1 |
| 7197646 | Fritz et al. | Mar 2007 | B2 |
| 7221949 | Clough | May 2007 | B2 |
| 7225974 | Yamauchi | Jun 2007 | B2 |
| 7237717 | Rao et al. | Jul 2007 | B1 |
| 7249093 | King | Jul 2007 | B1 |
| 7251624 | Lee et al. | Jul 2007 | B1 |
| 7260837 | Abraham et al. | Aug 2007 | B2 |
| 7263492 | Suresh et al. | Aug 2007 | B1 |
| 7263506 | Lee et al. | Aug 2007 | B2 |
| 7272610 | Torres | Sep 2007 | B2 |
| 7272728 | Pierson et al. | Sep 2007 | B2 |
| 7292723 | Tedesco et al. | Nov 2007 | B2 |
| 7293096 | Foltak et al. | Nov 2007 | B1 |
| 7296088 | Padmanabhan et al. | Nov 2007 | B1 |
| 7330824 | Kanojia et al. | Feb 2008 | B1 |
| 7330871 | Barber | Feb 2008 | B2 |
| 7340045 | Felger | Mar 2008 | B2 |
| 7346551 | Pe Jimenez et al. | Mar 2008 | B2 |
| 7346775 | Gasparinl et al. | Mar 2008 | B2 |
| 7349955 | Korb et al. | Mar 2008 | B1 |
| 7363170 | Seul et al. | Apr 2008 | B2 |
| 7373669 | Eisen | May 2008 | B2 |
| 7376618 | Anderson et al. | May 2008 | B1 |
| 7379891 | Donner et al. | May 2008 | B1 |
| 7404087 | Teunen | Jun 2008 | B2 |
| 7401082 | Keene et al. | Jul 2008 | B2 |
| 7403922 | Lewis et al. | Jul 2008 | B1 |
| 7428587 | Rowland et al. | Sep 2008 | B2 |
| 7436780 | Stephens | Oct 2008 | B2 |
| 7438226 | Helsper et al. | Oct 2008 | B2 |
| 7447494 | Law et al. | Nov 2008 | B2 |
| 7451487 | Oliver et al. | Nov 2008 | B2 |
| 7457401 | Lawyer et al. | Nov 2008 | B2 |
| 7457823 | Shraim et al. | Nov 2008 | B2 |
| 7475242 | Baird et al. | Jan 2009 | B2 |
| 7478182 | Schweig | Jan 2009 | B2 |
| 7487350 | Utin | Feb 2009 | B2 |
| 7496752 | Yamaguchi et al. | Feb 2009 | B2 |
| 7497374 | Helsper et al. | Mar 2009 | B2 |
| 7502610 | Maher | Mar 2009 | B2 |
| 7502933 | Jakobsson et al. | Mar 2009 | B2 |
| 7526796 | Lulich et al. | Apr 2009 | B2 |
| 7543740 | Greene et al. | Jun 2009 | B2 |
| 7552090 | Barber | Jun 2009 | B1 |
| 7555458 | Felger | Jun 2009 | B1 |
| 7562221 | Nyström et al. | Jul 2009 | B2 |
| 7577620 | Donner | Aug 2009 | B1 |
| 7581112 | Brown et al. | Aug 2009 | B2 |
| 7606560 | Labrou et al. | Oct 2009 | B2 |
| 7657626 | Zwicky | Feb 2010 | B1 |
| 7660902 | Graham et al. | Feb 2010 | B2 |
| 7665140 | Oliver et al. | Feb 2010 | B2 |
| 7665658 | Fields | Feb 2010 | B2 |
| 7673793 | Greene et al. | Mar 2010 | B2 |
| 7685629 | White et al. | Mar 2010 | B1 |
| 7698743 | Ohmori et al. | Apr 2010 | B2 |
| 7708200 | Helsper et al. | May 2010 | B2 |
| 7711846 | Padmanabhan et al. | May 2010 | B2 |
| 7739402 | Roese et al. | Jun 2010 | B2 |
| 7739512 | Hawkes | Jun 2010 | B2 |
| 7743409 | Gonzalez et al. | Jun 2010 | B2 |
| 7752084 | Pettitt | Jul 2010 | B2 |
| 7756783 | Crooks | Jul 2010 | B2 |
| 7761379 | Zoldi et al. | Jul 2010 | B2 |
| 7778846 | Suresh et al. | Aug 2010 | B2 |
| 7813937 | Pathria et al. | Oct 2010 | B1 |
| 7813944 | Luk et al. | Oct 2010 | B1 |
| 7849029 | Crooks et al. | Dec 2010 | B2 |
| 7849307 | Roskind | Dec 2010 | B2 |
| 7853526 | Milana | Dec 2010 | B2 |
| 7853533 | Eisen | Dec 2010 | B2 |
| 7856372 | Ullah | Dec 2010 | B2 |
| 7860783 | Yang et al. | Dec 2010 | B2 |
| 7861260 | Shkedi | Dec 2010 | B2 |
| 7865427 | Wright et al. | Jan 2011 | B2 |
| 7882217 | Katzir | Feb 2011 | B2 |
| 7908223 | Klein et al. | Mar 2011 | B2 |
| 7908645 | Varghese et al. | Mar 2011 | B2 |
| 7930285 | Abraham et al. | Apr 2011 | B2 |
| 7933984 | Smith et al. | Apr 2011 | B1 |
| 7937467 | Barber | May 2011 | B2 |
| 7940929 | Sengupta | May 2011 | B1 |
| 7945494 | Williams | May 2011 | B2 |
| 7945515 | Zoldi et al. | May 2011 | B2 |
| 7949564 | Hughes et al. | May 2011 | B1 |
| 7958029 | Bobich et al. | Jun 2011 | B1 |
| 7958246 | Barber | Jun 2011 | B2 |
| 7961857 | Zoldi et al. | Jun 2011 | B2 |
| 7970701 | Lewis et al. | Jun 2011 | B2 |
| 7983691 | Wong et al. | Jul 2011 | B1 |
| 7991716 | Crooks et al. | Aug 2011 | B2 |
| 7995996 | Link, II et al. | Aug 2011 | B2 |
| 8001376 | Utin | Aug 2011 | B2 |
| 8001597 | Crooks | Aug 2011 | B2 |
| 8015614 | Matsuzaki et al. | Sep 2011 | B2 |
| 8015921 | Leppanen et al. | Sep 2011 | B2 |
| 8019678 | Wright et al. | Sep 2011 | B2 |
| 8020763 | Kowalchyk et al. | Sep 2011 | B1 |
| 8024266 | Barber | Sep 2011 | B1 |
| 8025220 | Zoldi et al. | Sep 2011 | B2 |
| 8027439 | Zoldi et al. | Sep 2011 | B2 |
| 8032448 | Anderson et al. | Oct 2011 | B2 |
| 8037097 | Guo et al. | Oct 2011 | B2 |
| 8037511 | Lundy et al. | Oct 2011 | B1 |
| 8041597 | Li et al. | Oct 2011 | B2 |
| 8042164 | Sheynblat et al. | Oct 2011 | B2 |
| 8046271 | Jimenez et al. | Oct 2011 | B2 |
| 8060922 | Crichton et al. | Nov 2011 | B2 |
| 8065233 | Lee et al. | Nov 2011 | B2 |
| 8090648 | Zoldi et al. | Jan 2012 | B2 |
| 8108378 | Ott, IV et al. | Jan 2012 | B2 |
| 8121962 | Vaiciulis et al. | Feb 2012 | B2 |
| 8122082 | Klein | Feb 2012 | B2 |
| 8126816 | Mu et al. | Feb 2012 | B2 |
| 8131615 | Diev et al. | Mar 2012 | B2 |
| 8140689 | Barber | Mar 2012 | B2 |
| 8141148 | Thomas et al. | Mar 2012 | B2 |
| 8145560 | Kulkarni et al. | Mar 2012 | B2 |
| 8145762 | Barber | Mar 2012 | B2 |
| 8150968 | Barber | Apr 2012 | B2 |
| 8151327 | Eisen | Apr 2012 | B2 |
| 8166068 | Stevens | Apr 2012 | B2 |
| 8175897 | Lee et al. | May 2012 | B2 |
| 8176178 | Thomas et al. | May 2012 | B2 |
| 8180686 | Ryu et al. | May 2012 | B2 |
| 8181015 | Roskind | May 2012 | B2 |
| 8185953 | Rothstein et al. | May 2012 | B2 |
| 8190513 | Felger | May 2012 | B2 |
| 8190529 | Abe et al. | May 2012 | B2 |
| 8191148 | Oliver et al. | May 2012 | B2 |
| 8201099 | Osbourn et al. | Jun 2012 | B1 |
| 8204833 | Mu et al. | Jun 2012 | B2 |
| 8209744 | Zhu et al. | Jun 2012 | B2 |
| 8213898 | Choti et al. | Jul 2012 | B2 |
| 8214232 | Tyler et al. | Jul 2012 | B2 |
| 8214285 | Hu et al. | Jul 2012 | B2 |
| 8219415 | Tyler et al. | Jul 2012 | B2 |
| 8224348 | Bolon et al. | Jul 2012 | B2 |
| 8229844 | Felger | Jul 2012 | B2 |
| 8250631 | Iyengar et al. | Aug 2012 | B2 |
| 8266295 | Klein et al. | Sep 2012 | B2 |
| 8271891 | Osbourn et al. | Sep 2012 | B1 |
| 8280833 | Miltonberger | Oct 2012 | B2 |
| 8290838 | Thakur et al. | Oct 2012 | B1 |
| 8295898 | Ashfield et al. | Oct 2012 | B2 |
| 8296228 | Kloor | Oct 2012 | B1 |
| 8296229 | Yellin et al. | Oct 2012 | B1 |
| 8296245 | Barber et al. | Oct 2012 | B2 |
| 8296250 | Crooks et al. | Oct 2012 | B2 |
| 8306933 | Kawai et al. | Nov 2012 | B2 |
| 8311907 | Klein et al. | Nov 2012 | B2 |
| 8321269 | Linden et al. | Nov 2012 | B2 |
| 8326759 | Hammad | Dec 2012 | B2 |
| 8326760 | Ma et al. | Dec 2012 | B2 |
| 8326763 | Zuili | Dec 2012 | B2 |
| 8332338 | Vaiciulis et al. | Dec 2012 | B2 |
| 8332522 | Barber | Dec 2012 | B2 |
| 8370253 | Grossman et al. | Feb 2013 | B1 |
| 8370638 | Duane et al. | Feb 2013 | B2 |
| 8380831 | Barber | Feb 2013 | B2 |
| 8392987 | Sasamura et al. | Mar 2013 | B2 |
| 8407112 | Walter | Mar 2013 | B2 |
| 8417587 | Jimenez et al. | Apr 2013 | B2 |
| 8423458 | Barber | Apr 2013 | B2 |
| 8424061 | Rosenor | Apr 2013 | B2 |
| 8429070 | Hu et al. | Apr 2013 | B2 |
| 8443202 | White et al. | May 2013 | B2 |
| 8452715 | Barber | May 2013 | B2 |
| 8453226 | Hammad | May 2013 | B2 |
| 8462161 | Barber | Jun 2013 | B1 |
| 8464290 | Beyda et al. | Jun 2013 | B2 |
| 8468582 | Kuang et al. | Jun 2013 | B2 |
| 8484470 | Sakakihara et al. | Jul 2013 | B2 |
| 8495714 | Jones et al. | Jul 2013 | B2 |
| 8516439 | Brass et al. | Aug 2013 | B2 |
| 8539070 | Barber | Sep 2013 | B2 |
| 8543522 | Ryman-Tubb et al. | Sep 2013 | B2 |
| 8548137 | Zoldi et al. | Oct 2013 | B2 |
| 8559607 | Zoldi et al. | Oct 2013 | B2 |
| 8567669 | Griegel et al. | Oct 2013 | B2 |
| 8588816 | Collins | Nov 2013 | B2 |
| 8601109 | Johannsen | Dec 2013 | B2 |
| 8611856 | Yan et al. | Dec 2013 | B2 |
| 8612854 | Eisen et al. | Dec 2013 | B2 |
| 8660539 | Khambete et al. | Feb 2014 | B2 |
| 8683561 | Utin | Mar 2014 | B2 |
| 8688543 | Dominguez | Apr 2014 | B2 |
| 8751815 | Lunde et al. | Jun 2014 | B2 |
| 8762283 | Gerber et al. | Jun 2014 | B2 |
| 8762574 | Barber | Jun 2014 | B2 |
| 8763113 | Thomas et al. | Jun 2014 | B2 |
| 8776225 | Pierson et al. | Jul 2014 | B2 |
| 8779981 | Eisen et al. | Jul 2014 | B2 |
| 8781975 | Bennett et al. | Jul 2014 | B2 |
| 8782783 | Thomas et al. | Jul 2014 | B2 |
| 8799458 | Barber | Aug 2014 | B2 |
| 8817984 | Miller et al. | Aug 2014 | B2 |
| 8826393 | Eisen | Sep 2014 | B2 |
| 8838478 | Kretz et al. | Sep 2014 | B2 |
| 8838967 | Mills et al. | Sep 2014 | B1 |
| 8862514 | Eisen | Oct 2014 | B2 |
| 8862526 | Miltonberger | Oct 2014 | B2 |
| 8938671 | Eisen et al. | Jan 2015 | B2 |
| 8954560 | Johannsen | Feb 2015 | B2 |
| 8966276 | Nanopoulos et al. | Feb 2015 | B2 |
| 9060012 | Eisen | Jun 2015 | B2 |
| 9083735 | Reumann et al. | Jul 2015 | B2 |
| 9098617 | Pauley, Jr. et al. | Aug 2015 | B1 |
| 9112850 | Eisen | Aug 2015 | B1 |
| 9118646 | Pierson et al. | Aug 2015 | B2 |
| 9191370 | Barber et al. | Nov 2015 | B2 |
| 9196004 | Eisen | Nov 2015 | B2 |
| 9203837 | Pierson et al. | Dec 2015 | B2 |
| 9294448 | Miller et al. | Mar 2016 | B2 |
| 9298677 | Tollinger et al. | Mar 2016 | B2 |
| 9332020 | Thomas et al. | May 2016 | B2 |
| 9361597 | Britton et al. | Jun 2016 | B2 |
| 9378500 | Jimenez et al. | Jun 2016 | B2 |
| 9390384 | Eisen | Jul 2016 | B2 |
| 9396331 | Eisen et al. | Jul 2016 | B2 |
| 9412123 | Eisen | Aug 2016 | B2 |
| 9514248 | Guan et al. | Dec 2016 | B1 |
| 9521161 | Reumann et al. | Dec 2016 | B2 |
| 9521551 | Eisen et al. | Dec 2016 | B2 |
| 9559852 | Miller et al. | Jan 2017 | B2 |
| 9633201 | Katz | Apr 2017 | B1 |
| 9703983 | Eisen et al. | Jul 2017 | B2 |
| 9754256 | Britton et al. | Sep 2017 | B2 |
| 9754311 | Eisen | Sep 2017 | B2 |
| 9785973 | Tollinger et al. | Oct 2017 | B2 |
| 9948629 | Eisen | Apr 2018 | B2 |
| 9990631 | Eisen | Jun 2018 | B2 |
| 10021099 | Eisen et al. | Jul 2018 | B2 |
| 20010011243 | Dembo et al. | Aug 2001 | A1 |
| 20010011304 | Wesigner et al. | Aug 2001 | A1 |
| 20010016840 | Hijikata et al. | Aug 2001 | A1 |
| 20010016876 | Kurth et al. | Aug 2001 | A1 |
| 20010034712 | Colvin | Oct 2001 | A1 |
| 20010046096 | Worden | Nov 2001 | A1 |
| 20020035622 | Barber | Mar 2002 | A1 |
| 20020041328 | LeCompte et al. | Apr 2002 | A1 |
| 20020046157 | Solomon | Apr 2002 | A1 |
| 20020052852 | Bozeman | May 2002 | A1 |
| 20020056042 | van der Kaay et al. | May 2002 | A1 |
| 20020073046 | David | Jun 2002 | A1 |
| 20020073327 | Vellandi | Jun 2002 | A1 |
| 20020083079 | Meier et al. | Jun 2002 | A1 |
| 20020112171 | Ginter et al. | Aug 2002 | A1 |
| 20020128917 | Grounds | Sep 2002 | A1 |
| 20020138335 | Palmer et al. | Sep 2002 | A1 |
| 20020138577 | Teng | Sep 2002 | A1 |
| 20020153424 | Li | Oct 2002 | A1 |
| 20020156724 | Levchin et al. | Oct 2002 | A1 |
| 20020156836 | Janosik, Jr. et al. | Oct 2002 | A1 |
| 20020166063 | Lachman et al. | Nov 2002 | A1 |
| 20020167965 | Beasley et al. | Nov 2002 | A1 |
| 20030002732 | Gossett et al. | Jan 2003 | A1 |
| 20030002740 | Melikian et al. | Jan 2003 | A1 |
| 20030014327 | Skantze | Jan 2003 | A1 |
| 20030033161 | Walker et al. | Feb 2003 | A1 |
| 20030033356 | Tran et al. | Feb 2003 | A1 |
| 20030070080 | Rosen | Apr 2003 | A1 |
| 20030074301 | Solomon | Apr 2003 | A1 |
| 20030076242 | Burns et al. | Apr 2003 | A1 |
| 20030105707 | Audebert et al. | Jun 2003 | A1 |
| 20030105854 | Thorsteinsson et al. | Jun 2003 | A1 |
| 20030115334 | Bhat et al. | Jun 2003 | A1 |
| 20030115481 | Baird et al. | Jun 2003 | A1 |
| 20030120543 | Carey | Jun 2003 | A1 |
| 20030120586 | Litty | Jun 2003 | A1 |
| 20030140258 | Nelson et al. | Jul 2003 | A1 |
| 20030154214 | Tu et al. | Aug 2003 | A1 |
| 20030158751 | Suresh et al. | Aug 2003 | A1 |
| 20030163359 | Kanesaka | Aug 2003 | A1 |
| 20030163398 | Yoshioka et al. | Aug 2003 | A1 |
| 20030163413 | Wiczkowski | Aug 2003 | A1 |
| 20030172036 | Feigenbaum | Sep 2003 | A1 |
| 20030182551 | Frantz et al. | Sep 2003 | A1 |
| 20030208684 | Camacho et al. | Nov 2003 | A1 |
| 20030212618 | Keyes et al. | Nov 2003 | A1 |
| 20030233553 | Parks et al. | Dec 2003 | A1 |
| 20040001044 | Luciani et al. | Jan 2004 | A1 |
| 20040004733 | Barker et al. | Jan 2004 | A1 |
| 20040006553 | de Vries et al. | Jan 2004 | A1 |
| 20040010682 | Foster et al. | Jan 2004 | A1 |
| 20040027385 | Rekimoto et al. | Feb 2004 | A1 |
| 20040030912 | Merkle, Jr. et al. | Feb 2004 | A1 |
| 20040034794 | Mayer et al. | Feb 2004 | A1 |
| 20040073809 | Wing Keong | Apr 2004 | A1 |
| 20040088313 | Torres | May 2004 | A1 |
| 20040105431 | Monjas-Llorente et al. | Jun 2004 | A1 |
| 20040111621 | Himberger et al. | Jun 2004 | A1 |
| 20040117321 | Sancho | Jun 2004 | A1 |
| 20040139008 | Mascavaage, III | Jul 2004 | A1 |
| 20040153644 | McCorkendale et al. | Aug 2004 | A1 |
| 20040159699 | Nelson et al. | Aug 2004 | A1 |
| 20040166857 | Shim et al. | Aug 2004 | A1 |
| 20040171381 | Inselberg | Sep 2004 | A1 |
| 20040181598 | Paya et al. | Sep 2004 | A1 |
| 20040203750 | Cowdrey et al. | Oct 2004 | A1 |
| 20040230820 | Hui Hsu et al. | Nov 2004 | A1 |
| 20040236696 | Aoki et al. | Nov 2004 | A1 |
| 20040236702 | Fink et al. | Nov 2004 | A1 |
| 20040254890 | Sancho et al. | Dec 2004 | A1 |
| 20040260876 | Singh et al. | Dec 2004 | A1 |
| 20040260922 | Goodman et al. | Dec 2004 | A1 |
| 20050008148 | Jacobson | Jan 2005 | A1 |
| 20050015601 | Tabi | Jan 2005 | A1 |
| 20050022020 | Fremberg et al. | Jan 2005 | A1 |
| 20050033653 | Eisenberg et al. | Feb 2005 | A1 |
| 20050033703 | Holdsworth | Feb 2005 | A1 |
| 20050039034 | Doyle et al. | Feb 2005 | A1 |
| 20050039219 | Cooper et al. | Feb 2005 | A1 |
| 20050076230 | Redenbaugh et al. | Apr 2005 | A1 |
| 20050085931 | Willeby | Apr 2005 | A1 |
| 20050097320 | Golan et al. | May 2005 | A1 |
| 20050108177 | Sancho | May 2005 | A1 |
| 20050111054 | Umeda | May 2005 | A1 |
| 20050113092 | Coppinger et al. | May 2005 | A1 |
| 20050131826 | Cook | Jun 2005 | A1 |
| 20050185225 | Brawn et al. | Aug 2005 | A1 |
| 20050188423 | Motsinger et al. | Aug 2005 | A1 |
| 20050204159 | Davis et al. | Sep 2005 | A1 |
| 20050216278 | Eisen | Sep 2005 | A1 |
| 20050246551 | Dondl et al. | Nov 2005 | A1 |
| 20050278542 | Pierson et al. | Dec 2005 | A1 |
| 20060008779 | Shand et al. | Jan 2006 | A1 |
| 20060010072 | Eisen | Jan 2006 | A1 |
| 20060026669 | Zakas | Feb 2006 | A1 |
| 20060048211 | Pierson et al. | Mar 2006 | A1 |
| 20060064346 | Steenstra et al. | Mar 2006 | A1 |
| 20060069619 | Walker et al. | Mar 2006 | A1 |
| 20060080263 | Willis et al. | Apr 2006 | A1 |
| 20060126829 | Lai | Jun 2006 | A1 |
| 20060130132 | Dharmarajan | Jun 2006 | A1 |
| 20060136294 | Linden et al. | Jun 2006 | A1 |
| 20060155985 | Canard et al. | Jul 2006 | A1 |
| 20060161501 | Waserstein et al. | Jul 2006 | A1 |
| 20060190331 | Tollinger et al. | Aug 2006 | A1 |
| 20060200855 | Willis | Sep 2006 | A1 |
| 20060200856 | Salowey et al. | Sep 2006 | A1 |
| 20060224898 | Ahmed | Oct 2006 | A1 |
| 20060237531 | Heffez et al. | Oct 2006 | A1 |
| 20060253327 | Morris et al. | Nov 2006 | A1 |
| 20060253328 | Kohli et al. | Nov 2006 | A1 |
| 20060264202 | Hagmeier et al. | Nov 2006 | A1 |
| 20060281541 | Nguyen et al. | Dec 2006 | A1 |
| 20060282660 | Varghese et al. | Dec 2006 | A1 |
| 20060284838 | Tsatalos et al. | Dec 2006 | A1 |
| 20060287902 | Helsper et al. | Dec 2006 | A1 |
| 20070011078 | Jain et al. | Jan 2007 | A1 |
| 20070030528 | Quaeler et al. | Feb 2007 | A1 |
| 20070038568 | Greene et al. | Feb 2007 | A1 |
| 20070043837 | Kruse et al. | Feb 2007 | A1 |
| 20070061211 | Ramer et al. | Mar 2007 | A1 |
| 20070061273 | Greene et al. | Mar 2007 | A1 |
| 20070073630 | Greene et al. | Mar 2007 | A1 |
| 20070094594 | Matichuk et al. | Apr 2007 | A1 |
| 20070097076 | Gross | May 2007 | A1 |
| 20070097976 | Wood et al. | May 2007 | A1 |
| 20070101405 | Engle et al. | May 2007 | A1 |
| 20070107059 | Chasin et al. | May 2007 | A1 |
| 20070118892 | Sastry | May 2007 | A1 |
| 20070124246 | Lawyer et al. | May 2007 | A1 |
| 20070162763 | Bender et al. | Jul 2007 | A1 |
| 20070198410 | Labgold et al. | Aug 2007 | A1 |
| 20070199054 | Florencio et al. | Aug 2007 | A1 |
| 20070204044 | Rice et al. | Aug 2007 | A1 |
| 20070214151 | Scott et al. | Sep 2007 | A1 |
| 20070220594 | Tulsyan | Sep 2007 | A1 |
| 20070233599 | Ganesan et al. | Oct 2007 | A1 |
| 20070234070 | Horning et al. | Oct 2007 | A1 |
| 20070234409 | Eisen | Oct 2007 | A1 |
| 20070239604 | O'Connell et al. | Oct 2007 | A1 |
| 20070239606 | Eisen | Oct 2007 | A1 |
| 20070255821 | Ge et al. | Nov 2007 | A1 |
| 20070266257 | Camaisa et al. | Nov 2007 | A1 |
| 20070271466 | Mak | Nov 2007 | A1 |
| 20070294401 | Shkedi | Dec 2007 | A1 |
| 20080005394 | Crooks | Jan 2008 | A1 |
| 20080010367 | Chen et al. | Jan 2008 | A1 |
| 20080010678 | Burdette et al. | Jan 2008 | A1 |
| 20080015988 | Brown et al. | Jan 2008 | A1 |
| 20080021801 | Song et al. | Jan 2008 | A1 |
| 20080040653 | Levine | Feb 2008 | A1 |
| 20080040802 | Pierson et al. | Feb 2008 | A1 |
| 20080046562 | Butler | Feb 2008 | A1 |
| 20080052629 | Phillips et al. | Feb 2008 | A1 |
| 20080098222 | Zilberman | Apr 2008 | A1 |
| 20080101277 | Taylor | May 2008 | A1 |
| 20080104070 | Lonchar | May 2008 | A1 |
| 20080104672 | Lunde | May 2008 | A1 |
| 20080104684 | Lunde | May 2008 | A1 |
| 20080120195 | Shakkarwar | May 2008 | A1 |
| 20080120214 | Steele et al. | May 2008 | A1 |
| 20080133420 | Barber | Jun 2008 | A1 |
| 20080162200 | O'Sullivan et al. | Jul 2008 | A1 |
| 20080162202 | Khanna et al. | Jul 2008 | A1 |
| 20080162475 | Meggs | Jul 2008 | A1 |
| 20080163128 | Callanan et al. | Jul 2008 | A1 |
| 20080184372 | Hoshina | Jul 2008 | A1 |
| 20080189790 | Park | Aug 2008 | A1 |
| 20080201214 | Aaron | Aug 2008 | A1 |
| 20080204788 | Kelly et al. | Aug 2008 | A1 |
| 20080222706 | Renaud et al. | Sep 2008 | A1 |
| 20080235623 | Li | Sep 2008 | A1 |
| 20080249820 | Pathria et al. | Oct 2008 | A1 |
| 20080281606 | Kitts | Nov 2008 | A1 |
| 20080281941 | Park et al. | Nov 2008 | A1 |
| 20080288299 | Schultz | Nov 2008 | A1 |
| 20080301281 | Wang et al. | Dec 2008 | A1 |
| 20080313079 | Van Bosch et al. | Dec 2008 | A1 |
| 20080319774 | O'Sullivan et al. | Dec 2008 | A1 |
| 20080319841 | Oliver et al. | Dec 2008 | A1 |
| 20090018940 | Wang et al. | Jan 2009 | A1 |
| 20090024971 | Willner et al. | Jan 2009 | A1 |
| 20090044279 | Crawford et al. | Feb 2009 | A1 |
| 20090044282 | Govindaraju | Feb 2009 | A1 |
| 20090055398 | Zhu et al. | Feb 2009 | A1 |
| 20090083184 | Eisen | Mar 2009 | A1 |
| 20090089869 | Varghese | Apr 2009 | A1 |
| 20090106413 | Salo | Apr 2009 | A1 |
| 20090157417 | Bradley et al. | Jun 2009 | A1 |
| 20090164269 | Gupta et al. | Jun 2009 | A1 |
| 20090177692 | Chagoly et al. | Jul 2009 | A1 |
| 20090183010 | Schnell et al. | Jul 2009 | A1 |
| 20090205031 | Sato et al. | Aug 2009 | A1 |
| 20090222308 | Zoldi et al. | Sep 2009 | A1 |
| 20090228585 | Kosbab et al. | Sep 2009 | A1 |
| 20090234738 | Britton et al. | Sep 2009 | A1 |
| 20090241174 | Rajan et al. | Sep 2009 | A1 |
| 20090260064 | Mcdowell et al. | Oct 2009 | A1 |
| 20090265773 | Schultz | Oct 2009 | A1 |
| 20090271306 | Pierson | Oct 2009 | A1 |
| 20090307141 | Kongalath et al. | Oct 2009 | A1 |
| 20090280777 | Doherty | Nov 2009 | A1 |
| 20090292568 | Khosravani et al. | Nov 2009 | A1 |
| 20090296907 | Vendrow et al. | Dec 2009 | A1 |
| 20090298480 | Khambete et al. | Dec 2009 | A1 |
| 20090307119 | Ahles et al. | Dec 2009 | A1 |
| 20090313134 | Faith et al. | Dec 2009 | A1 |
| 20100004965 | Eisen | Jan 2010 | A1 |
| 20100005013 | Uriarte | Jan 2010 | A1 |
| 20100030641 | Ibenforth | Feb 2010 | A1 |
| 20100030777 | Panwar | Feb 2010 | A1 |
| 20100057623 | Kapur et al. | Mar 2010 | A1 |
| 20100070606 | Shenfield et al. | Mar 2010 | A1 |
| 20100082972 | Benco et al. | Apr 2010 | A1 |
| 20100094767 | Miltonberger | Apr 2010 | A1 |
| 20100094768 | Miltonberger | Apr 2010 | A1 |
| 20100106611 | Paulsen et al. | Apr 2010 | A1 |
| 20100107225 | Spencer et al. | Apr 2010 | A1 |
| 20100121716 | Golan | May 2010 | A1 |
| 20100138299 | Preston et al. | Jun 2010 | A1 |
| 20100145960 | Casteel et al. | Jun 2010 | A1 |
| 20100153540 | Li et al. | Jun 2010 | A1 |
| 20100157848 | Das et al. | Jun 2010 | A1 |
| 20100161424 | Sylvain | Jun 2010 | A1 |
| 20100161566 | Adair et al. | Jun 2010 | A1 |
| 20100169157 | Muhonen et al. | Jul 2010 | A1 |
| 20100169192 | Zoldi et al. | Jul 2010 | A1 |
| 20100192082 | Sodah | Jul 2010 | A1 |
| 20100199332 | Bachmann et al. | Aug 2010 | A1 |
| 20100199338 | Craddock et al. | Aug 2010 | A1 |
| 20100211464 | Zhu et al. | Aug 2010 | A1 |
| 20100223105 | Gassewitz et al. | Sep 2010 | A1 |
| 20100223145 | Dragt | Sep 2010 | A1 |
| 20100228625 | Priyadarshan et al. | Sep 2010 | A1 |
| 20100228638 | Mikan et al. | Sep 2010 | A1 |
| 20100257065 | Gupta et al. | Oct 2010 | A1 |
| 20100274678 | Rolf et al. | Oct 2010 | A1 |
| 20100293094 | Kolkowitz et al. | Nov 2010 | A1 |
| 20100306827 | Esteve Balducci et al. | Dec 2010 | A1 |
| 20100321296 | Gross | Dec 2010 | A1 |
| 20100333170 | Cox et al. | Dec 2010 | A1 |
| 20110022483 | Hammad | Jan 2011 | A1 |
| 20110022517 | Hammad | Jan 2011 | A1 |
| 20110035302 | Martell et al. | Feb 2011 | A1 |
| 20110047072 | Ciurea | Feb 2011 | A1 |
| 20110082768 | Eisen | Apr 2011 | A1 |
| 20110112901 | Fried et al. | May 2011 | A1 |
| 20110161228 | Suzuki et al. | Jun 2011 | A1 |
| 20110173281 | Smith | Jul 2011 | A1 |
| 20110184778 | Graepel et al. | Jul 2011 | A1 |
| 20110194679 | Patisaul et al. | Aug 2011 | A1 |
| 20110225091 | Plastina et al. | Sep 2011 | A1 |
| 20110238575 | Nightengale et al. | Sep 2011 | A1 |
| 20110251951 | Kolkowitz et al. | Oct 2011 | A1 |
| 20110258118 | Ciurea | Oct 2011 | A1 |
| 20110282778 | Wright et al. | Nov 2011 | A1 |
| 20110288932 | Marks et al. | Nov 2011 | A1 |
| 20110302087 | Crooks | Dec 2011 | A1 |
| 20110302096 | Lowry | Dec 2011 | A1 |
| 20110307341 | Zohar et al. | Dec 2011 | A1 |
| 20110314557 | Marshall | Dec 2011 | A1 |
| 20120022883 | Morrison | Jan 2012 | A1 |
| 20120030083 | Newman et al. | Feb 2012 | A1 |
| 20120030757 | Baikalov et al. | Feb 2012 | A1 |
| 20120030771 | Pierson et al. | Feb 2012 | A1 |
| 20120036042 | Graylin et al. | Feb 2012 | A1 |
| 20120041841 | Hu et al. | Feb 2012 | A1 |
| 20120054136 | Maulik | Mar 2012 | A1 |
| 20120054847 | Schultz et al. | Mar 2012 | A1 |
| 20120084203 | Mehew et al. | Apr 2012 | A1 |
| 20120094639 | Carlson et al. | Apr 2012 | A1 |
| 20120101939 | Kasower | Apr 2012 | A1 |
| 20120150742 | Poon et al. | Jun 2012 | A1 |
| 20120150750 | Law et al. | Jun 2012 | A1 |
| 20120157062 | Kim et al. | Jun 2012 | A1 |
| 20120158586 | Ganti et al. | Jun 2012 | A1 |
| 20120166533 | Rubinstein et al. | Jun 2012 | A1 |
| 20120173465 | Hore et al. | Jul 2012 | A1 |
| 20120179558 | Fischer | Jul 2012 | A1 |
| 20120197981 | Chan | Aug 2012 | A1 |
| 20120204262 | Thomas et al. | Aug 2012 | A1 |
| 20120215896 | Johannsen | Aug 2012 | A1 |
| 20120221470 | Lyon | Aug 2012 | A1 |
| 20120222111 | Oliver et al. | Aug 2012 | A1 |
| 20120233665 | Ranganathan et al. | Sep 2012 | A1 |
| 20120239553 | Gonen et al. | Sep 2012 | A1 |
| 20120239574 | Smith et al. | Sep 2012 | A1 |
| 20120239774 | Tola et al. | Sep 2012 | A1 |
| 20120278127 | Kirakosyan et al. | Nov 2012 | A1 |
| 20120295580 | Corner | Nov 2012 | A1 |
| 20120297380 | Colbert | Nov 2012 | A1 |
| 20120311162 | Paulsen et al. | Dec 2012 | A1 |
| 20120323788 | Keresman et al. | Dec 2012 | A1 |
| 20120323836 | Wright et al. | Dec 2012 | A1 |
| 20120330787 | Hanson et al. | Dec 2012 | A1 |
| 20130006743 | Moore et al. | Jan 2013 | A1 |
| 20130018789 | Kaufmann | Jan 2013 | A1 |
| 20130018791 | Mendicino et al. | Jan 2013 | A1 |
| 20130024300 | Choudhuri et al. | Jan 2013 | A1 |
| 20130036304 | Lin et al. | Feb 2013 | A1 |
| 20130040603 | Stahlberg et al. | Feb 2013 | A1 |
| 20130042298 | Plaza Fonseca et al. | Feb 2013 | A1 |
| 20130055388 | Thomas et al. | Feb 2013 | A1 |
| 20130073463 | Dimmick et al. | Mar 2013 | A1 |
| 20130073473 | Heath | Mar 2013 | A1 |
| 20130085841 | Singleton et al. | Apr 2013 | A1 |
| 20130097673 | Meehan et al. | Apr 2013 | A1 |
| 20130097701 | Moyle et al. | Apr 2013 | A1 |
| 20130103482 | Song et al. | Apr 2013 | A1 |
| 20130103629 | Vaiciulis et al. | Apr 2013 | A1 |
| 20130110637 | Bott | May 2013 | A1 |
| 20130111592 | Zhu et al. | May 2013 | A1 |
| 20130117832 | Gandhi | May 2013 | A1 |
| 20130144539 | Allen et al. | Jun 2013 | A1 |
| 20130148525 | Cuadra Sanchez et al. | Jun 2013 | A1 |
| 20130159195 | Kirillin et al. | Jun 2013 | A1 |
| 20130185764 | Krstić et al. | Jul 2013 | A1 |
| 20130197998 | Buhrmann et al. | Aug 2013 | A1 |
| 20130198066 | Wall et al. | Aug 2013 | A1 |
| 20130226717 | Ahluwalia et al. | Aug 2013 | A1 |
| 20130273879 | Eisen et al. | Oct 2013 | A1 |
| 20130339186 | French | Dec 2013 | A1 |
| 20140032902 | Agrawal et al. | Jan 2014 | A1 |
| 20140114821 | Yoshioka et al. | Apr 2014 | A1 |
| 20140120864 | Manolarakis et al. | May 2014 | A1 |
| 20140122343 | Einav et al. | May 2014 | A1 |
| 20140258125 | Gerber et al. | Sep 2014 | A1 |
| 20140289867 | Bukai | Sep 2014 | A1 |
| 20140361926 | Eisen et al. | Dec 2014 | A1 |
| 20150026027 | Priess et al. | Jan 2015 | A1 |
| 20150046989 | Oberheide | Feb 2015 | A1 |
| 20150106270 | Burrell et al. | Apr 2015 | A1 |
| 20150186901 | Miltonberger | Jul 2015 | A1 |
| 20150193769 | Barber | Jul 2015 | A1 |
| 20150193821 | Izumori et al. | Jul 2015 | A1 |
| 20150205978 | Eisen et al. | Jul 2015 | A1 |
| 20150242861 | Baldassano | Aug 2015 | A9 |
| 20150294316 | Eisen | Oct 2015 | A1 |
| 20150350856 | Circosta | Dec 2015 | A1 |
| 20160019546 | Eisen | Jan 2016 | A1 |
| 20160021084 | Eisen | Jan 2016 | A1 |
| 20160034954 | Tollinger et al. | Feb 2016 | A1 |
| 20160125461 | Sivaramakrishnan et al. | May 2016 | A1 |
| 20160203487 | Eisen | Jul 2016 | A1 |
| 20160246581 | Jimenez et al. | Aug 2016 | A1 |
| 20160321701 | Tollinger et al. | Nov 2016 | A1 |
| 20170039571 | Eisen | Feb 2017 | A1 |
| 20170142106 | Eisen et al. | May 2017 | A1 |
| 20180089459 | Eisen et al. | Mar 2018 | A1 |
| 20180101890 | Eisen | Apr 2018 | A1 |
| 20180121915 | Britton et al. | May 2018 | A1 |
| Number | Date | Country |
|---|---|---|
| 0 418 144 | Mar 1991 | EP |
| 0 923 039 | Jun 1999 | EP |
| 1 067 792 | Jan 2001 | EP |
| 1 209 935 | May 2002 | EP |
| 1 256 911 | Nov 2002 | EP |
| 1 201 070 | Jun 2006 | EP |
| 1 703 382 | Sep 2006 | EP |
| 1 197 032 | Aug 2007 | EP |
| 2 154 891 | Feb 2010 | EP |
| 2 491 101 | Nov 2012 | GB |
| 2 492 604 | Jan 2013 | GB |
| 05-257602 | Oct 1993 | JP |
| 2000-020467 | Jan 2000 | JP |
| 2000-137755 | May 2000 | JP |
| 2000-242582 | Sep 2000 | JP |
| 2000-276281 | Oct 2000 | JP |
| 2002-007697 | Jan 2002 | JP |
| 2002-297869 | Oct 2002 | JP |
| 2003-050910 | Feb 2003 | JP |
| 2005-063216 | Mar 2005 | JP |
| 2005-115644 | Apr 2005 | JP |
| 2005-135431 | May 2005 | JP |
| 2006-004333 | Jan 2006 | JP |
| 2007-272520 | Oct 2007 | JP |
| 2007-282249 | Oct 2007 | JP |
| 2008-022298 | Jan 2008 | JP |
| 2008-065363 | Mar 2008 | JP |
| 2008-171315 | Jul 2008 | JP |
| 2008-535124 | Aug 2008 | JP |
| 2008-243008 | Oct 2008 | JP |
| 2008-257434 | Oct 2008 | JP |
| 2008-269229 | Nov 2008 | JP |
| 2009-048538 | Mar 2009 | JP |
| 2009-122880 | Jun 2009 | JP |
| 2009-175984 | Aug 2009 | JP |
| 2010-020728 | Jan 2010 | JP |
| 2010-061254 | Mar 2010 | JP |
| 2010-122955 | Jun 2010 | JP |
| 2010-122956 | Jun 2010 | JP |
| 2010-225040 | Oct 2010 | JP |
| 2010-250664 | Nov 2010 | JP |
| 2011-065531 | Mar 2011 | JP |
| 2011-134252 | Jul 2011 | JP |
| 2011-159307 | Aug 2011 | JP |
| 2012-234503 | Nov 2012 | JP |
| 5216932 | Jun 2013 | JP |
| 10-1999-0015738 | Mar 1999 | KR |
| 10-0645983 | Nov 2006 | KR |
| 10-2008-0044558 | May 2008 | KR |
| 10-2009-0051977 | Sep 2009 | KR |
| 10-2010-0085888 | Jul 2010 | KR |
| WO 96041488 | Dec 1996 | WO |
| WO 97003410 | Jan 1997 | WO |
| WO 99050775 | Oct 1999 | WO |
| WO 01011450 | Feb 2001 | WO |
| WO 01033520 | May 2001 | WO |
| WO 01095550 | Dec 2001 | WO |
| WO 01097134 | Dec 2001 | WO |
| WO 02001462 | Jan 2002 | WO |
| WO 02071176 | Sep 2002 | WO |
| WO 02091226 | Nov 2002 | WO |
| WO 03017155 | Feb 2003 | WO |
| WO 03025868 | Mar 2003 | WO |
| WO 03075197 | Sep 2003 | WO |
| WO 03075197 | Dec 2003 | WO |
| WO 02037219 | May 2004 | WO |
| WO 2004038997 | May 2004 | WO |
| WO 2005038818 | Apr 2005 | WO |
| WO 2005099166 | Oct 2005 | WO |
| WO 2006135367 | Dec 2006 | WO |
| WO 2007001394 | Jan 2007 | WO |
| WO 2007045818 | Apr 2007 | WO |
| WO 2007072238 | Jun 2007 | WO |
| WO 2007075573 | Jul 2007 | WO |
| WO 2008029828 | Mar 2008 | WO |
| WO 2008054849 | May 2008 | WO |
| WO 2009132148 | Oct 2009 | WO |
| WO 2012054646 | Apr 2012 | WO |
| WO 2012061801 | May 2012 | WO |
| WO 2012142121 | Oct 2012 | WO |
| WO 2012142584 | Oct 2012 | WO |
| WO 2013006538 | Jan 2013 | WO |
| WO 2013142722 | Sep 2013 | WO |
| WO 2014022813 | Feb 2014 | WO |
| WO 2014078569 | May 2014 | WO |
| Entry |
|---|
| Banking Services Newsletter, “Keeping You Up-to-Date on Banking Developments Throughout the UC System”, University of California, Office of the President, Banking Services Group, Newsletter 1, Dec. 2005, p. 1. |
| Bharosa, “Bharosa Authenticator”, http://www.bharosa.com/authenticator.html, Jan. 18, 2007, pp. 3. |
| Bharosa, “Bharosa Announces Online Authentication Solution to Counter Check 21-Based Fraud”, http://www.bharosa.com/news/PR-110705.html, Jan. 18, 2007, pp. 2. |
| Darlin, Damon, “Opening the Door on the Credit Report and Throwing Away the Lock”, http://www.nytimes.com/2006/03/18/business/yourmoney/18money.html, The New York Times, Saturday Mar. 18, 2006, pp. 2. |
| Derfler, Jr. et al, “How Networks Work”, Millennium Edition, Que Corporation, Indianapolis, IN, Sep. 2000, [Uploaded in 2 parts]. |
| Gralla, Preston, “How the Internet Works”, Millennium Edition, Que Corporation, Indianapolis, IN, Aug. 1999, [Uploaded in 2 parts]. |
| Gueye et al., “Constraint-Based Geolocation of Internet Hosts”, ACM Internet Measurement Conference 2004, Oct. 25-27, 2004, Taormina, Sicily, Italy, vol. 14, No. 6, pp. 288-293. |
| “ISO 8583”, Wikipedia, http://en.wikipedia.org/wiki/ISO_8583, dated Apr. 13, 2015 in 14 pages. |
| Kohno et al., “Remote Physical Devie Fingerprinting”, Proceedings of 2005 IEEE Symposium on Security and Privacy, May 8-11, 2005, Oakland, CA, pp. 211-225. |
| Manavoglu et al., “Probabilistic User Behavior Models”, ICDM, Third IEEE International Conference on Data Mining, Nov. 19-22, 2003, pp. 203-210. |
| TechWeb, “Wells Fargo Intros Anti-Theft Alerts”, http://www.techweb.com/wire/166404177, Aug. 1, 2005, pp. 1. |
| “UPIC Marketing Guide—The Clearing House”, http://www.upic.com/infofiles/UPIC_Marketing_Guide.pdf, as printed Dec. 19, 2006. pp. 1-16. |
| White, Ron, “How Computers Work”, Millennium Edition, Que Corporation, Indianapolis, IN, Sep. 1999. [Uploaded in 2 parts]. |
| Official Communication in European Patent Application No. 05818903.6, dated Dec. 23, 2011. |
| Official Communication in European Patent Application No. 05818903.6, dated Mar. 18, 2014. |
| International Search Report and Written Opinion for Application No. PCT/US2005/035532, dated Oct. 29, 2007. |
| International Preliminary Report on Patentability and Written Opinion for Application No. PCT/US2005/035532, dated Jan. 9, 2008. |
| Official Communication in European Patent Application No. 6845722.5, dated Mar. 13, 2009. |
| Official Communication in European Patent Application No. 8159110.9, dated Oct. 27, 2008. |
| Official Communication in European Patent Application No. 8159110.9, dated Mar. 22, 2010. |
| International Search Report and Written Opinion for Application No. PCT/US2006/048251, dated Feb. 26, 2008. |
| International Preliminary Report on Patentability and Written Opinion for Application No. PCT/US2006/048251, dated Jun. 18, 2008. |
| International Search Report and Written Opinion for Application No. PCT/US2007/065776, dated Jul. 3, 2008. |
| International Preliminary Report on Patentability and Written Opinion for Application No. PCT/US2007/065776, dated Sep. 30, 2008. |
| International Search Report and Written Opinion received in PCT Application No. PCT/US2005/020750, dated Jun. 13, 2008. |
| International Preliminary Report on Patentability and Written Opinion received in PCT Application No. PCT/US2005/020750, dated Jul. 1, 2008. |
| Official Communication in European Patent Application No. 08165224.0, dated Nov. 15, 2010. |
| Supplementary European Search Report for Application No. EP09735653, dated Dec. 16, 2011. |
| Official Communication for Application No. EP09735653, dated Jan. 4, 2013. |
| International Search Report and Written Opinion for Application No. PCT/US2009/041462, dated Dec. 1, 2009. |
| International Preliminary Report on Patentability and Written Opinion for Application No. PCT/US2009/041462, dated Nov. 4, 2010. |
| International Search Report and Written Opinion for Application No. PCT/US2011/056948, dated Apr. 18, 2012. |
| International Preliminary Report on Patentability in Application No. PCT/US2011/056948, dated May 2, 2013. |
| International Search Report and Written Opinion for Application No. PCT/US2013/033357, dated Jul. 10, 2013. |
| International Preliminary Report on Patentability in Application No. PCT/US2013/033357, dated Sep. 23, 2014. |
| International Search Report and Written Opinion for Application No. PCT/US2013/053495, dated Nov. 22, 2013. |
| International Preliminary Report on Patentability in Application No. PCT/US2013/053495, dated Feb. 3, 2015. |
| International Search Report and Written Opinion for Application No. PCT/US2013/070146, dated Mar. 3, 2014. |
| International Preliminary Report on Patentability in Application No. PCT/US2013/070146, dated May 19, 2015. |
| Provisional Application as filed in U.S. Appl. No. 61/324,312, dated Apr. 15, 2010 in 15 pages. |
| Official Communication in European Patent Application No. 05818903.6, dated Jul. 18, 2017. |
| Number | Date | Country | |
|---|---|---|---|
| 62063567 | Oct 2014 | US |
