System and method for dynamic document matching and merging

Abstract

A system and method for matching and merging documents from disparate data sources into a single data store for a particular entity are provided. The system and method may be particularly useful for a healthcare system to match and merge data from disparate data sources about a healthcare provider.

Description

FIELD

The disclosure relates generally to a system and method for matching and merging disparate documents.

BACKGROUND

Data about an entity, such as a subject, company, idea or the like, may be stored in a plurality of disparate data sources. In order to be able to assemble the data about the entity from the disparate sources into a single data store, it is necessary to try to gather the various data from the various data sources and then determine a way to combine the data from the disparate data sources for the particular entity into the single data store.

In the healthcare industry, information/data about each healthcare provider, such as a doctor, a therapist, a nurse, a hospital, a medical practice and the like, may be stored in a plurality of disparate data sources. The information/data about the healthcare provider may include, for example, reviews, directions, rates and the like. The disparate data sources for the data/information for the healthcare provider may include publicly available Centers for Medicare and Medicaid Services' (CMS) National Plan and Provider Enumeration System (NPPES) data to privately curated and licensed data from the American Medical Association (AMA), among others.

The issues that must be confronted in order to successfully integrate the data from these various data sources into a single data store may include:

• • While the provider documents are structured, the available data fields are heterogeneous across data sources.
  • There is no strong identifier linking provider documents across data sources. Even a provider's name may be suspect for a number of reasons:
    • Names may be legally changed
    • Informal variations (i.e., nicknames)
    • Misspellings due to human error
    • Inconsistent localization from non-Roman alphabets
    • Multiple providers with the same name
  • No single data source can be trusted as authoritative, as there is no central mechanism in place to update each concerned organization as provider information changes over time.
  • As there is no central mechanism for updating provider information, the data available from NPPES, AMA and others invariably become out of sync even among the commonly available data fields.
  • While the data from NPPES, AMA and others provide a top-level view of their own provider directories, they too have combined data from potentially thousands of lower level sources, and errors may have propagated through their own system.
  • There are more than one million individual healthcare providers in the United States, and manual curation and inspection of all providers' data is not feasible.

Thus, it is desirable to provide a system and method for dynamic data identification and combining so that, for example, data from disparate data sources for a healthcare provider may be combined into a single data store.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a top level process flow for a method for matching and merging documents;

FIG. 2 is an example of an implementation of a system for matching and merging documents;

FIG. 3 illustrates more details of the system shown in FIG. 2;

FIG. 4 illustrates an example of a source provider document that may be matched and merged using the process shown in FIG. 1;

FIG. 5 illustrates an example of an aligned and structured provider document that may be matched and merged using the process shown in FIG. 1;

FIG. 6 illustrates an example of a match set;

FIG. 7 is an example of an excerpt from a merged document with provenance generated by the method in FIG. 1; and

FIG. 8 is an example of an excerpt from a merged document with ranked values and provenance generated by the method in FIG. 1.

DETAILED DESCRIPTION OF ONE OR MORE EMBODIMENTS

The disclosure is particularly applicable to a healthcare system in which healthcare provider data is matched and merged and it is in this context that the disclosure will be described. It will be appreciated, however, that the system and method has greater utility since the system and method may be used with any type of entity for which it is desirable to be able to match and merge data about the entity from disparate data sources. Furthermore, the system and method may be used in any industry for which it is desirable to be able to match and merge data about the entity from disparate data sources. For purposes of this disclosure, an entity may be a subject, an idea, a professional, a person, a corporation, a business entity and the like.

In an example healthcare embodiment, a healthcare system may have a goal of providing healthcare pricing transparency and connecting consumers directly to healthcare providers. To provide that healthcare pricing transparency, the healthcare system needs to maintain a comprehensive and up-to-date directory of healthcare providers. In order to build this provider directory, data are combined from disparate sources ranging from the publicly available Centers for Medicare and Medicaid Services' (CMS) National Plan and Provider Enumeration System (NPPES) data to privately curated and licensed data from the American Medical Association (AMA), among others. These data take the form of structured records on a per-provider basis, referred to herein as provider documents.

The system and method provide a computational process to match provider documents from disparate sources which refer to the same provider and merge those documents into a single comprehensive view while taking into account the relative trustworthiness of the data sources for each available data field. The generated single comprehensive view facilitates a more accurate services purchasing and recommendation experience for the healthcare consumer as well as the practitioner in the application domain. The ability to dynamically match disparate data sources with data hygiene metric is crucial in evaluating the behaviors and ratings for ranking practitioners that will be listed in a marketplace of the healthcare system. This improved matching model further facilitates a faceted search paradigm much like one would search for a camera purchase at an internet marketplace site.

The system may include one or more of the following components:

• • a plurality of provider document matcher algorithms.
  • a method for providing human verification feedback on a random sampling of match algorithm results.
  • a statistical model built from the human verification feedback loop, used to select and combine result match sets from the match algorithms.
  • a method to merge the original provider documents contained within each selected match set, maintaining provenance for each field/value pair in the merged document.
  • a method to resolve conflicting values for available fields in the merged provider documents.
  • an Extract Transform and Load (ETL) method for distributing these processes via a transform pipeline that allows for optimized and distributed processing of the disparate data sets
  • a distributed process that utilizes a message queuing and enterprise service bus to dynamically process the ETL information.

FIG. 1 is a top level process flow for a method 100 for matching and merging documents. The system that implements the process flow is designed to allow for multiple matcher algorithms in order to take advantage of the relative strengths of each algorithm while compensating for their individual weaknesses. For instance, a strict matcher may utilize statistically significant combinations of biographic identifiers such as each component of a person's full name as well as birth date and birth place, which would produce few false positives in the search space yet would produce a false negative if an actual matching document in the search space lacked any component of the full name (e.g., middle name) or the birth information. Conversely, a loose matcher that allows for variations in a person's name (e.g., nicknames) would be more successful in reducing false negatives but would produce far more false positives.

For example, the system may use Bayesian Identity Resolution in which comparators and weight ranges are specified for a subset of the fields in the documents which are determined to be the best features for determining matches. When document pairs are evaluated, each field in the documents are compared using the specified comparator and the result is scaled to the specified weight range resulting in a weighted match score for the field. These weighted field match scores are combined using Bayes' theorem to provide an overall match score for the two documents. If this document match score is above a designated threshold than the two documents are considered to be a match, otherwise they are considered not to match.

As another example, the system may use ElasticSearch. ElasticSearch is a distributed, RESTful, free/open source search server based on Apache Lucene, an open source information retrieval software library. To perform document matching using Elasticsearch, a collection of documents is first “indexed” using the Elasticsearch API. Then a collection of documents is iterated upon, constructing a precise boolean query based on select fields from the iterated document. If the necessary fields are present in this “query” document, the query is issued against the Elasticsearch index, and results indicate a positive match which is saved into results collection. For collection deduplication the iterated collection may be the same collection that was indexed. Alternatively for record “linkage” an entirely different collection may be iterated upon.

Prior to running the ensemble of matcher algorithms, each of the source documents (raw files in FIG. 1) are imported 102 and assigned a unique identifier. Those identifiers remain with the documents as they are transformed 104, for example, to align vocabulary (i.e., field names) where appropriate, and to de-normalize the logical data structures within the documents (e.g., grouping components of a mailing address within an ‘address’ sub-document). FIGS. 4 and 5 show an example of a source provider document and its aligned/structured version, respectively.

Following the initial data cleansing, each matcher algorithm may be run (matcher processes 106) against the entire set of N provider documents from all sources (our search space). This may be viewed as a sequence of queries using M canonical data source documents as the query documents for which we wish to find corresponding matches in the search space, resulting in M match sets (see FIG. 6 for an example of a match set). Ultimately the matched documents will be folded into the canonical documents to provide an updated, unified view of the data. An alternative embodiment may run the matcher algorithms using the N search space documents as the query documents to perform de-duplication, but for the purposes of folding multiple sources of provider documents into the comprehensive directory we use the same set of query documents across all matcher instantiations.

The generated match sets do not contain the actual matching documents, but rather contain references to the matching documents' storage locations and unique identifiers as shown in FIG. 6. The unique identifiers refer to those assigned to the matching documents during initial import 102. Additionally the match sets have associated provenance metadata for the matcher algorithms by which they are produced, including the algorithm's identifier and the parameters specific to its instantiation. The matcher provenance is used to differentiate match set results by the statistical model that selects and combines the match sets.

A statistical model may be constructed using the results of human evaluation of a random sample of match sets produced by the matcher ensemble. The human evaluator may be presented with the query document and each pair-wise combination with the matching documents represented by a match set. The evaluator determines whether the two documents refer to the same provider, and the determination (or score) is stored for future reference. It is possible that a match set contains both correct and incorrect matches.

The collection of match scores forms the basis of the training data for building the statistical model, along with the feature vector for each document in the training data. For example, an example of the feature vector may be:

{
query_source: “ppd_quarterly_startup”,
match_source: “nppes_npi”,
actual_match: True,
field_distances: {
“address.address_lines”: 0,
“address.city”: 0,
“address.zipcode”: 4,
“address.state”: 0,
“address.country”: 0,
“name.first_name”: 0,
“name.last_name”: 0,
“name.middle_name”: 5,
“gender”: 0,
“practice_phone”: 0
}
}

A sparse representation of the feature vector for one record in the training data set. This shows that a provider document in the ppd_quarterly_startup source was correctly matched with a provider document in the nppes_npi source. The presence of a field name in the field_distances data structure indicates that the field was present in both documents, and the associated number is the Levenshtein distance between the field values in the two documents. These field names are based on the example in FIG. 5, which is the data from FIG. 4 with transformed vocabulary.

These features (and all the features of the entire training data set) are the predictors for the Bayesian classifier.

The feature vectors may be comprised of individual data points such as document sources, available document fields, similarity of fields between query and matching documents. Bayesian inference may then be used to determine whether a proposed match as presented in a match set is predicted to be valid. By taking each match in a match set into consideration individually it is possible to accept or reject subsets. As the same set of query documents is used across matchers, the accepted matches for each query document across all matchers are able to be combined into complete match sets.

At this point in the process, the combined match sets are still represented by references to the documents of interest. The next step is to merge the referenced documents (108) (including both the query and match documents) into a single document, with values from all fields present in each. Provenance is maintained for all field/value combinations to track their origins. New unique identifiers are assigned to the resulting merged documents, even if the merge resulted from a singleton match set.

The merged documents may have conflicting values for any given field. The process may thus have a resolve process 110 to resolve such conflicts and rank the values according to confidence in each value's correctness. The resolve process 110 may be accomplished using a combination of heuristics including majority rule (value support), predetermined confidence for data sources (e.g., trusting state medical boards for practitioner licensing data), or once again a statistical model built from human feedback. For example, a “majority rule” resolver would determine the most consistent data value for a given field based on which value for the given data filed that occurs most often. At least three sources would be needed to determine a “winner”. For the merged document in FIG. 8, this resolver would score the ‘name’ field from ppd_quarterly_startup_dbl and state_licensures_TX sources as 0.66, since those two (first_name, middle_name, last_name) tuples out of the three were identical. The state_licensure_KS would score 0.33, since it only occurred once out of the three. In this case the winner could either be the ppd_quarterly_startup or state_licensure_TX since their scores are identical.

The canonical documents, which were used as the query documents by the matcher ensemble, now have the field/value combinations from matching documents folded in, along with rankings for each. The consumer of these new documents, as outputted by the system, may choose to utilize the ranked values as appropriate, the simplest case being only to take the highest ranking values. Alternately, the combined documents with ranked values may be preserved as is for display in a faceted browsing system for exploration by the user. The combined documents, in the healthcare example scenario, may be stored in a master directory 112 for healthcare providers.

FIG. 2 is an example of an implementation of a system 200 for matching and merging documents and FIG. 3 illustrates more details of the system shown in FIG. 2. The system 200 may have one or more data sources 202, such as data source 1, data source 2, . . . , data source N as shown in FIG. 2, from which data about a particular entity may be pulled. In the healthcare example scenario, the entity may be a healthcare provider and the one or more data sources may be publicly available Centers for Medicare and Medicaid Services' (CMS) National Plan and Provider Enumeration System (NPPES) data and/or privately curated and licensed data from the American Medical Association (AMA). In the healthcare example scenario, the data in the data sources may take the form of structured records on a per-provider basis, referred to herein as provider documents.

The one or more data sources 202 may be geographically dispersed or co-located, but each may have a connection to a communication path 204 and may be implemented as a software or hardware based data store or database. The one or more data sources 202 may have data obtained from them over a communication path 204 by a backend unit 206. The communication path 204 may be any wired or wireless network that allows the backend unit 206 to collect data from the data sources, such as the Internet, a wireless data or computer network, a wired data network and the like.

The backend unit 206 may be implemented using one or more cloud computing resources or one or more server computing resources such as at least a processor and a memory. The backend unit may further comprise a plurality of components wherein each component performs one or more processes to implement the matching and merging functionality of the system. Each component may be a plurality of lines of computer code that may be resident in the memory of the cloud computing resources or one or more server computing resources and executed by the processor of the cloud computing resources or one or more server computing resources. Alternatively, each component may be a piece of hardware that implements the operations and processes described. For example, each component may be a programmable logic device, a microprocessor or microcontroller with microcode, an application specific integrated circuit and the like.

The components of the backend unit 206 may include an import and transform component 206A that may perform the import and transform processes 102,104 described above with reference to FIG. 1 and a matcher component 206B that may house the one or more matcher algorithms described above that perform the matching process 106 in FIG. 1. The components of the backend unit 206 may further include a merge component 206C that may perform the merge process 110 and a resolver component 206D that may perform the resolve process 112 described above with reference to FIG. 1.

In addition to the components, the backend unit 206 may be coupled to a repository 208 that may store the match sets, the merged documents and the merged documents with rank values. In the healthcare example scenario, the repository 208 may also store the healthcare provider directory based on the merged documents with rank values.

FIG. 3 illustrates the system when used for a healthcare example scenario in which healthcare provider data is generated from the disparate data sources. Thus, as shown in FIG. 3, each disparate data source may have one or more provider documents that contain data about a particular healthcare provider. Similar to the process shown in FIG. 1, the provider documents may be import and transformed into aligned, structured provider data that may be stored in the repository 208 shown in FIG. 2. As described above, the aligned, structured provider data documents may be matched using one or more matchers and each matcher generates a match set (an example of which is shown in FIG. 6.) The match sets for each matcher (with the provenance) may be stored in the repository and a human verification process may be performed as described above.

Once the human review process is completed, the combined, accepted matches and their match sets may be merged together with the provenance from the match sets. An example of an excerpt from such a document is shown in FIG. 7. Those merged document with provenance may be unresolved and those documents may be input to the resolver 206D which then generates merged documents with ranked values. An example of an excerpt from a merged document with ranked values provenance is shown in FIG. 8.

While the foregoing has been with reference to a particular embodiment of the invention, it will be appreciated by those skilled in the art that changes in this embodiment may be made without departing from the principles and spirit of the disclosure, the scope of which is defined by the appended claims.

Claims

1. A system for matching and merging data for an entity from disparate data sources, comprising: one or more data sources for disparate sources, each data source having a raw file about an entity, each raw file having a plurality of data fields and each field has a value associated with the entity;a computer system having a processor, the computer system coupled to each of the one or more data sources;the processor of the computer system being configured to receive the one or more raw files about the entity from the one or more data sources;the processor of the computer system being configured to perform each of a plurality of matching processes against each of the one or more raw files about the entity to generate a match set for each raw file matched with one of the plurality of matching processes based on results from each of the plurality of matching processes with each of the raw files wherein the match set for each raw file matched by each one of the plurality of matching processes a reference to a document storage location of the raw file, a unique identifier for the raw file and provenance metadata for the one matching process including an identifier of the matching process and parameters associated with the matching process;the processor of the computer system being configured to merge the one or more raw files represented into a merged document that has the at least one matched data field in each of the one or more raw files; andthe processor of the computer system being configured to identify conflicting values in each data field of the merged document.

2. The system of claim 1, wherein the processor is configured to transform, before performing the plurality of matching processes, the one or more raw files to initial data cleanse the one or more raw files and generate one or more transformed files about the entity.

3. The system of claim 1, wherein the processor is configured to rank each conflicting value for a data field based on a confidence of a correctness of each conflicting value.

4. The system of claim 1, wherein the processor is configured to perform a sequence of queries on the one or more raw files, wherein each query generates the match set.

5. The system of claim 1, wherein the processor is configured to use a Bayesian identity resolution process and uses an ElasticSearch process.

6. The system of claim 1, wherein the entity is one of a healthcare provider, subject, an idea, a professional, a person, a corporation and a business entity.

7. The system of claim 1, wherein the entity is a healthcare provider and the one or more raw files are a Centers for Medicare and Medicaid Services' (CMS) National Plan and Provider Enumeration System (NPPES) data file and an American Medical Association (AMA) file.

8. A method for matching and merging data for an entity from disparate data sources, comprising: receiving, by a computer system, one or more raw files from disparate sources about an entity, each raw file having a plurality of data fields and each field has a value associated with the entity;performing, by the computer system, each of a plurality of matching processes against each of the one or more raw files about the entity to generate a match set for each raw file matched with one of the plurality of matching processes based on results from each of the plurality of matching processes with each of the raw files wherein the match set for each raw file matched by each one of the plurality of matching processes a reference to a document storage location of the raw file, a unique identifier for the raw file and provenance metadata for the one matching process including an identifier of the matching process and parameters associated with the matching process;merging, by the computer system, the one or more raw files represented into a merged document that has the at least one matched data field in each of the one or more raw files; andidentifying, by the computer system, conflicting values in each data field of the merged document.

9. The method of claim 8 further comprising transforming, by the computer system before performing the plurality of matching processes, the one or more raw files to initial data cleanse the one or more raw files and generate one or more transformed files about the entity.

10. The method of claim 8, wherein identifying the conflicting values further comprises ranking each conflicting value for a data field based on a confidence of a correctness of each conflicting value.

11. The method of claim 8, wherein performing the plurality of matching processes further comprises performing a sequence of queries on the one or more raw files, wherein each query generates the match set.

12. The method of claim 11, wherein the match set for a particular raw file has a reference to a storage location of the particular raw file and a unique identifier.

13. The method of claim 8, wherein performing the plurality of matching processes further comprises using a Bayesian identity resolution process and using an ElasticSearch process.

14. The method of claim 8, wherein the entity is one of a healthcare provider, subject, an idea, a professional, a person, a corporation and a business entity.

15. The method of claim 8, wherein the entity is a healthcare provider and the one or more raw files are a Centers for Medicare and Medicaid Services' (CMS) National Plan and Provider Enumeration System (NPPES) data file and an American Medical Association (AMA) file.

16. A system for matching and merging data for an entity from disparate data sources, comprising: one or more data sources for disparate sources, each data source having a raw file about an entity, each raw file having a plurality of data fields and each field has a value associated with the entity;a computer system having a processor, the computer system coupled to each of the one or more data sources;the processor of the computer system being configured to perform each of a plurality of matching processes against each of the one or more raw files about the entity to generate a match set for each raw file matched with one of the plurality of matching processes wherein the match set for each raw file matched by each one of the plurality of matching processes a reference to a document storage location of the raw file, a unique identifier for the raw file and provenance metadata for the one matching process including an identifier of the matching process and parameters associated with the matching process, the plurality of matching processes including a strict matcher process that utilizes statistically significant combinations of biographic identifiers including each component of a person's full name, a birth date and a birth place and a loose matcher process that allows for a variation in a person's name;the processor of the computer system being configured to merge the one or more raw files represented into a merged document that has the at least one matched data field in each of the one or more raw files; andthe processor of the computer system being configured to identify conflicting values in each data field of the merged document.

17. A method for matching and merging data for an entity from disparate data sources, comprising: receiving, by a computer system, one or more raw files from disparate sources about an entity, each raw file having a plurality of data fields and each field has a value associated with the entity;performing, by the computer system, a plurality of matching processes against the one or more raw files about the entity to generate a match set for each raw file matched with one of the plurality of matching processes wherein the match set for each raw file matched by each one of the plurality of matching processes a reference to a document storage location of the raw file, a unique identifier for the raw file and provenance metadata for the one matching process including an identifier of the matching process and parameters associated with the matching process, the plurality of matching processes including a strict matcher process that utilizes statistically significant combinations of biographic identifiers including each component of a person's full name, a birth date and a birth place and a loose matcher process that allows for a variation in a person's name;merging, by the computer system, the one or more raw files represented into a merged document that has the at least one matched data field in each of the one or more raw files; andidentifying, by the computer system, conflicting values in each data field of the merged document.