Identifying Ancestral Relationships using a Continuous Stream of Input

Abstract

Identification of inheritance-by-descent haplotype matches between individuals is described. A set of tables including word match, haplotypes and segment match tables are populated. DNA samples are received and stored. A word identification module extracts haplotype values from each sample. The word match table is indexed according to the unique combination of position and haplotype. Each column represents a different sample, and each cell indicates whether that sample includes that haplotype at that position. The haplotypes table includes the raw haplotype data for each sample. The segment match table is indexed by sample identifier, and columns represent other samples. Each cell is populated to indicate for each identified sample pair which position range(s) include matching haplotypes for both samples. The tables are persistently stored in databases of the matching system. As new sample data is received, each table is updated to include the newly received samples, and additional matching takes place.

Description

BACKGROUND

1. Field

The disclosed embodiments relate to identifying individuals in an existing dataset of genetic information that are related to individuals whose genetic information is newly analyzed.

2. Description of Related Art

Although humans are, genetically speaking, almost entirely identical, small differences in our DNA are responsible for much of the variation between individuals. Stretches of DNA that are determined to be relevant for some purpose are referred to as haplotypes. Haplotypes are identified based on consecutive single nucleotide polymorphisms (SNPs) of varying length. Certain haplotypes shared by individuals suggests a familial relationship between those individuals based on a principal known as identity-by-descent (IBD).

Because identifying segments of IBD DNA between pairs of genotyped individuals is useful in many applications, numerous methods have been developed to perform IBD analysis (Purcell et al. 2007, Gusev et al. 2009, Browning and Browning 2011, Browning and Browning 2013). However, these approaches do not scale for continuously growing very large datasets. For example, the existing GERMLINE implementation is designed to take a single input file containing all individuals to be compared against one another. While appropriate for the case in which all samples are genotyped and analyzed simultaneously, this approach is not practical when samples are collected incrementally.

SUMMARY

Described embodiments enable identification of IBD and consequently familial relationships within received samples and between a received sample and samples in an existing data set. An initial configuration includes populating a set of tables including a word match table, a haplotypes table and a segment match table. A set of phased DNA samples are received, e.g., from a DNA service, and stored in a DNA database. A word identification module extracts haplotype values from each sample. The word match table is indexed in one embodiment according to haplotypes, for example a specific haplotype on a specific chromosome. Each column of the word match table represents a different sample, and each cell includes an indication of whether that sample includes that haplotype at that position. The haplotypes table is populated to include the raw haplotype data for each sample. The segment match table is indexed by sample identifier, and columns represent other samples. Each cell of the table is populated to indicate for each identified sample pair which position range(s) include matching haplotypes for both samples.

The tables are persistently stored in databases of the matching system. Subsequently, as new sample data is received, each of the tables is updated to include the newly received samples, and additional matching takes place. The persistence of the tables avoids the necessity of recomputing relationships with the addition of each new sample, thus allowing for rapid and efficient scaling of the identification system and accommodation of continuous or periodic input of new sample data.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of a system architecture and environment in accordance with one embodiment.

FIG. 2 illustrates an example of a word match table according to one embodiment.

FIG. 3 illustrates an example of a haplotype table in accordance with one embodiment.

FIG. 4 illustrates an example of a segment match table in accordance with one embodiment.

FIG. 5 illustrates a process for processing new samples in accordance with one embodiment.

FIG. 6 illustrates a process for updating a word match table in accordance with one embodiment.

FIG. 7 illustrates a process for updating a segment match table in accordance with one embodiment.

DETAILED DESCRIPTION

FIG. 1 is a block diagram of the architecture and environment of system 100 according to one embodiment. System 100 includes a word identification module 103, a word match database 107, a segment match database 109, a haplotype database 111, and a DNA database 113. Also depicted in FIG. 1 are individual 101 (i.e. a human or other organism), DNA extraction service 102, and a DNA quality control (QC) and matching preparation service 115. Each of these illustrated features is described further below. Note that for purposes of clarity, only one of each item is included in the figure, but when implemented multiple instances of any or all of the depicted modules may be employed, as will be appreciated by those of skill in the art.

System 100 may be implemented in hardware or a combination of hardware and software. For example, system 100 may be implemented by one or more computers having one or more processors executing application code to perform the steps described here, and data may be stored on any conventional storage medium and, where appropriate, include a conventional database server implementation. For purposes of clarity and because they are well known to those of skill in the art, various components of a computer system, for example, processors, memory, input devices, network devices and the like are not shown in FIG. 1. In some embodiments, a distributed computing architecture is used to implement the described features. One example of such a distributed computing platform is the Apache Hadoop project available from the Apache Software Foundation.

Individuals 101 provide DNA samples for analysis of their genetic data. In one embodiment, an individual uses a sample collection kit to provide a sample, e.g., saliva, from which genetic data can be reliably extracted according to conventional methods. DNA extraction service 102 receives the sample and genotypes the genetic data, for example by extracting the DNA from the sample and identifying values of single nucleotide polymorphisms (SNPs) present within the DNA. DNA QC and matching preparation service 115 phases the genetic data and assesses data quality by checking various attributes such as genotyping call rate, genotyping heterozygosity rate, and agreement between genetic and self-reported gender. System 100 receives the genetic data from DNA extraction service 102 and stores the genetic data in DNA database 113.

Initial Configuration

For ease of explanation, we assume an initial set of DNA samples have been collected and are stored in DNA database 113. A decision for the implementer is to select a segment or window length to be used by system 100. In practice, window sizes of 100 or more SNPs are appropriate, but for purposes of illustration here we assume without loss of generality a window size of 10 SNP markers. The value of each marker will be one of two genetic bases, and each individual is associated with two sequences of values called haplotypes, because each person has two copies of each chromosome. For example, the genetic data for individual 1, haplotype 1, markers 1 through 10 might be: GCCATATGGC.

In processing the initial samples, word identification module 103 and matching module 105 populate an initial set of tables that we refer to below as the word match table, haplotypes table and segment match table. In the illustrated embodiment, the word match table is stored in word match database 107; the haplotypes table is stored in haplotype database 111; and the segment match table is stored in segment match database 109.

FIG. 2 illustrates a word match table 202 according to one embodiment. Word identification module 103 obtains genetic data for an individual from DNA database 113 and uses the data to populate the table. In one embodiment, each row of word match table 202 specifies a haplotype observed in a particular genomic window, and each column specifies an individual (referred to interchangeably as a user). A table cell (i.e. a particular row/column combination) therefore indicates whether that user has that haplotype, and reading across a row enables identification of all such users with the haplotype. In various embodiments, the haplotypes are hashed prior to insertion into the table 202. Depending on the window size and number of samples, this table may be very large. However, it is also very sparse as each individual has at most two distinct haplotypes for each window. Distributed, scalable big data stores such as Apache HBase are particularly well suited to storing and querying very large, very sparse tables such as this. In one embodiment, the row keys for word match table 202 are of the form [CHROMOSOME] [POSITION] [HAPLOTYPE], where [CHROMOSOME] and [POSITION] denote the unique genomic location of the first SNP in a particular window and [HAPLOTYPE] is the sequence of bases observed in this window.

For example, referring still to FIG. 2, a row key of the form chr1_—0000000010_ACTACGACCA refers to the haplotype ACTACGACCA observed in the window beginning at the 10^th SNP on chromosome 1. Executing a “Get” operation against the Window Match table with this key returns a list of all users having that haplotype at that position, which could be, for example, the following collection of columns:

U₇, U₅, U₇.

This indicates that samples U₁, U₅, and U₇ have at least one copy of that haplotype in this window on this chromosome. Word identification module 103 adds a new row to table 202 each time a haplotype is observed for a first time at a particular window on a particular chromosome, and adds an indicator to each user's column if that user's sample includes the presence of that haplotype.

Referring now to FIG. 3, word identification module 103 also populates a haplotype table 302 with the raw phased haplotype data for each window and each sample. The row keys for haplotype table 302 are of the form [CHROMOSOME] [USER ID]. The columns in haplotype table 302 represent the pair of haplotypes observed in each window for the specified sample and chromosome. Each column is indexed by [POSITION] [EVEN OR ODD]. In the illustrated embodiment, the position is the first base in that particular window as numbered from the beginning of the sequence of nucleic acids for that sample. The value for each cell is the sequence for the specified sample at the given chromosome and position. When populating the table with new data, word identification module 103 arbitrarily labels each chromosomal haplotype pair as ODD or EVEN, thus while these definitions are consistent between neighboring windows on the same chromosome, there is no relationship between different chromosomes of the same individual. In the example illustrated in FIG. 3, the row key is of the form Chr1_U₁. This key would return all haplotype data for each window for sample U₁ on chromosome 1. Storing the sequence of each window is useful for fuzzy matches between individuals after exact matches have been determined, as described further below. Word identification module 103 in the illustrated embodiment continues to add rows for each user and each chromosome until all of the initial samples are reflected in haplotypes table 302.

Next, and referring now to FIG. 4, matching module 105 populates the segment match table 402 to identify individuals who have at least one haplotype in common at a given location. Matching module 105 uses data from the word match table 202 as well as the haplotype table 302. These pairs of individuals are stored in the segment match database 109. The operation of the matching module 105 is discussed in greater detail below.

FIG. 4 illustrates a segment match table 402 according to one embodiment. Each row index represents a specific user and chromosome combination. Each column represents an individual and each cell contains a coordinate array indicating the locations of matching segments on that chromosome between the specified pair of individuals. In various embodiments, only matching segments of at least a threshold length, e.g., 5 Mb, 1 cM, etc., as determined by the implementer, are considered a match for purposes of insertion into table 402. The row key for this table in the illustrated embodiment is of the form [CHROMOSOME] [USER ID]. In the first row of FIG. 4, the row key Chr1_U₁ specifies the row containing matches for user 1 along chromosome 1. Performing a “Get” operation against the segment match table 402 with key Chr1_U₁ returns column U₂ for user 2 as well as other users with whom user 1 has segments in common. For example, the operation might return:

• U₂, U₄, U₅₃₈, U₇₀₃₄,indicating that the sample with ID U1 has segments in common with samples U₂, U₄, U₅₃₈, and U₇₀₃₄.

The cell for U₂ might contain values 10-40, 120-130, 550-700, 800-4560, indicating that along chromosome 1, for segment ranges 10-40, 120-130, 550-700, 800-4560 samples U₁ and U₂ contain at least one haplotype that is identical or nearly identical.

Matching module 105 proceeds to populate the segment match table 402 for each user and each chromosome. In one embodiment, a fuzzy matching algorithm is used to extend identified matches between the haplotypes of two users. For example, suppose that within chromosome 1, U₁ has an exact segment matching with U₅₃₂ from SNPs 100-299. Matching module 105 executes a fuzzy extension process that attempts to extend this match on both sides while allowing for small numbers of unmatched bases. To extend this segment on the left flank, matching module 105 performs a “Get” call against haplotypes table 302, supplying the row keys chr1_U₁ and chr1_U₅₃₂, requesting the following columns:

0000000000_E, 0000000000_O, 0000000010_E, 0000000010_O . . . 0000000090_E, 0000000090_O.

The cells of this table contain the actual haplotypes for these windows and matching module 105 can extend a match by locating the sites containing alternate homozygotes (e.g., T/T for one sample and G/G for the other). Depending on the parameters specified by the implementer, fuzzy match extension proceeds until x alternate homozygotes are encountered. This process is repeated for each flank of each segment, updating the appropriate cells in segment match table 402 to reflect the longer matching string. Matching module 105 proceeds to populate the segment match table 402 for each user and each chromosome.

Addition of New Samples

After the initial population of tables 302, 202 and 402 in their respective databases 111, 107, 109 by word identification module 103 and matching module 105, new samples from additional individuals 101 can be received, and relationships between those individuals and both the previously processed individuals and other new individuals can be determined as described below.

Referring to FIG. 5, a process for processing new samples can be described generally as follows. A new sample is received 502 and stored in DNA database 113 as described above. Word match table 202 is updated 504, haplotypes table 302 is updated 506, and segment match table 402 is also updated 508 to reflect the newly identified relationships. We describe each of these processes in turn.

Referring to FIG. 6, word identification module 103 obtains 602 the new sample data, e.g., from DNA database 113. Word identification module 103 then updates the word match table 202, which, as noted, is persistently stored in database 107. To update the table 202, word identification module 103 first adds 604 a column to the world table to accommodate values for the new user. Next 606, for each haplotype in the user sample, identification module 103 determines 608 whether that haplotype is already in table 202. If not, module 103 adds 610 a new row for the table. Once the appropriate row is added or found, module 103 updates the user's column to indicate 612 the presence of the haplotype in the user's sample. The process then repeats 606 for each haplotype and for each newly received sample. The updated word match table 202 remains stored in database 107.

Next, word identification module 103 updates 506 haplotypes table 302 by adding a new row for each chromosome of the new user sample, and inserting the relevant haplotype data for each window.

In one embodiment, word identification module 103 updates 504, 506 the word match and haplotypes tables for each new sample being processed in a current batch, and then matching module 105 updates 508 the segment match table 402. Referring to FIG. 7, matching module 105 obtains 702 the new sample data and adds 704 a column to segment match table 402 for the new user. Next 706, for each chromosome of the new user, in an embodiment such as the one illustrated in FIG. 4 in which each row represents a particular chromosome for a particular user, matching module 105 adds 707 a row to segment match table 402. Then 708, for each of the existing users with data in table 402, matching module 105 determines 710 whether any haplotype matches of a minimum threshold length exist between the new user's sample and the existing user's sample on that chromosome. If so, matching module 105 extends 712 the match where possible, using a fuzzy logic approach described above, and enters 714 the results of the match in the appropriate cell of table 402. This process then continues, matching the new user against each existing user across each chromosome.

In one embodiment, following the update procedure of FIG. 5, a reducer is executed that eliminates duplicate matches due to reciprocal comparisons and creates an output file with the results from the process run. The data in the file is then usable to inform the implementer about potential relationships due to IBD between the various user samples. In some embodiments, system 100 is implemented as part of a service that enables subscribing users to submit samples and receive reports about other subscribers to whom they may be related, for example for those users who share a sufficiently high number of haplotype matches as described above. Depending on a user's privacy settings and opt-in preferences, examples of information provided might include blind (anonymous) introductions, account identifiers, or names or other contact information.

In addition to the embodiments specifically described above, those of skill in the art will appreciate that the invention may additionally be practiced in other embodiments. Within this written description, the particular naming of the components, capitalization of terms, the attributes, data structures, or any other programming or structural aspect is not mandatory or significant unless otherwise noted, and the mechanisms that implement the described invention or its features may have different names, formats, or protocols. Further, the system may be implemented via a combination of hardware and software, as described, or entirely in hardware elements. Also, the particular division of functionality between the various system components described here is not mandatory; functions performed by a single module or system component may instead be performed by multiple components, and functions performed by multiple components may instead be performed by a single component. Likewise, the order in which method steps are performed is not mandatory unless otherwise noted or logically required. It should be noted that the process steps and instructions of the present invention could be embodied in software, firmware or hardware, and when embodied in software, could be downloaded to reside on and be operated from different platforms used by real time network operating systems.

Algorithmic descriptions and representations included in this description are understood to be implemented by computer programs. Furthermore, it has also proven convenient at times, to refer to these arrangements of operations as modules or code devices, without loss of generality.

Unless otherwise indicated, discussions utilizing terms such as “selecting” or “computing” or “determining” or the like refer to the action and processes of a computer system, or similar electronic computing device, that manipulates and transforms data represented as physical (electronic) quantities within the computer system memories or registers or other such information storage, transmission or display devices.

The present invention also relates to an apparatus for performing the operations herein. This apparatus may be specially constructed for the required purposes, or it may comprise a general-purpose computer selectively activated or reconfigured by a computer program stored in the computer. Such a computer program may be stored in a computer readable storage medium, such as, but is not limited to, any type of disk including floppy disks, optical disks, DVDs, CD-ROMs, magnetic-optical disks, read-only memories (ROMs), random access memories (RAMs), EPROMs, EEPROMs, magnetic or optical cards, application specific integrated circuits (ASICs), or any type of media suitable for storing electronic instructions, and each coupled to a computer system bus. Furthermore, the computers referred to in the specification may include a single processor or may be architectures employing multiple processor designs for increased computing capability.

The algorithms and displays presented are not inherently related to any particular computer or other apparatus. Various general-purpose systems may also be used with programs in accordance with the teachings above, or it may prove convenient to construct more specialized apparatus to perform the required method steps. The required structure for a variety of these systems will appear from the description above. In addition, a variety of programming languages may be used to implement the teachings above.

Finally, it should be noted that the language used in the specification has been principally selected for readability and instructional purposes, and may not have been selected to delineate or circumscribe the inventive subject matter. Accordingly, the disclosure of the present invention is intended to be illustrative, but not limiting, of the scope of the invention.

Claims

1. A computer-implemented method for identifying related individuals, the method comprising: receiving a first set of genetic data, the first set including genetic data sampled from a first plurality of human individuals;for each sample in the received first set: for each of a plurality of haplotypes in the sample, storing an indication in a first table that the sample contains the haplotype, the first table having rows indexed by haplotype;storing in a second table a plurality of haplotype values included in the sample, the second table having rows indexed by individual;storing in a third table a range of haplotype matches between the sample and each other sample, the third table indexed by individual;receiving a second set of genetic data, the second set including genetic data sampled from a second plurality of human individuals;retrieving from a persistent storage device the first, second, and third tables;for each sample in the received second set: for each of a plurality of haplotypes in the sample, updating the first table with an indication that the sample contains the haplotype;updating the second table to include a plurality of haplotype values included in the sample;storing in the third table a range of haplotype matches between the sample and each other sample; andproviding to at least two of the first or second plurality of human individuals an indication that a number of haplotype matches between them exceeds a threshold number.

2. The method of claim 1 wherein updating the first table with an indication that the sample contains the haplotype further comprises: determining that the haplotype is not in the first table;adding the haplotype to a row in the first table; andindicating in the added row that the sample contains the haplotype.

3. The method of claim 1 wherein the samples of the two individuals to whom the indication is provided include matching haplotype segments of at least 1 centimorgan.

4. The method of claim 1 wherein the first table is indexed by a hash value of the haplotype and updating the first table further comprises: identifying a haplotype in the sample;hashing the haplotype;identifying in the first table a row indexed by the hashed haplotype; andstoring in the row of the first table an indication that the sample contains the haplotype.

5. A computer program product for identifying related individuals, the computer program product stored on a non-transitory computer readable medium and including instructions configured to cause a processor to execute steps comprising: receiving a first set of genetic data, the first set including genetic data sampled from a first plurality of human individuals;for each sample in the received first set: for each of a plurality of haplotypes in the sample, storing an indication in a first table that the sample contains the haplotype, the first table having rows indexed by haplotype;storing in a second table a plurality of haplotype values included in the sample, the second table having rows indexed by individual;storing in a third table a range of haplotype matches between the sample and each other sample, the third table indexed by individual;receiving a second set of genetic data, the second set including genetic data sampled from a second plurality of human individuals;retrieving from a persistent storage device the first, second, and third tables;for each sample in the received second set: for each of a plurality of haplotypes in the sample, updating the first table with an indication that the sample contains the haplotype;updating the second table to include a plurality of haplotype values included in the sample;storing in the third table a range of haplotype matches between the sample and each other sample; andproviding to at least two of the first or second plurality of human individuals an indication that a number of haplotype matches between them exceeds a threshold number.

6. The computer program product of claim 5 wherein updating the first table with an indication that the sample contains the haplotype further comprises: determining that the haplotype is not in the first table;adding the haplotype to a row in the first table; andindicating in the added row that the sample contains the haplotype.

7. The computer program product of claim 5 wherein the samples of the two individuals to whom the indication is provided include matching haplotype segments of at least 1 centimorgan.

8. The computer program product of claim 5 wherein the first table is indexed by a hash value of the haplotype and updating the first table further comprises: identifying a haplotype in the sample;hashing the haplotype;identifying in the first table a row indexed by the hashed haplotype; andstoring in the row of the first table an indication that the sample contains the haplotype.

9. A system for identifying related individuals, the system comprising: a DNA database including genetic data sampled from a first and a second plurality of human individuals;a word match database including a stored word match table;a haplotypes database including a stored haplotypes table;a segment match database including a stored segment match table;a word identification module, coupled to the DNA database, the word match database and haplotypes database, configured to, for each sample in the first plurality of samples: for each of a plurality of haplotypes in the sample, store an indication in the word match table that the sample contains the haplotype, the word match table having rows indexed by haplotype;store in the haplotypes table a plurality of haplotype values included in the sample, the haplotypes table having rows indexed by individual;a matching module, coupled to the DNA database, the word match database, the haplotypes database and the segment match database configured to, for each sample in the first plurality of samples, store in the segment match table a range of haplotype matches between the sample and each other sample, the segment match table indexed by individual;wherein the word identification module is further configured to retrieve the words match table and the haplotypes table from the word match database and segment match database and, for each sample in the second plurality of samples: for each of a plurality of haplotypes in the sample, updating the word match table with an indication that the sample contains the haplotype;updating the haplotypes table to include a plurality of haplotype values included in the sample;and wherein the matching module is further configured to, for each sample in the second plurality of samples: store in the segment match table a range of haplotype matches between the sample and each other sample; andprovide to at least two of the first or second plurality of human individuals an indication that a number of haplotype matches between them exceeds a threshold number.

10. The system of claim 9 wherein the word identification module is further configured to update the word match table with an indication that the sample contains the haplotype by: determining that the haplotype is not in the word match table;adding the haplotype to a row in the word match table; andindicating in the added row that the sample contains the haplotype.

11. The system of claim 9 wherein the samples of the two individuals to whom the indication is provided include matching haplotype segments of at least 1 centimorgan.

12. The system of claim 9 wherein the word match table is indexed by a hash value of the haplotype and wherein the word identification module is further configured to update the word match table by: identifying a haplotype in the sample;hashing the haplotype;identifying in the word match table a row indexed by the hashed haplotype; andstoring in the row of the first table an indication that the sample contains the haplotype.