Patent Application
20230207047
The technology disclosed relates to artificial intelligence type computers and digital data processing systems and corresponding data processing methods and products for emulation of intelligence (i.e., knowledge based systems, reasoning systems, and knowledge acquisition systems); and including systems for reasoning with uncertainty (e.g., fuzzy logic systems), adaptive systems, machine learning systems, and artificial neural networks. In particular, the technology disclosed relates to using neural networks to analyze ordered data.
The following are incorporated by reference for all purposes as if fully set forth herein:
The subject matter discussed in this section should not be assumed to be prior art merely as a result of its mention in this section. Similarly, a problem mentioned in this section or associated with the subject matter provided as background should not be assumed to have been previously recognized in the prior art. The subject matter in this section merely represents different approaches, which in and of themselves can also correspond to implementations of the claimed technology.
Predicting functional consequences of variants relies at least in part on the assumption that crucial amino acids for protein families are conserved through evolution due to negative selection (i.e., amino acid changes at these sites were deleterious in the past), and that mutations at these sites have an increased likelihood of being pathogenic (causing disease) in humans. Homologous sequences of a target protein are collected and aligned in a multiple sequence alignment (MSA). A metric of conservation is computed based on weighted scores and/or frequencies of different amino acids observed in the target position in the MSA.
An MSA is generally the alignment of three or more biological sequences, protein, or nucleic acid, of similar length. From the alignment, the degree of homology can be inferred and the evolutionary relationships among the sequences studied. An MSA is also a tool used to identify the evolutionary relationships and common patterns among genes. Alignments are generated and analyzed using computational algorithms. Dynamic and heuristic approaches are used in most MSA algorithms. One of the objectives of alignment is to detect structural or functional identities and similarities between residues in protein sequences relative to other protein sequences.
Homolog information pertaining to aligned sequences in the MSA can be represented by two matrices (evolutionary conservation metrics): a position-specific scoring matrix (PSSM) and a position-specific frequency matrix (PSFM). PSSMs and PSFMs reflect the conservation of residues at specific positions of protein chains based on evolutionary information.
PSSMs and PSFMs are agnostic to the different degrees of evolutionary proximity between the species of non-query sequences in the MSA and the species of a query sequence in the MSA. That is, varied evolutionary distances between species of non-query sequences in the MSA, and the species of a query sequence in the MSA are not accounted for in the PSSMs and PSFMs. Variants from all species and sequences in the MSA contribute equally to the PSSMs and PSFMs. For example, a variant in chimp (closely related to human) increases the frequency of an amino acid by the same amount as the same variant in Zebrafish (distant to human). Such a deficiency in the evolutionary conservation state information can cause inaccurate phenotype prediction.
Therefore, an opportunity arises to make evolutionary conservation metrics like PSSMs and PSFMs aware of the relative evolutionary proximities of non-query species to a query species in an MSA. More complete evolutionary conservation state information and more accurate phenotype prediction may result.
In the drawings, like reference characters generally refer to like parts throughout the different views. Also, the drawings are not necessarily to scale, with an emphasis instead generally being placed upon illustrating the principles of the technology disclosed. In the following description, various implementations of the technology disclosed are described with reference to the following drawings, in which.
The following discussion is presented to enable any person skilled in the art to make and use the technology disclosed and is provided in the context of a particular application and its requirements. Various modifications to the disclosed implementations will be readily apparent to those skilled in the art, and the general principles defined herein can be applied to other implementations and applications without departing from the spirit and scope of the technology disclosed. Thus, the technology disclosed is not intended to be limited to the implementations shown but is to be accorded the widest scope consistent with the principles and features disclosed herein.
The detailed description of various implementations will be better understood when read in conjunction with the appended drawings. To the extent that the figures illustrate diagrams of the functional blocks of the various implementations, the functional blocks are not necessarily indicative of the division between hardware circuitry. Thus, for example, one or more of the functional blocks (e.g., modules, processors, or memories) can be implemented in a single piece of hardware (e.g., a general purpose signal processor or a block of random access memory, hard disk, or the like) or multiple pieces of hardware. Similarly, the programs can be stand-alone programs, can be incorporated as subroutines in an operating system, can be functions in an installed software package, and the like. It should be understood that the various implementations are not limited to the arrangements and instrumentality shown in the drawings.
The processing engines and databases of the figures, designated as modules, can be implemented in hardware or software, and need not be divided up in precisely the same blocks as shown in the figures. Some of the modules can also be implemented on different processors, computers, or servers, or spread among a number of different processors, computers, or servers. In addition, it will be appreciated that some of the modules can be combined, operated in parallel or in a different sequence than that shown in the figures without affecting the functions achieved. The modules in the figures can also be thought of as flowchart steps in a method. A module also need not necessarily have all its code disposed contiguously in memory; some parts of the code can be separated from other parts of the code with code from other modules or other functions disposed in between.
This discussion is organized as follows. First, we introduce the notion that existing evolutionary conservation profiles are agnostic to different degrees of evolutionary proximity between species. We then propose the technology disclosed as a solution to provide the missing differential evolutionary proximity context. This is followed by an example system that generates the disclosed weighted summary statistics and uses them for phenotype prediction. After that, we show an example multiple sequence alignment (MSA) and contrast calculating weighted summary statistics using a uni-weighted approach from the disclosed differentially-weighted approach. Advancing further, we give many examples of using the weighted summary statistics as inputs to a variety of deep neural networks for variant pathogenicity prediction and protein contact map generation. Finally, some particular implementations of the technology disclosed are discussed.
Multiple sequence alignment (MSA) is an alignment of multiple homologous protein sequences to a target protein. MSA is an important step in comparative analyses and property prediction of biological sequences since a lot of information, for example, evolution and coevolution clusters, are generated from the MSA and can be mapped to a target sequence of choice or on the protein structure.
Sequence profiles of a protein sequence X of length L are an L×20 matrix, either in the form of a position-specific scoring matrix (PSSM) or a position-specific frequency matrix (PSFM), collectively position-specific matrices (PSMs). PSSMs represent the evolutionary conservation of alternate amino acids that are candidates for filling an amino acid vacancy in a vacancy-containing protein. PSSMs represent the evolutionary conservation of alternate amino acids based at least in part on structural (or spatial) compatibility between substitute amino acids and adjacent amino acids in a neighborhood of the amino acid vacancy (e.g., the right and left flanking amino acids around the amino acid vacancy). On the other hand, PSFMs represent the conservation of amino acids in a e.g., human protein sequence across aligned protein sequences of other species by determining, on a location-by-location basis, the frequency of occurrence of an amino acid in the human protein sequence across the aligned protein sequences of other species. PSFMs and PSSMs can be used to identify the evolutionary conservation profiles of homologous proteins of the plurality of species and differentiate most-conserved amino acids from non-conserved amino acids. In one implementation, the columns of a PSSM and a PSFM are indexed by the alphabet (or category) of amino acids, and each row corresponds to a position in the protein sequence X. PSSMs and PSFMs contain the substitution scores and the frequencies, respectively, of the amino acids at different positions in the protein sequence X. Each row of a PSFM is normalized to sum to one. The sequence profiles of the protein sequence X are computed by aligning X with multiple sequences in a protein database that have statistically significant sequence similarities with the protein sequence X. The sequence profiles contain more general evolutionary and structural information of the protein family that the protein sequence X belongs to, and thus, can be applied to various biological applications including phenotype prediction, evolutionary analysis, protein function prediction. One application of the sequence profiles of protein sequence X, for example, is to analyze the pathogenicity of variants when they are introduced in protein sequence X.
A protein sequence (called query sequence, e.g., a reference amino acid sequence of a protein) can be used as a seed to search and align homologous sequences from a protein database (e.g., SWISSPROT) using, for example, a PSI-BLAST program. The aligned sequences share some homologous segments and belong to the same protein family. The aligned sequences are further converted into two profiles to express their homologous information: PSSM and PSFM. Both PSSM and PSFM are matrices with twenty rows and L columns, where L is the total number of amino acids in the query sequence. Each column of a PSSM represents the log-likelihood of the residue substitutions at the corresponding positions in the query sequence. The (i, j)-th entry of the PSSM matrix represents the chance of the amino acid in the j-th position of the query sequence being mutated to amino acid type i during the evolution process. A PSFM contains the weighted observation frequencies of each position of the aligned sequences. Specifically, the (i, j)-th entry of the PSFM matrix represents the possibility of having amino acid type i in position j of the query sequence.
PSSMs and PSFMs are agnostic to the different degrees of evolutionary proximity between the species of non-query sequences in the MSA and the species of a query sequence in the MSA. That is, varied evolutionary distances between species of non-query sequences in the MSA, and the species of a query sequence in the MSA are not accounted for in the PSSMs and PSFMs. Variants from all species and sequences in the MSA contribute equally to the PSSMs and PSFMs. For example, a variant in chimp (closely related to human) increases the frequency of an amino acid by the same amount as the same variant in Zebrafish (distant to human). Such a deficiency in the evolutionary conservation state information can cause inaccurate phenotype prediction.
PSSMs and PSFMs, and their variants, are referred to herein as “evolutionary conservation metrics” or “evolutionary conservation measures” or “evolutionary profiles” or “evolutionary conservation summary statistics” or “conservation summary statistics” or “uni-weighted summary statistics,” “summary statistics,” or “PSMs.”
The majority of available evolutionary conservation metrics use dynamically-generated profiles (e.g., using BLAST). As a result, the number of alignments and the set of species are different for each protein. Without constraining the number of species, it can reach into millions of species for a protein and make learning species weights impossible. The technology disclosed relies on the availability of whole-protein human sequence alignments. As an example, for only Multiz100 alignments (a group of multiple alignments of 100 vertebrate species and evolutionary conservation using phastCons and phyloP methods generated and made publicly available by the Genomics Institute at the University of California Santa Cruz), each species being weighted equally within the alignment is reasonable because the species included in Multiz100 were carefully selected and balanced by taxonomic ranks. However, the number of available whole proteome alignments is rapidly increasing, and as a result, redundancy and diversity between alignments also increase. Therefore, it is increasingly unclear how to optimally turn alignment data into high-quality evolutionary profiles. The technology disclosed provides a way to solve this problem automatically.
The technology disclosed generates evolutionary profiles like the PSSMs and PSFMs using weights assigned to per-species sequences in the MSA. The weights can be assigned proportional to the evolutionary proximity between a given species of a non-query sequence in the MSA and a target species of a query sequence in the MSA (or inversely proportional to the evolutionary distance between the given species and the target species). Greater weights can be assigned to those non-query sequences in the MSA whose species are evolutionary closer to the target species. Smaller weights are assigned to those non-query sequences in the MSA whose species are evolutionary farther from the target species. Proportional proximity can be a starting point, adjusted according to judgment of a researcher or a consensus among colleagues.
The assignment of weights can further be leveraged to determine the optimal alignment to use as input. A weighted evolutionary profile may be informative to a wide variety of genetics studies and thus, the input alignment that is optimal will be influenced by the target application for the evolutionary profile and can be expected to differ between distinct target applications. For example, certain applications of the disclosed system may benefit more from an input alignment comprising primarily evolutionarily proximate or primarily evolutionary distant species, while other applications may necessitate an evolutionary profile generated from a more diverse set of species.
The weights can be learned through backpropagation and may not be proportional to the evolutionary proximity among species. For example, training by backward propagation has revealed that presence of many sequencing or alignment errors can cause a lower weight to be learned for a species that is closely related to the target species than the weight for an accurately sequenced, well-aligned species at a greater evolutionary distance.
BLAST queries of a protein sequence database are not species-constrained. Thus, for each protein BLAST query, the number, type, and order of species returned in the MSA can vary. This makes it highly non-trivial to incorporate relationships among species in a machine learning model. For example, sequences corresponding to a plurality of species can be encoded and used to train a machine learning model. With millions of species, the weighting would have millions of per-species sequences; hence, also requiring millions of additional model parameters and millions of additional data samples to train the machine learning model. Alternatively, one can use phylogenic trees; however, this requires extra machine learning modules trained to deal with the structure and format of the phylogenic trees. Moreover, phylogenic trees can themselves contain errors and would abstract away precise species information. In contrast, the use of fixed-species MSAs (e.g., Multiz100 alignments) to calculate PSSMs and PSFMs improves interpretability of data because included species are consistent for each of the MSAs. Nevertheless, fixed-species MSAs may still face the challenge that every species, regardless of its evolutionary distance from human species, is treated in the same way.
The introduction of weighted PSSMs and weighted PSFMs addresses species homology and gene conservation (e.g., comparing monkeys to humans versus comparing fish to humans) and non-linear relationships between species with respect to human pathogenicity (e.g., the decision whether conservation of a certain amino acid in fish should be considered to predict human pathogenicity may depend on the presence of that amino acid in certain other primate species). Finally, the technology disclosed also addresses the valuable information that an entire protein region around a protein position in each species carries. For example, considering a window of ten amino acids rather than a single amino acid position may better reflect how similar a particular protein region from another species is to humans, and therefore the extent to which mutations in that region should be considered in the calculation of human pathogenicity in a mutation.
Scores and frequencies in the weighted PSSMs and weighted PSFMs generated by the technology disclosed are modulated by the weights assigned to the underlying sequences whose position-wise occurrences/hits (for example, of amino acids or nucleotides) contribute to the scores and frequencies. Greater weights assigned to sequences of evolutionary proximate species to the target species (where evolutionary proximate species refer to species with close evolutionary distance to the target species, as measured by the number of nucleotide substitutions per site between two homologous DNA sequences, or the number of amino acid substitutions per site between two homologous protein species, where evolutionary distance is inversely proportional to weight) can result in augmented/increased/enhanced scores and frequencies in the weighted PSSMs and weighted PSFMs generated by the technology disclosed. Smaller weights assigned to sequences of evolutionary distant species from the target species can result in attenuated/decreased/diminished scores and frequencies in the weighted PSSMs and weighted PSFMs generated by the technology disclosed. In other words, conservation information from closely related species (e.g., monkey) to human can be more informative with respect to phenotype prediction and pathogenicity prediction in human, compared to distantly related species (e.g., fish).
The technology disclosed also takes into consideration non-linear relationship among species-specific weights by using convolutions. Non-linear relationships exist among species with respect to human pathogenicity. For example, the decision whether conservation of an amino acid in fish should be considered to predict human pathogenicity may depend on the presence of that amino acid in a certain monkey species. The flexibility in the size of convolutional kernels allows capturing the context information of an amino acid or a protein site of interest. The phenotypic effect of a mutation at one genetic site often depends on alleles at other sites. As a result, any mutation is expected to be contingent on earlier mutations and the fate of this mutation depends on the evolutionary history thereof.
The technology disclosed can also use convolutional kernels with different sizes to process protein sites including the amino acid of interest as well as neighboring amino acids in the protein sites. For example, instead of only analyzing a single amino acid, a kernel size covering multiple amino acids in a protein site of interest can better reflect how similar this protein site from another species is to human and how far mutation should be considered in the calculation of pathogenicity of a mutation in human.
The disclosed weighted summary statistics are referred to herein as “weighted evolutionary conservation metrics” or “weighted evolutionary conservation measures” or “weighted evolutionary profiles” or “weighted evolutionary conservation summary statistics” or “weighted conservation summary statistics” or “differentially-weighted summary statistics” or “weighted summary statistics.” Weighted position-specific scoring matrices (PSSMs) and weighted position-specific frequency matrices (PSFMs) are examples of the disclosed weighted summary statistics.
Implementations of the technology disclosed use the term “residue” and “amino acid” interchangeably. A person skilled in the art will appreciate that the technology disclosed can be analogously applied to other genomic/biological/chemical units and/or accumulations and/or combinations thereof, such as nucleotides, atoms, peptides, and so on. Similarly, the genomic/biological/chemical units can originate from a variety of natural and synthetic resources, such as DNA, RNA, proteins, and so on.
A person skilled in the art will also appreciate that the technology disclosed can be analogously applied to any biomolecule. As used herein, a “biomolecule” includes at least one of a biopolymer, nucleoside, nucleic acid, polynucleotide, oligonucleotide, protein, enzyme, polypeptide, antibody, antigen, ligand, receptor, polysaccharide, carbohydrate, polyphosphate, cell, tissue, organism, or fragment thereof or any other biologically active chemical compound(s) such as analogs or mimetics of the aforementioned species.
A sequence-to-weight mapping 102 assigns respective weights 118 to respective sequences 120 in a multiple sequence alignment (MSA) 112. Multiple sequence alignment (MSA) is a method for grouping a series of related proteins. MSA 112 arranges proteins in a matrix whose rows are individual protein sequences and whose columns contain amino acids that either come from the same position in some ancestral sequence (homologous) or play a common structural or functional role.
The weighting logic 130 has access to the sequence-to-weight mapping 102. The weighting logic 130 is configured to determine a weighted summary statistic 140. The weighted summary statistic 140 can be considered a weighted evolutionary conservation summary statistic in which the evolutionary conservation scores or frequencies are differentially weighted according to weights assigned to underlying sequences by the sequence-to-weight mapping 102. Determination of weights is discussed below.
In one implementation, the weighted summary statistic 140 is determined for a given residue category from residue categories 150 (e.g., twenty-one amino acid categories (including stop or gap amino acid category)) at a given position from positions 114 in the multiple sequence alignment (MSA) 112. In such an implementation, the weighted summary statistic 140 is based on one or more weights in the respective weights 118 of one or more sequences in the respective sequences 120 that have a residue of the given residue category at the given position. In some implementations, weights are initially assigned to be equal for all sequences within the alignment, and each respective weight is differentiable and learned during model training for system 100.
In some implementations, a phenotyping logic 170 is configured to generate the phenotype prediction 180 for the given position based on the weighted summary statistic 140. In some implementations, the phenotyping logic 170 can be a variant pathogenicity prediction logic (e.g., PrimateAI, PrimateAI 2D, PrimateAI 3D, JigsawAI). In other implementations, the phenotyping logic 170 can be a protein contact map generation logic. In yet other implementations, the phenotyping logic 170 can be a protein functionality prediction logic.
The uni-weighted summary statistics 300 are a sum of N constant weights respectively assigned to the N sequences by a sequence-to-weight mapping 320. The sum is calculated using the species proximity-agnostic uniform weighting logic 330. Using amino acids in protein sequences as an example, let Σ={A, . . . , W, -} be the set of all 20 amino acids A, . . . , W plus a gap token denoted as “-”. M is an ISI-way multiple sequence alignment S (|S|=7 in this example) and Mis is the amino acid at protein position i in sequence s∈S.
The evolutionary profile of an amino acid A at protein position i is defined as fiA=Σs w[Mis=A], where [ ] is the Iverson bracket and
is a normalization constant. Thus, all values are divided by 7 in this example. We interpret f as a signal for phenotype prediction. A constant w implies that variants from all species and sequences are equally important contributors and thus all have a weight of 1/7. For example, a variant in chimp (closely related to human) increases the frequency of an amino acid by the same amount as the same variant in Zebrafish (distant to human).
In the implementation illustrated in
Next, a species proximity-aware differential weighting logic 430 is discussed. We disclose a new weighted evolutionary profile
where ws is the weight of each sequence. As shown in
Per the sequence-to-weight mapping 420 in
and
Since there are 7 sequences in the MSA 210, the denominator is left as 7 throughout the figures for clarity.
In some implementations, instead of calculating each ws from the alignments themselves (e.g., via similarity to other sequences), each ws can be initialized to
before training. The parameters can be left differentiable during training. For example, this means that a model (e.g., PrimateAI, PrimateAI 2D, PrimateAI 3D, JigsawAI) can learn itself how important each of the sequences are as contributors to the phenotype signal of
The training process of the species proximity-aware differential weighting logic 430 will be described in detail in accordance with
The weighted summary statistics disclosed herein can be used to analyze sequences of residues including amino acids, nucleotides, carbohydrates, lipids, or other types of biological polymers. That is, the multiple sequence alignments (MSAs) for which the disclosed weighted summary statistics are generated can include amino acids, nucleotides, carbohydrates, lipids, or other types of biological polymers (e.g., DNA, RNA). Also, each residue can be a position/element in the sequences, which are arranged, for example, as multiple sequence alignments.
In another implementation, a multiple sequence alignment (MSA) aligns a query sequence with a plurality of non-query sequences of a species group that is homologous to the first species. Weights assigned to non-query sequences in the plurality of non-query sequences can be proportional to a degree of homology between the first species and species in the species group. The degree of homology can be measured by i) an evolutionary distance between the first species and the species in the species group, and/or ii) a number of mismatches between the query sequence and the non-query sequences, and/or iii) chemical similarity of the residues.
Homology, in biology, refers to the similarity of the structure, physiology, or development of different species of organisms based upon their descent from a common evolutionary ancestor. Homology is contrasted with “analogy,” which is a functional similarity of structure-based not upon common evolutionary origins but upon similar environmental benefit and functional usage. Thus, the forelimbs of such widely differing mammals as humans, bats, and deer are homologous; the form of construction and the number of bones in these varying limbs are practically identical and represent adaptive modifications of the forelimb structure of their common early mammalian ancestors. Analogous structures, on the other hand, can be represented by the wings of birds and of insects; the structures are used for flight in both types of organisms, but they have no common ancestral origin at the beginning of their evolutionary development.
The query sequence can belong to a human. The non-query sequences can belong to other humans and/or non-human primates. Those species in the species group that are evolutionarily proximate to the first species can be assigned greater weights than those species in the species group that are evolutionarily distant from the first species. The greater weights can cause the species in the species group that are evolutionarily proximate to the first species to contribute more respectively to the weighted summary statistic than the species in the species group that are evolutionarily distant from the first species.
In other implementations, the query sequence can be assigned a maximum weight. In some implementations, the phenotyping logic 170 (
When MSAs are input to the weighting logic, they can be one-hot encoded.
As another example, the multiz100 alignments cover the entire human proteome, as they come from a whole-genome DNA-level MSA. The MSA can be three billion positions long, because there are three billion DNA positions in the human genome.
The weighting logic can process the three-dimensional one-hot encoded MSA and generate two-dimensional weighted evolutionary profiles as output. As illustrated in
The parameters of the weighting logic 704 including species-specific weights, and parameters of the phenotyping logic 708 can be optimized during training via backpropagation. During training, the weighting logic 704 and phenotyping logic 708 gradually combines simpler features into complex features so that the most suitable hierarchical representations can be learned from training samples. The forward pass sequentially computes the output and propagates the function signals forward through the weighting logic 704 and phenotyping logic 708. In the final output layer, an objective loss function measures error between the estimated outputs and the given labels of the training samples. To minimize the training error, the backward pass uses the chain rule to backpropagate error signals and compute gradients with respect to all parameters throughout the weighting logic 704 and phenotyping logic 708. Finally, the parameters are updated using optimization algorithms based on stochastic gradient descent. Whereas batch gradient descent performs parameter updates for each complete dataset, stochastic gradient descent provides stochastic approximations by performing the updates for each small set of data examples. Several optimization algorithms stem from stochastic gradient descent. For example, the Adagrad and Adam training algorithms perform stochastic gradient descent while adaptively modifying learning rates based on update frequency and moments of the gradients for each parameter, respectively.
The secondary structure and solvent accessibility sub-models 850 and 860 are pre-trained on known protein crystal structures for the Protein DataBank, and allow backpropagation during the pathogenicity model training. The pretrained secondary structure sub-model 850 is used to generate secondary structure predictions of the reference and alternative sequences, namely, 3-state secondary structures of alpha helix (H), beta sheet (B), and coils (C). The pretrained solvent accessibility sub-model 860 is used to generate solvent accessibility predictions of the reference and alternative sequences, namely, 3-state solvent accessibility: buried (B), intermediate (I), and exposed (E). For each amino acid residue within the reference and alternative sequences, the secondary structure and solvent accessibility sub-models predict one state of secondary structure and one state of solvent accessibility, respectively.
The reference sequence 802 and alternative sequence 804 are passed through 1D convolution layers 806 and 808, respectively, to generate respective feature map. Similarly, the weighted primate, mammal, and vertebrate evolutionary profiles 514, 524, and 534 are also passed through 1D convolution layers 810, 812 and 814, respectively, to generate respective feature maps. The feature map of the reference sequence 802 is merged with the three types of weighted evolutionary profile feature maps, followed by batch normalization with the activation of ReLU and passing through 1D convolution (see 816). The feature map of the alternative sequence 804 is merged with the three types of weighted evolutionary profile feature maps, followed by batch normalization with the activation of ReLU and passing through 1D convolution (see 818). The merged feature map corresponding to the reference sequence 802 and the merged feature map corresponding to the alternative sequence 804 are subsequently concatenated with the reference sequence secondary structure prediction 820, the alternative sequence secondary structure prediction 822, the reference sequence solvent accessibility prediction 824 and the alternative sequence solvent accessibility prediction 826 (see 828). The concatenated features are passed through 1D convolution 830 and groups of residual blocks with convolution filters having the same convolution window size, batch normalization and ReLU layers (see 832 and 834). In one embodiment, the last three residual blocks have an atrous rate of 2 for the 1D convolutions, to give higher coverage for the kernels. The output generated from the groups of residual blocks is passed through 1D convolution and activation sigmoid 836, followed by a global maxpooling 838 and sigmoid 840. The generated output represents the pathogenicity prediction 842 of the variant (i.e., an example of phenotype prediction 180).
Additional details about various embodiments of the PrimateAI model can be found in Sundaram, L. et al. Predicting the clinical impact of human mutation with deep neural networks. Nat. Genet. 50, 1161-1170 (2018); US Patent Application Nos. 62/573,144; 62/573,149; 62/573,153; 62/582,898; 16/160,903; 16/160,986; 16/160,968; and Ser. No. 16/407,149, which are incorporated herein by reference as if fully set forth herein. Further particular implementations of the disclosed system and inputs for the PrimateAI model are described further below under ‘Particular Implementations’.
Another one of the inputs can be distance channels 902. In one implementation, three-dimensional protein structures are represented using voxels in a voxel grid. The voxels in the voxel grid have a plurality of distance channels (e.g., twenty-one distance channels for the twenty-one amino acid categories, respectively (including stop or gap amino acid category)). The distance channels 902 can be independently generated for each of the twenty-one amino acid categories. Consider, for example, the Alanine (A) amino acid category. Further consider, for example, that the voxel grid is of size 3×3×3 and has twenty-seven voxels. Then, in one implementation, an Alanine distance channel includes twenty-seven distance values for the twenty-seven voxels in the voxel grid, respectively. The twenty-seven distance values in the Alanine distance channel are measured from respective centers of the twenty-seven voxels in the voxel grid to respective nearest atoms in the Alanine amino acid category. In other implementations, the distance channels 902 are generated based on distances between voxel centers and atoms across a plurality of atomic elements irrespective of amino acids.
In some implementations, the distances are normalized by a maximum scan radius to generate normalized distances. In another implementation, the distance channels 902 are generated based on distances between voxel centers and alpha-carbon atoms on an amino acid-basis. In some implementations, the distances are normalized by the maximum scan radius to generate normalized distances. In yet another implementation, the distance channels 902 are generated based on distances between voxel centers and beta-carbon atoms on an amino acid-basis. In some implementations, the distances are normalized by the maximum scan radius to generate normalized distances. In yet another implementation, the distance channels 902 are generated based on distances between voxel centers and side chain atoms on an amino acid-basis. In some implementations, the distances are normalized by the maximum scan radius to generate normalized distances. In yet another implementation, the distance channels 902 are generated based on distances between voxel centers and backbone atoms on an amino acid-basis. In some implementations, the distances are normalized by the maximum scan radius to generate normalized distances. In yet another implementation, the distance channels 902 are generated based on distances (one feature) between voxel centers and the respective nearest atoms irrespective of atom type and amino acid type. In yet another implementation, the distance channels are generated based on distances (one feature) between voxel centers and atoms from non-standard amino acids. In some implementations, the distances between the voxels and the atoms are calculated based on polar coordinates of the voxels and the atoms. The polar coordinates are parameterized by angles between the voxels and the atoms. In one implementation, this angel information is used to generate an angle channel for the voxels (i.e., independent of the distance channels). In some implementations, angles between a nearest atom and neighboring atoms (e.g., backbone atoms) can be used as features that are encoded with the voxels.
Another one of the inputs can be a feature 904 indicating missing atoms within a specified radius.
Another one of the inputs can be one-hot encoding 906 of the reference amino acid. Another one of the inputs can be one-hot encoding 908 of the variant/alternative amino acid.
Another one of the inputs can be a feature 910 indicating missing residue or missing evolutionary profile.
Another one of the inputs can be annotations channels 912 generated by an annotations generator. In one implementation, the annotations channels 912 can be generated based on molecular processing annotations. In another implementation, the annotations channels 912 can be generated based on regions annotations. In yet another implementation, the annotations channels 912 can be generated based on sites annotations. In yet another implementation, the annotations channels 912 can be generated based on amino acid modifications annotations. In yet another implementation, the annotations channels 912 can be generated based on secondary structure annotations. In yet another implementation, the annotations channels 912 can be generated based on experimental information annotations.
Another one of the inputs can be structure confidence channels 914 generated by a structure confidence generator. In one implementation, the structure confidence 914 can be generated based on global model quality estimations (GMQEs). In another implementation, the structure confidence 914 can be generated based on qualitative model energy analysis (QMEAN) scores. In yet another implementation, the structure confidence channels 914 can be generated based on template modeling scores. Examples of the template modeling scores can include minimum template modeling scores, mean template modeling scores, and maximum template modeling scores.
A person skilled in the art will appreciate that any permutation and combination of the input channels can be concatenated into an input for processing through the pathogenicity classifier 920 for the pathogenicity prediction 930 of the target variant. In some implementations, only a subset of the input channels may be concatenated. The input channels can be concatenated in any order. In one implementation, the input channels can be concatenated into a single tensor by a tensor generator (input encoder). This single tensor can then be provided as input to the pathogenicity classifier 920 for the pathogenicity prediction 930 of the target variant.
Additional details about the PrimateAI 3D model can be found in U.S. patent application Ser. No. 17/232,056; 63/175,495; 63/175,767; and Ser. No. 17/468,411, which are incorporated herein by reference as if fully set forth herein.
Another input to the pathogenicity classifier 1106 is gapped spatial representation 1102 of a target protein. The “gaped spatial representation” of a protein is such a spatial representation of the protein that excludes at least one gap amino acid in the protein. In one implementation, a gap amino acid is excluded by excluding (or not considering or ignoring) one or more atoms or atom-types of the gap amino acid when generating the gaped spatial representation. For example, the atoms of the gap amino acid can be excluded from the calculations (or selections or computations) that produce the distance channels, the evolutionary profiles, the annotation channels, and/or the structure confidence channels. In other implementations, the gaped spatial representation can be generated by excluding the gap amino acid from other feature channels as well.
The spatial configurations of non-gap amino acids can be encoded as amino acid class-wise distance channels. Each of the amino acid class-wise distance channels has voxel-wise distance values for voxels in a plurality of voxels. The voxel-wise distance values specify distances from corresponding voxels in the plurality of voxels to atoms of the non-gap amino acids. The spatial configurations of the non-gap amino acids are determined based on spatial proximity between the corresponding voxels and the atoms of the non-gap amino acids. The spatial configuration of a gap amino acid is excluded from the gapped spatial representation by disregarding distances from the corresponding voxels to atoms of the gap amino acid when determining the voxel-wise distance values. The spatial configuration of the gap amino acid is excluded from the gaped spatial representation by disregarding spatial proximity between the corresponding voxels and the atoms of the gap amino acid.
Another input to the pathogenicity classifier 1106 is alternate residue representation 1104. An alternative residue (amino acid) can be generated by a nucleotide variant at a particular position. In some implementations, the alternate amino acid is an amino acid that is same as a reference amino acid. In other implementations, the alternate amino acid is an amino acid that is different from the reference amino acid. The representation of the alternate amino acid can be a voxelized one-hot encoding of the alternate amino acid. The voxelized dimensionality of the voxelized one-hot encoded alternate residue (amino acid) can be 21×1×1×1, where 21 denotes the twenty-one amino acid categories including 20 types of amino acids and 1 stop or gap amino acid category.
Additional details about the JigsawAI model can be found in 63/253,122; 63/281,579; 63/281,592; 17/533,091, which are incorporated herein by reference as if fully set forth herein.
Protein contact maps 1270 represent the distance between all possible amino acid residue pairs of a 3D protein structure using a binary two-dimensional matrix. For two residues i and j, for example, the ijth element of the matrix is 1 if the two residues are closer than a predetermined threshold, and 0 otherwise. Various contact definitions have been proposed—the distance between the Cα-Cα atom with threshold 6-12 Å; distance between Cβ-Cβ atoms with threshold 6-12 Å (Cα is used for Glycine); and distance between the side-chain centers of mass.
Protein contact maps provide a more reduced representation of a protein structure than its full 3D atomic coordinates. The advantage is that protein contact maps are invariant to rotations and translations, which makes them more easily predictable by machine learning methods. It has also been shown that under certain circumstances (e.g., low content of erroneously predicted contacts) it is possible to reconstruct the 3D coordinates of a protein using its protein contact map. Protein contact maps are also used for protein superimposition and to describe similarity between protein structures. They are either predicted from protein sequence or calculated from a given structure.
A protein contact map describes the pairwise spatial and functional relationship of amino acids (residues) in a protein and contains key information for protein 3D structure prediction. In some implementations, two residues of a protein are in contact if their Euclidean distance is <8 Å. The distance of two residues can be calculated using Cα or Cβ atoms, corresponding to Cα- or Cβ-based contacts. A protein contact map can also be considered a binary L×L matrix, where L is the protein length. In this matrix, an element with value 1 indicates the corresponding two residues are in contact; otherwise, they are not in contact.
Additional details about the PrimateAI 2D model can be found in U.S. Patent Application No. 63/229,897, which is incorporated herein by reference as if fully set forth herein.
In one implementation of the technology disclosed, the number of convolutional filters can be increased to generate multiple alternative frequency profiles from the same alignments. In another implementation, the kernel size can be increased to calculate frequencies of multiple protein sites at a time in tandem. In some implementations, activation functions can be added after each convolution and stack layers to introduce non-linearities between species.
In one implementation, the weighting logic 130 is communicably linked to the storage subsystem 1302 and the user interface input devices 1338.
User interface input devices 1338 can include a keyboard; pointing devices such as a mouse, trackball, touchpad, or graphics tablet; a scanner; a touch screen incorporated into the display; audio input devices such as voice recognition systems and microphones; and other types of input devices. In general, use of the term “input device” is intended to include all possible types of devices and ways to input information into computer system 1300.
User interface output devices 1356 can include a display subsystem, a printer, a fax machine, or non-visual displays such as audio output devices. The display subsystem can include an LED display, a cathode ray tube (CRT), a flat-panel device such as a liquid crystal display (LCD), a projection device, or some other mechanism for creating a visible image. The display subsystem can also provide a non-visual display such as audio output devices. In general, use of the term “output device” is intended to include all possible types of devices and ways to output information from computer system 1300 to the user or to another machine or computer system.
Storage subsystem 1302 stores programming and data constructs that provide the functionality of some or all of the modules and methods described herein. These software modules are generally executed by processors 1358.
Processors 1358 can be graphics processing units (GPUs), field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), and/or coarse-grained reconfigurable architectures (CGRAs). Processors 1358 can be hosted by a deep learning cloud platform such as Google Cloud Platform™, Xilinx™, and Cirrascale™. Examples of processors 1078 include Google's Tensor Processing Unit (TPU)™, rackmount solutions like GX4 Rackmount Series™, GX10 Rackmount Series™, NVIDIA DGX-1™, Microsoft' Stratix V FPGA™, Graphcore's Intelligent Processor Unit (IPU)™, Qualcomm's Zeroth Platform™ with Snapdragon Processors™, NVIDIA's Volta™, NVIDIA's DRIVE PX™, NVIDIA's JETSON TX1/TX2 MODULE™, Intel's Nirvana™, Movidius VPU™, Fujitsu DPI™, ARM's DynamicIQ™, IBM TrueNorth™, Lambda GPU Server with Testa V100s™, and others.
Memory subsystem 1312 used in the storage subsystem 1302 can include a number of memories including a main random access memory (RAM) 1332 for storage of instructions and data during program execution and a read only memory (ROM) 1334 in which fixed instructions are stored. A file storage subsystem 1336 can provide persistent storage for program and data files, and can include a hard disk drive, a floppy disk drive along with associated removable media, a CD-ROM drive, an optical drive, or removable media cartridges. The modules implementing the functionality of certain implementations can be stored by file storage subsystem 1336 in the storage subsystem 1302, or in other machines accessible by the processor.
Bus subsystem 1342 provides a mechanism for letting the various components and subsystems of computer system 1300 communicate with each other as intended. Although bus subsystem 1342 is shown schematically as a single bus, alternative implementations of the bus subsystem can use multiple busses.
Computer system 1300 itself can be of varying types including a personal computer, a portable computer, a workstation, a computer terminal, a network computer, a television, a mainframe, a server farm, a widely-distributed set of loosely networked computers, or any other data processing system or user device. Due to the ever-changing nature of computers and networks, the description of computer system 1300 depicted in
The technology disclosed can be practiced as a system, method, or article of manufacture. One or more features of an implementation can be combined with the base implementation. Implementations that are not mutually exclusive are taught to be combinable. One or more features of an implementation can be combined with other implementations. This disclosure periodically reminds the user of these options. Omission from some implementations of recitations that repeat these options should not be taken as limiting the combinations taught in the preceding sections—these recitations are hereby incorporated forward by reference into each of the following implementations.
One or more implementations and features of the technology disclosed or elements thereof can be implemented in the form of a computer product, including a non-transitory computer readable storage medium with computer usable program code for performing the method steps indicated. Furthermore, one or more implementations and features of the technology disclosed or elements thereof can be implemented in the form of an apparatus including a memory and at least one processor that is coupled to the memory and operative to perform exemplary method steps. Yet further, in another aspect, one or more implementations and features of the technology disclosed or elements thereof can be implemented in the form of means for carrying out one or more of the method steps described herein; the means can include (i) hardware module(s), (ii) software module(s) executing on one or more hardware processors, or (iii) a combination of hardware and software modules; any of (i)-(iii) implement the specific techniques set forth herein, and the software modules are stored in a computer readable storage medium (or multiple such media).
The features described in this section can be combined as features. In the interest of conciseness, the combinations of features are not individually enumerated and are not repeated with each base set of features. The reader will understand how features identified in the features described in this section can readily be combined with sets of base features identified as implementations in other sections of this application. These features are not meant to be mutually exclusive, exhaustive, or restrictive; and the technology disclosed is not limited to these features but rather encompasses all possible combinations, modifications, and variations within the scope of the claimed technology and its equivalents.
Other implementations of the features described in this section can include a non-transitory computer readable storage medium storing instructions executable by a processor to perform any of the features described in this section. Yet another implementation of the features described in this section can include a system including memory and one or more processors operable to execute instructions, stored in the memory, to perform any of the features described in this section.
In one implementation, the technology disclosed describes a system. The system comprises a memory that stores a sequence-to-weight mapping. The sequence-to-weight mapping assigns respective weights to respective sequences in a multiple sequence alignment. The multiple sequence alignment can align a hundred to five hundred sequences, in some implementations, and even more sequences in other implementations.
The system further comprises a weighting logic. The weighting logic has access to the sequence-to-weight mapping. The weighting logic is configured to determine a weighted summary statistic for a given residue category at a given position in the multiple sequence alignment based on one or more weights of one or more sequences in the multiple sequence alignment that have a residue of the given residue category at the given position.
The system further comprises a phenotyping logic. The phenotyping logic is configured to generate a phenotype prediction for the given position based on the weighted summary statistic.
In one implementation, a single sequence in the multiple sequence alignment has the residue of the given residue category at the given position. In such an implementation, the weighted summary statistic is a single weight assigned to the single sequence.
In another implementation, N sequences in the multiple sequence alignment have the residue of the given residue category at the given position. In such an implementation, the weighted summary statistic is a sum of N weights respectively assigned to the N sequences by the sequence-to-weight mapping.
In some implementations, the weighted summary statistic can be a weighted evolutionary profile, for example, a weighted position-specific scoring matrix (wPSSM), or a weighted position-specific frequency matrix (wPSFM). In other implementations, the weighted summary statistic can be a weighted position-specific substitution or conservation frequency. In yet other implementations, the weighted summary statistic can be a weighted position-specific substitution or conservation score.
In some implementations, the multiple sequence alignment aligns a query sequence of a first species with a plurality of non-query sequences of a species group that is homologous to the first species. Weights assigned to non-query sequences in the plurality of non-query sequences can be proportional to a degree of homology between the first species and species in the species group. In one implementation, the degree of homology can be measured by an evolutionary distance between the first species and the species in the species group. In another implementation, the degree of homology can be measured by a number of mismatches between the query sequence and the non-query sequences.
In some implementations, the query sequence belongs to a human, and the non-query sequences belong to non-human primates. Those species in the species group that are evolutionarily proximate to the first species are assigned greater weights than those species in the species group that are evolutionarily distant from the first species. The greater weights cause the species in the species group that are evolutionarily proximate to the first species to contribute more to the weighted summary statistic than the species in the species group that are evolutionarily distant from the first species.
In other implementations, the query sequence is assigned a maximum weight (e.g., 1).
In some implementations, the phenotyping logic can be used as a proxy for a weighted contribution of a non-query sequence to an evolutionary constraint of a variant residue in the query sequence.
As discussed above, the sequence-to-weight mapping assigns the respective weights to the respective sequences in the multiple sequence alignment. In some implementations, the respective weights can be differentiable and learned during a training of the phenotyping logic. The respective weights can be implemented as part of respective convolution filters that are assigned to the respective sequences and are learned during the training of the phenotyping logic.
In some implementations, the sequence-to-weight mapping can assign a plurality of weight sets to a given sequence in the multiple sequence alignment. Weight sets in the plurality of weight sets can be separately applied to generate a plurality of weighted summary statistics for the given sequence. The plurality of weight sets can be implemented as part of a plurality of convolution filters that is assigned to the given sequence and is learned during the training of the phenotyping logic. In some implementations, convolution filters in the plurality of convolution filters can be applied as a convolutional layer, for example, with or without a bias term or an activation function. The convolution filters can also be interspersed with activation functions, and stacked across multiple convolution layers.
In yet other implementations, an increase in a kernel size of a convolution filter in the plurality of convolution filters can cause a determination of multiple weighted summary statistics for multiple positions in the multiple sequence alignment. In yet further implementations, the increase can allow for a determination of multiple weighted summary statistics from a plurality of sequences in the multiple sequence alignment in parallel.
In some implementations, the application of a non-linear activation function following the convolution layer can introduce non-linear relationships between the non-query sequences in the multiple sequence alignment. In some implementations, the application of a plurality of stacked convolution layers in parallel can introduce non-linear relationships between the non-query sequences.
In some implementations, the phenotyping logic can be a variant pathogenicity prediction logic. In other implementations, the phenotyping logic can be a protein contact map generation logic. In yet other implementations, the phenotyping logic can be a protein functionality prediction logic.
In another implementation, the technology disclosed describes a system. The system comprises a memory that stores a sequence-to-weight mapping. The sequence-to-weight mapping assigns respective weights to respective sequences in a multiple sequence alignment. The multiple sequence alignment can align a hundred to five hundred sequences, in some implementations, and even more sequences in other implementations.
The system further comprises a weighting logic. The weighting logic has access to the sequence-to-weight mapping. The weighting logic is configured to determine a weighted summary statistic for a given nucleotide category at a given position in the multiple sequence alignment based on one or more weights of one or more sequences in the multiple sequence alignment that have a nucleotide of the given nucleotide category at the given position.
The system further comprises a phenotyping logic. The phenotyping logic is configured to generate a phenotype prediction for the given position based on the weighted summary statistic.
This application claims priority to or the benefit the following applications: U.S. Provisional Patent Application No. 63/294,830, titled “Species-differentiable Evolutionary Profiles,” by Tobias H A M P and Kai-How F A R H, filed Dec. 29, 2021 (Attorney Docket No. ILLM 1068-1/IP-2302-PRV).U.S. Provisional Patent Application No. 62/294,828, titled “Inter-Model Prediction Score Recalibration,” by Tobias H A M P and Kai-How F A R H, filed Dec. 29, 2021 (Attorney Docket No. ILLM 1067-1/IP-2301-PRV);U.S. Provisional Patent Application No. 63/294,827, titled “Deep Learning Network for Evolutionary Conservation,” by Sabrina RASHID and Kai-How F A R H, filed Dec. 29, 2021 (Attorney Docket No. ILLM 1066-1/IP-2299-PRV);U.S. Provisional Patent Application No. 63/294,820, titled “Identifying Genes With Differential Selective Constraint Between Humans And Non-human Primates,” by Hong GAO, Joshua Goodwin Jon MCMASTER-SCHRAIBER, Kai-How F A R H, filed Dec. 29, 2021 (Attorney Docket No. ILLM 1065-1/IP-2298-PRV);U.S. Provisional Patent Application No. 63/294,816, titled “Classifying Millions of Variants of Uncertain Significance Using Primate Sequencing And Deep Learning,” by Hong GAO, Tobias H A M P, Joshua Goodwin Jon MCMASTER-SCHRAIBER, Laksshman SUNDARAM, Kai-How F A R H, filed Dec. 29, 2021 (Attorney Docket No. ILLM 1064-1/IP-2297-PRV); andU.S. Provisional Patent Application No. 63/294,813, titled “Periodic Mask Pattern for Revelation Language Models,” by Tobias H A M P, Anastasia Susanna Dagmar DIETRICH, Yibing W U, Jeffrey Mark E D E, Kai-How F A R H, filed Dec. 29, 2021 (Attorney Docket No. ILLM 1063-1/IP-2296-PRV); The priority provisional applications are hereby incorporated by reference for all purposes.
| Number | Date | Country | |
|---|---|---|---|
| 63294830 | Dec 2021 | US | |
| 63294828 | Dec 2021 | US | |
| 63294827 | Dec 2021 | US | |
| 63294820 | Dec 2021 | US | |
| 63294816 | Dec 2021 | US | |
| 63294813 | Dec 2021 | US |
