Patent Application
20150347672
The subject matter described herein mainly relates to bioinformatics, genomic processing arts, proteomic processing arts, and related arts.
Genomic and proteomic analyses have substantial realized and potential promise for clinical application in medical fields such as oncology, where various cancers are known to be associated with specific combinations of genomic mutations/variations and/or high or low expression levels for specific genes, which play a role in growth and evolution of cancer, e.g. cell proliferation and metastasis. For example, the Wnt signaling pathway affects regulation of cell proliferation, and is highly regulated. High Wnt pathway activity due to loss of regulation has been correlated to cancer, among which with malignant colon tumors. While not being limited to any particular theory of operation, it is believed that deregulation of the Wnt pathway in malignant colon cells leads to high Wnt pathway activity that in turn causes cell proliferation of the malignant colon cells, i.e. spread of colon cancer. On the other hand, abnormally low pathway activity might also be of interest, for example in the case of osteoporosis.
Technologies for acquiring genomic and proteomic data have become readily available in clinical settings. For example, measurements by microarrays are routinely employed to assess gene expression levels, protein levels, methylation, and so forth. Automated gene sequencing enables cost-effective identification of genetic variations in DNA and mRNA. Quantitative assessment of mRNA levels during gene sequencing holds promise as yet another clinical tool for assessing gene expression levels.
In spite of (or, perhaps, because of) these advances, clinical application of genomic and proteomic analyses faces a substantial hurdle—data overload. For example, the number of identifiable mutations in a single clinical sample can number in the hundreds of thousands or more. Most of these mutations are so called bystander mutations without specific contribution to cancer growth, and only a few do contribute to cancer growth and functional evolution, and these present the targets for effective treatment. A single microarray can generate gene expression levels for tens of thousands of genes. Processing these large quantities of data to identify clinically useful information, like for example in the application of choosing the right therapy, is difficult.
One approach is to limit the analysis to a few canonical or standardized tests, such as tests approved by the U.S. Food and Drug Administration (FDA). In such an approach, a specific indicator or combination of indicators (e.g., mutations and/or specified high or low gene expression levels) is detected in order to test “positive” for the indicated disease condition (e.g., a particular type of cancer). The canonical test is supported by clinical studies that have shown strong correlation with the disease condition or with treatment efficacy. This approach is useful only for those clinical conditions for which a canonical test has been developed, e.g. specific diagnosis of a disease, or predicting response to a drug in a specific cancer type at a specific stage, and is also rigid as it is only applicable for the canonical conditions.
Another approach is based on identification of functionally related groups of genomic or proteomic indicators. For example, the Wnt pathway comprises a cascade of proteomic reactions. Major components of this chain include (but are not limited to) binding of the Wnt signaling protein to a frizzled surface receptor of the cell which causes activation of proteins of the disheveled family of proteins which in turn impact the level of transcription agents such as β-catenin/TCF4 based protein complexes in the cell nucleus. These transcription agents, in turn, control transcription of target mRNA molecules that in turn are translated into target proteins of the Wnt pathway. Clinical studies have shown some correlations between regulatory proteins of the Wnt pathway and the activity of the Wnt pathway.
However, applying such clinical study results to the diagnosis and clinical evaluation of a specific patient is difficult due to the complexity of signaling pathways, e.g. the Wnt pathway. As a simple example, measurement of the expression level of a protein that is “upstream” in the Wnt pathway may fail to detect abnormal behavior of a protein that is “downstream” in the Wnt pathway. It is believed that the Wnt pathway includes numerous feedback mechanisms and the simplified concept of “upstream” and “downstream” may be inapplicable for a substantial portion of the Wnt pathway; more generally, abnormal behavior in one portion of the protein cascade comprising the Wnt pathway may have more or less effect on other portions of the protein cascade, and on the activity of the Wnt pathway as a whole. Still further, in some clinical studies protein expression levels for regulatory proteins of the signaling cascade are assessed by measuring mRNA expression levels of the genes that encode for the regulatory proteins.
This is an indirect measurement that may not accurately assess the regulatory protein expression level, and hardly ever reflects the amount of active proteins (after a specific post-translational modification like phosphorylation).
The main problem underlying the present invention was thus to provide suitable methods and means for performing genomic and, respectively, proteomic analyses. Specific aspects of the underlying problem as well as further objections in connection with the present invention become apparent when studying the description, the examples provided herein and, in particular, when studying the attached claims.
The present invention provides new and improved methods and apparatuses as disclosed herein.
In accordance with a main aspect of the present invention, the above problem is solved by a specific method for assessing cellular signaling pathway activity using linear combination(s) of target gene expressions, namely a method comprising:
wherein the inferring is performed by a digital processing device using the model of the cellular signaling pathway.
The medical subject may be a human or an animal. Moreover, the “target gene(s)” may be “direct target genes” and/or “indirect target genes” (as described herein).
Preferred is a method wherein for each of the one or more target gene(s) one or more expression level(s) measured in the extracted sample of the tissue and/or cells of the medical subject are provided, and wherein the one or more linear combination(s) comprise a linear combination of all expression levels of the one or more expression level(s) provided for the one or more target gene(s).
Also preferred is a method wherein for each of the one or more target gene(s) one or more expression level(s) measured in the extracted sample of the tissue and/or cells of the medical subject are provided, and wherein the one or more linear combination(s) comprise a linear combination including for each of the one or more target gene(s) a weighted term, each weighted term being based on only one expression level of the one or more expression level(s) provided for the respective target gene.
Also preferred is a method wherein for each of the one or more target gene(s) one or more expression level(s) measured in the extracted sample of the tissue and/or cells of the medical subject are provided, wherein the one or more linear combination(s) comprise for each of the one or more target gene(s) a first linear combination of all expression levels of the one or more expression level(s) provided for the respective target gene, and wherein the model is further based at least in part on a further linear combination including for each of the one or more target gene(s) a weighted term, each weighted term being based on the first linear combination for the respective target gene.
The cellular signaling pathway may be a Wnt pathway, an ER (Estrogen Receptor) pathway, an AR (Androgen Receptor) pathway or an HH (Hedgehog) pathway.
Thus, according to a preferred embodiment the cellular signaling pathway comprises a Wnt pathway, an ER pathway, an AR pathway or an HH pathway.
Particularly suitable target genes are described in the following text passages as well as the examples below (see e.g. Tables 1-9). Thus, according to a preferred embodiment the target gene(s) is/are selected from the group comprising or consisting of target genes listed in Table 1 or Table 6 (for Wnt pathway), target genes listed in Table 2, Table 5 or Table 7 (for ER pathway), target genes listed in Table 3 or Table 8 (for HH pathway) and target genes listed in Table 4 or Table 9 (for AR pathway).
Particularly preferred is a method wherein the inferring comprises:
1inferring activity of a Wnt pathway in the tissue and/or cells of the medical subject based at least on expression levels of one or more, preferably at least three, target gene(s) of the Wnt pathway measured in the extracted sample of the tissue and/or cells of the medical subject selected from the group comprising or consisting of: KIAA1199, AXIN2, RNF43, TBX3, TDGF1, SOX9, ASCL2, IL8, SP5, ZNRF3, KLF6, CCND1, DEFA6 and FZD7.
Further preferred is a method, wherein the inferring is further based on expression levels of at least one target gene of the Wnt pathway measured in the extracted sample of the tissue and/or cells of the medical subject selected from the group comprising or consisting of: NKD1, OAT, FAT1, LEF1, GLUL, REG1B, TCF7L2, COL18A1, BMP7, SLC1A2, ADRA2C, PPARG, DKK1, HNF1A and LECT2.
Particularly preferred is a method wherein the inferring comprises:
Further preferred is a method, wherein the inferring is further based on expression levels of at least one target gene of the ER pathway measured in the extracted sample of the tissue and/or cells of the medical subject selected from the group comprising or consisting of: AP1B1, ATP5J, COL18A1, COX7A2L, EBAG9, ESR1, HSPB1, IGFBP4, KRT19, MYC, NDUFV3, PISD, PRDM15, PTMA, RARA, SOD1 and TRIM25.
A method wherein the inferring comprises
Further preferred is a method, wherein the inferring is further based on expression levels of at least one target gene of the HH pathway measured in the extracted sample of the tissue and/or cells of the medical subject selected from the group comprising or consisting of: BCL2, FOXA2, FOXF1, H19, HHIP, IL1R2, JAG2, JUP, MIF, MYLK, NKX2.2, NKX2.8, PITRM1 and TOM1.
A method wherein the inferring comprises
Further preferred is a method, wherein the inferring is further based on expression levels of at least one target gene of the AR pathway measured in the extracted sample of the tissue and/or cells of the medical subject selected from the group comprising or consisting of: APP, NTS, PLAU, CDKN1A, DRG1, FGF8, IGF1, PRKACB, PTPN1, SGK1 and TACC2.
Another aspect of the present invention relates to a method (as described herein), further comprising:
The present invention also relates to a method (as described herein) comprising:
and/or
and/or
and/or
Preferably, the set of target genes of the Wnt pathway includes at least nine, preferably all target genes selected from the group comprising or consisting of: KIAA1199, AXIN2, RNF43, TBX3, TDGF1, SOX9, ASCL2, IL8, SP5, ZNRF3, KLF6, CCND1, DEFA6 and FZD7,
and/or
the set of target genes of the ER pathway includes at least nine, preferably all target genes selected from the group comprising or consisting of: CDH26, SGK3, PGR, GREB1, CA12, XBP1, CELSR2, WISP2, DSCAM, ERBB2, CTSD, TFF1 and NRIP1,
and/or
the set of target genes of the HH pathway includes at least nine, preferably all target genes selected from the group comprising or consisting of: GLI1, PTCH1, PTCH2, IGFBP6, SPP1, CCND2, FST, FOXL1, CFLAR, TSC22D1, RAB34, S100A9, S100A7, MYCN, FOXM1, GLI3, TCEA2, FYN and CTSL1,
and/or
the set of target genes of the AR pathway includes at least nine, preferably all target genes selected from the group comprising or consisting of: KLK2, PMEPA1, TMPRSS2, NKX3—1, ABCC4, KLK3, FKBP5, ELL2, UGT2B15, DHCR24, PPAP2A, NDRG1, LRIG1, CREB3L4, LCP1, GUCY1A3, AR and EAF2.
A method, wherein
the set of target genes of the Wnt pathway further includes at least one target gene selected from the group comprising or consisting of: NKD1, OAT, FAT1, LEF1, GLUL, REG1B, TCF7L2, COL18A1, BMP7, SLC1A2, ADRA2C, PPARG, DKK1, HNF1A and LECT2,
and/or
the set of target genes of the ER pathway further includes at least one target gene selected from the group comprising or consisting of: AP1B1, ATP5J, COL18A1, COX7A2L, EBAG9, ESR1, HSPB1, IGFBP4, KRT19, MYC, NDUFV3, PISD, PRDM15, PTMA, RARA, SOD1 and TRIM25,
and/or
the set of target genes of the HH pathway further includes at least one target gene selected from the group comprising or consisting of: BCL2, FOXA2, FOXF1, H19, HHIP, IL1R2, JAG2, JUP, MIF, MYLK, NKX2.2, NKX2.8, PITRM1 and TOM1,
and/or
the set of target genes of the AR pathway further includes at least one target gene selected from the group comprising or consisting of: APP, NTS, PLAU, CDKN1A, DRG1, FGF8, IGF1, PRKACB, PTPN1, SGK1 and TACC2, is particularly preferred.
The sample(s) to be used in accordance with the present invention can be, e.g., a sample obtained from a breast lesion, or from a colon of a medical subject known or suspected of having colon cancer, or from a liver of a medical subject known or suspected of having liver cancer, or so forth, preferably via a biopsy procedure or other sample extraction procedure. The tissue of which a sample is extracted may also be metastatic tissue, e.g. (suspected) malignant tissue originating from the colon, breast, liver, or other organ that has spread outside of the colon, breast, liver, or other organ. The cells of which a sample is extracted may also be tumorous cells from hematologic malignancies (such as leukemia). In some cases, the cell sample may also be circulating tumor cells, that is, tumor cells that have entered the bloodstream and may be extracted as the extracted tissue sample using suitable isolation techniques. The term “extracted sample” as used herein also encompasses the case where tissue and/or cells of the medical subject have been taken from the medical subject and e.g. put on a microscope slide and where for performing the claimed method a portion of this sample is extracted, e.g. by means of Laser Capture Microdissection (LCM) or by scraping off the cells of interest from the slide.
The phrase “the cellular signaling pathway is operating abnormally” refers to the case where the “activity” of the pathway is not as expected, wherein the term “activity” may refer to the activity of the transcription factor complex in driving the target genes to expression, i.e. the speed by which the target genes are transcribed. Normal may be when it is inactive in tissue where it is expected to be inactive and active where it is expected to be active. Furthermore, there may be a certain level of activity that is considered normal, and anything higher or lower may be considered abnormal.
In accordance with another disclosed aspect, an apparatus comprises a digital processor configured to perform a method according to the invention as described herein.
In accordance with another disclosed aspect, a non-transitory storage medium stores instructions that are executable by a digital processing device to perform a method according to the invention as described herein. The non-transitory storage medium may be a computer-readable storage medium, such as a hard drive or other magnetic storage medium, an optical disk or other optical storage medium, a random access memory (RAM), read-only memory (ROM), flash memory, or other electronic storage medium, a network server, or so forth. The digital processing device may be a handheld device (e.g., a personal data assistant or smartphone), a notebook computer, a desktop computer, a tablet computer or device, a remote network server, or so forth.
In accordance with another disclosed aspect, a computer program comprises program code means for causing a digital processing device to perform a method according to the invention as described herein. The digital processing device may be a handheld device (e.g., a personal data assistant or smartphone), a notebook computer, a desktop computer, a tablet computer or device, a remote network server, or so forth.
One advantage resides in a clinical decision support (CDS) system providing clinical recommendations based on a mathematical analysis of one or more cellular signaling pathway(s), for example using a mathematical model of a Wnt pathway, an ER pathway, an AR pathway and/or an HH pathway.
Another advantage resides in an improved transparency of a mathematical model that is based at least in part on one or more linear combination(s).
Another advantage resides in providing a CDS system recommending targeted treatment for loss of regulation of a cellular signaling pathway.
Another advantage resides in providing a CDS system that is designed to detect loss of regulation for a particular cellular signaling pathway, such as a Wnt pathway, an ER pathway, an AR pathway or an HH pathway, and is readily adapted to provide recommendations for different types of cancer sourced by that particular cellular signaling pathway.
The present invention as described herein can, e.g., also advantageously be used in connection with
Further advantages will be apparent to those of ordinary skill in the art upon reading and understanding the attached figures, the following description and, in particular, upon reading the detailed examples provided herein below.
The following examples merely illustrate particularly preferred methods and selected aspects in connection therewith. The teaching provided therein may be used for constructing several tests and/or kits, e.g. to detect, predict and/or diagnose the abnormal activity of one or more cellular signaling pathways. Furthermore, upon using methods as described herein drug prescription can advantageously be guided, drug prediction and monitoring of drug efficacy (and/or adverse effects) can be made, drug resistance can be predicted and monitored, e.g. to select subsequent test(s) to be performed (like a companion diagnostic test). The following examples are not to be construed as limiting the scope of the present invention.
As disclosed herein, by constructing a mathematical model (e.g., the illustrative “two-layer” model shown in
The expression levels of the target genes are preferably measurements of the level of mRNA, which can be the result of e.g. (RT)-PCR and microarray techniques using probes associated with the target genes' mRNA sequences, and of RNA-sequencing. In another embodiment the expression levels of the target genes can be measured by protein levels, e.g. the concentrations of the proteins encoded by the target genes.
The aforementioned expression levels can optionally be converted in many ways that might or might not suit the application better. Here, we have used four different transformations of the expression levels, in this case microarray-based mRNA levels:
One of the simplest models that can be constructed is shown in
An alternative to the “most discriminant probesets” model, it is possible, in the case where possibly multiple expression levels are measured per target gene, to make use of all the expression levels that are provided per target gene. In such a model, one or more expression level(s) are provided for each of the one or more target gene(s) and the one or more linear combination(s) comprise a linear combination of all expression levels of the one or more expression level(s) provided for the one or more target gene(s). In other words, for each of the one or more target gene(s), each of the one or more expression level(s) provided for the respective target gene may be weighted in the linear combination by its own (individual) weight. This variant is called an “all probesets” model in the following. It has an advantage of being relatively simple while making use of all the provided expression levels.
Both models as described above have in common that they are what may be regarded as “single-layer” models, in which the level of the TF element is calculated based on a linear combination of expression levels.
After the level of the TF element has been determined by evaluating the respective model, the determined TF element level can be thresholded in order to infer the activity of the cellular signaling pathway. A method to calculate such an appropriate threshold is by comparing the determined TF element level w/c of training samples known to have a passive pathway and training samples with an active pathway. A method that does so and also takes into account the variance in these groups is given by using a threshold
where σ and μ are the standard deviation and the mean of the training samples. In case only a small number of samples are available in the active and/or passive training samples, a pseudocount may be added to the calculated variances based on the average of the variances of the two groups:
where ν is the variance of the groups and x a positive pseudocount. The standard deviation σ can next be obtained by taking the square root of the variance ν.
The threshold can be subtracted from the determined level of the TF element w/c for ease of interpretation, resulting in the pathway's activity score, such that negative values corresponds to passive pathways and positive values to active pathways.
The calculation of the summary values can, in a preferred version of the “two-layer” model, include defining a threshold for each target gene using the training data and subtracting the threshold from the calculated linear combination, yielding the gene summary. Here the threshold may be chosen such that a negative gene summary level corresponds with a downregulated target gene and that a positive gene summary level corresponds with an upregulated target gene. Also, it is possible that the gene summary values are transformed using e.g. one of the above-mentioned transformations (fuzzy, discrete, etc.) before they are combined in the “second (upper) layer”.
After the level of the TF element has been determined by evaluating the “two-layer” model, the determined TF element level can be thresholded in order to infer the activity of the cellular signaling pathway, as described above.
In the following, the models described above are collectively denoted as “(pseudo-) linear models.”
A transcription factor (TF) is a protein complex (that is, a combination of proteins bound together in a specific structure) or a protein that is able to regulate transcription from target genes by binding to specific DNA sequences, thereby controlling the transcription of genetic information from DNA to mRNA. The mRNA directly produced due to this action of the transcription complex is herein referred to as a “direct target gene”. Pathway activation may also result in more secondary gene transcription, referred to as “indirect target genes”. In the following, (pseudo-)linear models comprising or consisting of direct target genes, as direct links between pathway activity and mRNA level, are preferred, however the distinction between direct and indirect target genes is not always evident. Here a method to select direct target genes using a scoring function based on available literature data is presented. Nonetheless, accidental selection of indirect target genes cannot be ruled out due to limited information and biological variations and uncertainties.
Specific pathway mRNA target genes were selected from the scientific literature, by using a ranking system in which scientific evidence for a specific target gene was given a rating, depending on the type of scientific experiments in which the evidence was accumulated. While some experimental evidence is merely suggestive of a gene being a target gene, like for example a mRNA increasing on an microarray of an embryo in which it is known that the HH pathway is active, other evidence can be very strong, like the combination of an identified pathway transcription factor binding site and retrieval of this site in a chromatin immunoprecipitation (ChIP) assay after stimulation of the specific pathway in the cell and increase in mRNA after specific stimulation of the pathway in a cell line.
Several types of experiments to find specific pathway target genes can be identified in the scientific literature, such as (but not limited to):
In the simplest form one can give every potential target mRNA 1 point for each of these experimental approaches in which the target mRNA was identified.
Alternatively, points can be given incrementally; meaning one technology 1 point, second technology adds a second point, and so on. Using this relatively ranking strategy, one can make a list of most reliable target genes.
Alternatively, ranking in another way can be used to identify the target genes that are most likely to be direct target genes, by giving a higher number of points to the technology that provides most evidence for an in vivo direct target gene, in the list above this would mean 8 points for experimental approach 1), 7 to 2), and going down to one point for experimental approach 8. Such a list may be called “general target gene list”.
Despite the biological variations and uncertainties, the inventors assumed that the direct target genes are the most likely to be induced in a tissue-independent manner. A list of these target genes may be called “evidence curated target gene list”. These curated target lists have been used to construct computational models that can be applied to samples coming from different tissue and/or cell sources.
The “general target gene list” probably contains genes that are more tissue specific, and can be potentially used to optimize and increase sensitivity and specificity of the model for application at samples from a specific tissue, like breast cancer samples.
The following will illustrate exemplary how the selection of an evidence curated target gene list specifically was constructed for the ER pathway.
For the purpose of selecting ER target genes used as input for the (pseudo-)linear models described herein, the following three criteria were used:
The selection was done by defining as ER target genes the genes for which enough and well documented experimental evidence was gathered proving that all three criteria mentioned above were met. A suitable experiment for collecting evidence of ER differential binding is to compare the results of, e.g., a ChIP/CHIP experiment in a cancer cell line that responds to estrogen (e.g., the MCF-7 cell line), when exposed or not exposed to estrogen. The same holds for collecting evidence of mRNA transcription.
The foregoing discusses the generic approach and a more specific example of the target gene selection procedure that has been employed to select a number of target genes based upon the evidence found using above mentioned approach. The lists of target genes used in the (pseudo-)linear models for exemplary pathways, namely the Wnt, ER, HH and AR pathways are shown in Table 1, Table 2, Table 3 and Table 4, respectively.
The target genes of the ER pathway used for the (pseudo-)linear models of the ER pathway described herein (shown in Table 2) contain a selection of target genes based on their literature evidence score; only the target genes with the highest evidence scores (preferred target genes according to the invention) were added to this short list. The full list of ER target genes, including also those genes with a lower evidence score, is shown in Table 5.
A further subselection or ranking of the target genes of the Wnt, ER, HH and AR pathways shown in Table 1, Table 2, Table 3 and Table 4 was performed based on a combination of the literature evidence score and the odds ratios calculated using the training data sets linking the probeset nodes to the corresponding target gene nodes. The odds ratios are calculated using a cutoff value, e.g. the median of all training samples if the same number of active and passive training samples are used; every value above the cutoff is declared to be high and below the cutoff low. This is done for the training samples where the pathway is known to be active or passive. Subsequently the odds ratio for a specific target gene or probeset can be calculates as follows:
f(active, low)=n(active, low)/(n(active, low)+n(active, high))
f(passive, low)=n(passive, low)/(n(passive, low)+n(passive, high))
Odds ratio=f(passive, low)/(1−f(passive, low))*(1−f(active, low))/f(active, low) (3)
With n(active, low) the number of training samples known to have an active pathway that were found to have an expression level below the cutoff, n(passive, low) the number of training samples known to have a passive pathway that were found to have an expression level below the cutoff, and so on. f(active, low) and f(passive, low) the fraction of samples known to have an active or passive pathway, respectively, and found to have an expression level below the cutoff.
Alternatively, to avoid undefined odds ratios (division by zero) one can add a for example a pseudocount to the fraction calculation, e.g.:
f(active, low)pseudo=(n(active, low)+1)/(n(active, low)+n(active, high)+2)
f(passive, low)pseudo=(n(passive, low)+1)/(n(passive, low)+n(passive, high)+2) (4)
Alternatively, one can also replace the absolute number of samples exhibiting a probative activity by assuming some uncertainty (noise) in the measurement setting and calculate for each training sample a probability of being either “low” or “high” assuming e.g. a normal distribution (called “soft evidence”). Subsequently, the fraction calculations can be calculated following the aforementioned calculations.
f(active, low)soft=(Σp(active, low)+1)/(Σp(active, low)+Σp(active, high)+2)
f((passive, low)soft=(Σp(passive, low)+1)/(Σp(passive, low)+Σp(passive, high)+2) (5)
With p(active, low) and p(passive, low) the probability for each sample that the observation is below the cutoff, assuming a standard distribution with the mean equal to the measured expression level of the respective training sample and a standard deviation equal to an estimation of the uncertainty associated with the expression level measurement, e.g. 0.25 on a log2 scale. These probabilities are summed up over all the training samples, and next the pseudocount is added.
The odds ratio is an assessment of the importance of the target gene in inferring activity of the pathways. In general, it is expected that the expression level of a target gene with a higher odds ratio is likely to be more informative as to the overall activity of the pathway as compared with target genes with lower odds ratios. However, because of the complexity of cellular signaling pathways it is to be understood that more complex interrelationships may exist between the target genes and the pathway activity—for example, considering expression levels of various combinations of target genes with low odds ratios may be more probative than considering target genes with higher odds ratios in isolation. In Wnt, ER, HH and AR modeling reported herein, it has been found that the target genes shown in Table 6, Table 7, Table 8 and Table 9 are of a higher probative nature for predicting the Wnt, ER, HH and AR pathway activities as compared with the lower-ranked target genes (thus, the target genes shown in Tables 6 to 9 are particularly preferred according to the present invention). Nonetheless, given the relative ease with which acquisition technology such as microarrays can acquire expression levels for large sets of genes, it is contemplated to utilize some or all of the target genes of Table 6, Table 7, Table 8 and Table 9, and to optionally additionally use one, two, some, or all of the additional target genes of ranks shown in Table 1, Table 2, Table 3 and Table 4, in the (pseudo-)linear models as depicted in
205423 at
202325 s at
203749 s at
209081 s at
201256 at
200766 at
200642 at
205009 at
237268 at
224806 at
204274 at
200670 at
205225 at
210855 at
201508 at
202431 s at
220038 at
205792 at
201841 s at
216836 s at
201650 at
203963 at
226616 s at
202599 s at
208305 at
202392 s at
232306 at
231931 at
36499 at
211921 x at
The list of Wnt target genes constructed based on literature evidence following the procedure described herein (Table 1) is compared to another list of target genes not following above mentioned procedure. The alternative list is a compilation of genes indicated by a variety of data from various experimental approaches to be a Wnt target gene published in three public sources by renowned labs, known for their expertise in the area of molecular biology and the Wnt pathway. The alternative list is a combination of the genes mentioned in Table S3 from Hatzis et al. (Hatzis P, 2008), the text and Table S1A from de Sousa e Melo (de Sousa E Melo F, 2011) and the list of target genes collected and maintained by Roel Nusse, a pioneer in the field of Wnt signaling (Nusse, 2012). The combination of these three sources resulted in a list of 124 genes (=broad literature list, see Table 10). Here the question whether the performance in predicting Wnt activity in clinical samples by the algorithm derived from this alternative list is performing similarly or better compared to the model constructed on the basis of the existing list of genes (=evidence curated list, Table 1) is discussed.
The next step consisted of finding the probesets of the Affymetrix® GeneChip Human Genome U133 Plus 2.0 array that corresponds with the genes. This process was performed using the Bioconductor plugin in R and manual curation for the probesets relevance based on the UCSC genome browser, similar to the (pseudo-)linear models described herein, thereby removing e.g. probesets on opposite strands or outside gene exon regions. For two of the 124 genes there are no probesets available on this microarray-chip and therefore could not be inserted in the (pseudo-)linear model, these are LOC283859 and WNT3A. In total 287 probesets were found to correspond to the remaining 122 genes (Table 11).
Subsequently the (pseudo-)linear model was constructed similar to
The trained (pseudo-)linear models were then tested on various data sets to infer the activity score of the Wnt pathway. The Wnt pathway is designated to be “on”, i.e., active, when the activity level is positive. Summarized results of the trained broad literature model and the evidence curated model are shown in
Evidently, it could be deduced that the broad literature model generally predicts more extreme activity scores for Wnt signaling being on or off. In addition, the alternative model predicts similar results for the colon cancer data sets (GSE20916, GSE4183, GSE15960), but more than expected samples with predicted active Wnt signaling in breast cancer (GSE12777) and medulloblastoma sample (GSE10327) data sets.
In conclusion, the broad literature target genes list results in approximately equally well predictions of Wnt activity in colon cancer on the one hand, but worse predictions (more false positives) in other cancer types on the other hand. This might be a result of the alternative list of targets genes being too much biased towards colon cells specifically, thus too tissue specific; both de Sousa E Melo et al. and Hatzis et al. main interest was colorectal cancer although non-colon-specific Wnt target genes may be included. In addition, non-Wnt-specific target genes possibly included in these lists may be a source of the worsened predictions of Wnt activity in other cancer types. The alternative list is likely to contain more indirectly regulated target genes, which probably makes it more tissue specific. The original list is tuned towards containing direct target genes, which are most likely to represent genes that are Wnt sensitive in all tissues, thus reducing tissue specificity.
Before the (pseudo-)linear models as exemplary described herein can be used to infer pathway activity in a test sample the weights indicating the sign and magnitude of the correlation between the nodes and a threshold to call whether a node is either “absent” or present” need to be determined. One can use expert knowledge to fill in the weights and threshold a priori, but typically models are trained using a representative set of training samples, of which preferably the ground truth is known. E.g. expression data of probesets in samples with a known present transcription factor complex (=active pathway) or absent transcription factor complex (=passive pathway). However, it is impractical to obtain training samples from many different kinds of cancers, of which it is known what the activation status is of the pathway to be modeled. As a result, available training sets consist of a limited number of samples, typically from one type of cancer only. Herein a method is described to determine the parameters necessary to classify test samples as having an active or passive pathway.
Known in the field are a multitude of training algorithms (e.g. regression) that take into account the model topology and changes the model parameters, here weight and threshold, such that the model output, here weighted linear score, is optimized. Herein we demonstrate two exemplary methods that can be used to calculate the weights directly from the expression levels without the need of an optimization algorithm.
Preferably, the training of the (pseudo-)linear models of the Wnt, ER, HH and AR pathways is done using public data available on the Gene Expression Omnibus (accessible at http://www.ncbi.nlm.nih.gov/geo/, cf. above).
The first method, defined here as “black and white”-method boils down to a ternary system with the weighting factors being an element of {−1, 0, 1}. If we would put this in the biological context the −1 and 1 corresponds to genes or probes that are down- and upregulated in case of pathway activity, respectively. In case a probe or gene cannot be statistically proven to be either up- or downregulated, it receives a weight of 0. Here we have used a left-sided and right-sided, two sample t-test of the expression levels of the active pathway samples versus the expression levels of the samples with a passive pathway to determine whether a probe or gene is up- or downregulated given the used training data. In cases where the average of the active samples is statistically larger than the passive samples, i.e. the p-value is below a certain threshold, e.g. 0.3, then the probeset or target gene is determined to be upregulated. Conversely, in cases where the average of the active samples is statistically lower than the passive samples this probeset or target gene is determined to be downregulated upon activation of the pathway. In case the lowest p-value (left- or right-sided) exceeds the aforementioned threshold we define the weight of this probe or gene to be 0.
In another preferred embodiment, an alternative method to come to weights and threshold(s) is used. This alternative method is based on the logarithm (e.g. base e) of the odds ratio, and therefore called “log odds”-weights. The odds ratio for each probe or gene is calculated based on the number of positive and negative training samples for which the probe/gene level is above and below a corresponding threshold, e.g. the median of all training samples (equation 3). A pseudo-count can be added to circumvent divisions by zero (equation 4). A further refinement is to count the samples above/below the threshold in a somewhat more probabilistic manner, by assuming that the probe/gene levels are e.g. normally distributed around its observed value with a certain specified standard deviation (e.g. 0.25 on a 2-log scale), and counting the probability mass above and below the threshold (equation 5).
Alternatively, one can employ optimization algorithms known in the field such as regression to determine the weights and the threshold(s) of the (pseudo-)linear models described herein.
One has to take special attention to the way the parameters are determined for the (pseudo-)linear models to generalize well. Alternatively, one can use other machine learning methods such as Bayesian networks that are known in the field to be able to generalize quite well by taking special measures during training procedures.
Preferably, the training of the (pseudo-)linear models of the Wnt, ER, HH and AR pathways is done using public data available on the Gene Expression Omnibus (accessible at http://www.ncbi.nlm.nih.gov/geo/). The models were exemplary trained using such public data.
With reference to
With reference to
With reference to
In
Further details and examples for using trained (pseudo-)linear models (e.g. of Wnt, ER, AR and HH pathway) to predict the respective pathway activities are explained in Example 6 below.
The above mentioned training process can be employed to other (pseudo-)linear models of clinical applications. Here it is shown and proven to work for the exemplary (pseudo-)linear models constructed using herein disclosed method representing cellular signaling pathways, more specifically the Wnt, ER, AR and HH pathways.
The following will exemplary illustrate how to use e.g. the (pseudo-)linear models to diagnose the activity of a cellular signaling pathway.
The exemplary (pseudo-)linear model of the Wnt consists of a node representing the transcription factor complex, the exemplary selected readout for pathway activity, and “all probesets” mentioned in Table 1 feeding into the transcription factor complex node is trained as described herein, was used to predict the Wnt pathway activity score and it state, active or passive, in various, previously not used for training, data sets to infer how well the trained (pseudo-)linear model operates. The predicted pathway activity scores and associated activity calls calculated for a set of medulloblastoma cancer samples (GSE10327, see
The exemplary trained (pseudo-)linear model of the HH pathway consisting of two-layers, with all the probesets and target genes mentioned in Table 3 on the first and second layer, respectively was used to predict the HH activity in a set of medulloblastoma cancer samples (GSE10327, see
The exemplary trained (pseudo-)linear model of the ER pathway based on the “most discriminative probesets” and the “log odds” as depicted in Table 2 as described herein was used to predict the ER pathway activity score in a set of breast cancer samples of the GSE12276 data set. The resulting ER pathway activity scores are shown in
The exemplary trained AR (pseudo-)linear model based on “all probesets” mentioned in Table 4 and weights calculated using the “black and white”-method and fuzzy transformed expression data of LNCaP cells (GSE7868) as described herein was used to predict the activity of the AR pathway in prostate samples (GSE17951, fuzzy transformed). The calculated AR activity scores for the three groups of samples (from left to right: biopsy, control and tumor) are shown in
Early developmental pathways, like Wnt and HH, are thought to play a role in metastasis caused by cancer cells which have reverted to a more stem cell like phenotype, called cancer stem cells. Indeed, sufficient evidence is available for the early developmental pathways, such as Wnt pathway, to play a role in cancer metastasis, enabling metastatic cancer cells to start dividing in the seeding location in another organ or tissue. Metastasis is associated with bad prognosis, thus activity of early developmental pathways, such as the Wnt and HH pathway, in cancer cells is expected to be predictive for bad prognosis. This is supported by the fact that breast cancer patients, from the GSE12276 data set, that were identified having an active ER pathway but not having an active Wnt or HH pathway using the (pseudo-)linear models described herein had a better prognosis than patients identified having either an active HH or Wnt pathway or both, as illustrated by the Kaplan-Meier plot in
The following exemplary illustrates how to use (pseudo-)linear models of cellular signaling pathways for therapy planning, prediction of drug efficacy, monitoring of drug efficacy and related activities.
The (pseudo-)linear model of the ER pathway, constructed using a node for the transcription factor presence and a layer of probesets (Table 2) associated with the target genes of the ER pathway, analogous to
With respect to
The control Tamoxifen resistant cell line, indicated by TamR.Ctrl, is predicted to have an inactive ER pathway for every time point after Tamoxifen addition (1, 2, 3, 6, 12, 24, and 48 h). It is not surprising that treatment of the Tamoxifen resistant cell line stimulated with E2 and treated with Tamoxifen, indicated by TamR.E2_Tam (fourth group), is ineffective, which is also illustrated by the predicted inactivity of the ER pathway for this group over the same time points. According to analysis of the Tamoxifen resistant cell line (TamR.Ctrl) the driving force of the uncontrolled cell proliferation is not due to active ER signaling; therefore treating it with an ER antagonist will not inhibit cell proliferation. This illustrates that treatment with Tamoxifen is not recommended in case of a negative predicted ER pathway activity.
On the other hand, the wild type MCF7 cell line, known to be Tamoxifen sensitive, treated with 17beta-estradiol (wt1.E2, eleventh group) slowly reacts to the hormone treatment which is visible in the increasing ER positive activity predictions. Treating such a cell line with ER inhibitors such as Tamoxifen will inhibit the ER pathway which is illustrated by the decreasing ER pathway activity score in time of the MCF7 samples stimulated with E2 and treated with Tamoxifen (wt2.E2_Tam, twelfth group).
In another example, a publically available data set of MCF7 cell lines stimulated with or deprived of ER stimulating agent (E2) with expression levels measured at 12 hours, 24 hours and 48 hours after starting stimulation or deprivation (GSE11352) was used to calculate the ER activity scores using the trained ER (pseudo-)linear model as described herein. The ER pathway activity score increases for longer exposure times to the ER stimulating agent (first three groups) and decreases in case of prolonged starvation in the control (last three groups), although prolonged deprivation increases slightly after 48 hours again. With the exception of the starvation of 48 hours, the predicted ER activity scores nicely correlates with the knowledge that prolonged stimulation result in higher ER activity and vice versa. Inversely, this example implies that the ER activity score can be used to monitor efficacy or inefficacy of stimulation or inhibition of ER activity treatments.
Similar to therapy response monitoring, a pathway model can be used in drug development to assess the effectiveness of various putative compounds. For instance, when screening many compounds for a possible effect on a certain pathway in a cancer cell line, the respective pathway model can be used to determine whether the activity of the pathway goes up or down after application of the compound or not. Often, this check is done using only one or a few of putative markers of the pathway's activity, which increases the chance of ineffective monitoring of the treatment effect. Furthermore, in follow-up studies on animal or patient subjects, the pathway models can be used similarly to assess the effectiveness of candidate drugs, and to determine an optimal dose to maximally impact pathway activity.
An example of ineffective monitoring of new drug compounds is illustrated by the predicted AR pathway activity in the GSE7708 samples as shown in
Instead of applying the mentioned (pseudo-)linear models on mRNA input data coming from microarrays or RNA sequencing, it may be beneficial in clinical applications to develop dedicated assays to perform the sample measurements, for instance on an integrated platform using qPCR to determine mRNA levels of target genes. The RNA/DNA sequences of the disclosed target genes can then be used to determine which primers and probes to select on such a platform.
Validation of such a dedicated assay can be done by using the microarray-based (pseudo-)linear models as a reference model, and verifying whether the developed assay gives similar results on a set of validation samples. Next to a dedicated assay, this can also be done to build and calibrate similar (pseudo-)linear models using mRNA-sequencing data as input measurements.
The following will illustrate how (pseudo-)linear models can be employed in (clinical) pathway research, that is research interested to find out which pathways are involved in certain diseases, which can be followed up for more detailed research, e.g. to link mutations in signaling proteins to changes in pathway activation (measured with the model). This is relevant to investigate the initiation, growth and evolution and metastasis of specific cancers (the pathophysiology).
The (pseudo-)linear models of the Wnt, ER, HH and AR pathway, constructed using at least a node for the transcription factor presence and a layer of nodes representing the target genes' mRNA expression levels as measured by their associated probesets (Table 1, Table 2, Table 3 and Table 4), analogous to
Suppose the researcher is interested in looking into the cellular signaling pathway or pathways and the specific deregulation(s) that drive(s) the uncontrolled cell proliferation. The researcher can analyze the microarray data using the above mentioned (pseudo-)linear models to find which pathways are presumably the cause of uncontrolled cell proliferation. Shown in
With reference to
Another example is given in
In summary, the illustrations described herein indicate the ability of trained (pseudo-)linear models (as described above) to support the process of finding the cause of uncontrolled cell proliferation in a more directed method. By employing the (pseudo-) linear models to screen the samples for pathway activities, the predicted pathway activities can pinpoint the possible pathways for the uncontrollable cell proliferation, which can be followed up for more detailed research, e.g. to link mutations in signaling proteins or other known deregulations to changes in activation (as measured with the model).
As described herein, the process to develop and train a (pseudo-)linear model of cellular signaling pathways can be used to construct a (pseudo-)linear model for other pathways that could also be employed in connection with the present invention.
If a candidate drug is developed to, for instance, block the activity of a certain pathway that drives tumor growth, and this drug is going into clinical trial, then a proper selection of the subjects to enroll in such a trial is essential to prove potential effectiveness of the drug. In such a case, patients that do not have the respective pathway activated in their tumors should be excluded from the trial, as it is obvious that the drug cannot be effective if the pathway is not activated in the first place. Hence, a pathway model that can predict pathway activity, such as the (pseudo-)linear models described herein, can be used as a selection tool, to only select those patients that are predicted to have the respective pathway activated.
If a tumor is analyzed using different pathway models, and the models predict deregulation of a certain pathway, then this may guide the selection of subsequent tests to be performed. For instance, one may run a proximity ligation assay (PLA) to confirm the presence of the respective transcription complex (Soderberg O, 2006). Such a PLA can be designed to give a positive result if two key proteins in a TF complex have indeed bound together, for instance beta-catenin and TCF4 in the TF complex of the Wnt pathway.
Another example is that the pathway predicted to be deregulated is analyzed in more detail with respect to the signaling cascade. For instance, one may analyze key proteins in this pathway to determine whether there are mutations in the DNA regions encoding for their respective genes, or one may test for the abundance of these proteins to see whether they are higher or lower than normal. Such tests may indicate what the root cause is behind the deregulation of the pathway, and give insights on which available drugs could be used to reduce activity of the pathway.
These tests are selected to confirm the activity of the pathway as identified using the (pseudo-)linear models. However selection of companion diagnostic tests is also possible. After identification of the pathway using the model, for targeted therapy choice only those companion diagnostics tests need to be performed (the selection), which are applicable to the identified pathway.
Similar to the previous example, if a tumor is analyzed and the pathway models predict deregulation of a certain pathway, and optionally a number of additional tests have been performed to investigate the cause of deregulation, then an oncologist may select a number of candidate drugs to treat the patient. However, treatment with such a drug may require a companion diagnostic test to be executed first, for instance to comply with clinical guidelines or to ensure reimbursement of the treatment costs, or because regulatory (FDA) it is required to perform the companion diagnostic test prior to giving the drug. An example of such a companion diagnostic test is the Her2 test for treatment of breast cancer patients with the drug Herceptin (Trastuzumab). Hence, the outcome of the pathway models can be used to select the candidate drugs and the respective companion diagnostic tests to be performed.
With reference to
The CDS system 10 receives as input information pertaining to a medical subject (e.g., a hospital patient, or an outpatient being treated by an oncologist, physician, or other medical personnel, or a person undergoing cancer screening or some other medical diagnosis who is known or suspected to have a certain type of cancer such as colon cancer, breast cancer, or liver cancer, or so forth). The CDS system 10 applies various data analysis algorithms to this input information in order to generate clinical decision support recommendations that are presented to medical personnel via the display device 14 (or via a voice synthesizer or other device providing human-perceptible output). In some embodiments, these algorithms may include applying a clinical guideline to the patient. A clinical guideline is a stored set of standard or “canonical” treatment recommendations, typically constructed based on recommendations of a panel of medical experts and optionally formatted in the form of a clinical “flowchart” to facilitate navigating through the clinical guideline. In various embodiments the data processing algorithms of the CDS 10 may additionally or alternatively include various diagnostic or clinical test algorithms that are performed on input information to extract clinical decision recommendations, such as machine learning methods disclosed herein.
In the illustrative CDS systems disclosed herein (e.g., CDS system 10), the CDS data analysis algorithms include one or more diagnostic or clinical test algorithms that are performed on input genomic and/or proteomic information acquired by one or more medical laboratories 18. These laboratories may be variously located “on-site”, that is, at the hospital or other location where the medical subject is undergoing medical examination and/or treatment, or “off-site”, e.g. a specialized and centralized laboratory that receives (via mail or another delivery service) a sample of tissue and/or cells of the medical subject that has been extracted from the medical subject (e.g., a sample obtained from a breast lesion, or from a colon of a medical subject known or suspected of having colon cancer, or from a liver of a medical subject known or suspected of having liver cancer, or so forth, via a biopsy procedure or other sample extraction procedure). The tissue of which a sample is extracted may also be metastatic tissue, e.g. (suspected) malignant tissue originating from the colon, breast, liver, or other organ that has spread outside of the colon, breast, liver, or other organ. The cells of which a sample is extracted may also be tumorous cells from hematologic malignancies (such as leukemia). In some cases, the cell sample may also be circulating tumor cells, that is, tumor cells that have entered the bloodstream and may be extracted as the extracted tissue sample using suitable isolation techniques. The extracted sample is processed by the laboratory to generate genomic or proteomic information. For example, the extracted sample may be processed using a microarray (also variously referred to in the art as a gene chip, DNA chip, biochip, or so forth) or by quantitative polymerase chain reaction (qPCR) processing to measure probative genomic or proteomic information such as expression levels of genes of interest, for example in the form of a level of messenger ribonucleic acid (mRNA) that is transcribed from the gene, or a level of a protein that is translated from the mRNA transcribed from the gene. As another example, the extracted sample may be processed by a gene sequencing laboratory to generate sequences for deoxyribonucleic acid (DNA), or to generate an RNA sequence, copy number variation, or so forth. Other contemplated measurement approaches include immunohistochemistry (IHC), cytology, fluorescence in situ hybridization (FISH), proximity ligation assay or so forth, performed on a pathology slide. Other information that can be generated by microarray processing, mass spectrometry, gene sequencing, or other laboratory techniques includes methylation information. Various combinations of such genomic and/or proteomic measurements may also be performed.
In some embodiments, the medical laboratories 18 perform a number of standardized data acquisitions on the extracted sample of the tissue and/or cells of the medical subject, so as to generate a large quantity of genomic and/or proteomic data. For example, the standardized data acquisition techniques may generate an (optionally aligned) DNA sequence for one or more chromosomes or chromosome portions, or for the entire genome of the tissue and/or cells. Applying a standard microarray can generate thousands or tens of thousands of data items such as expression levels for a large number of genes, various methylation data, and so forth. This plethora of genomic and/or proteomic data, or selected portions thereof, are input to the CDS system 10 to be processed so as to develop clinically useful information for formulating clinical decision support recommendations.
The disclosed CDS systems and related methods relate to processing of genomic and/or proteomic data to assess activity of various cellular signaling pathways. However, it is to be understood that the disclosed CDS systems (e.g., CDS system 10) may optionally further include diverse additional capabilities, such as generating clinical decision support recommendations in accordance with stored clinical guidelines based on various patient data such as vital sign monitoring data, patient history data, patient demographic data (e.g., gender, age, or so forth), patient medical imaging data, or so forth. Alternatively, in some embodiments the capabilities of the CDS system 10 may be limited to only performing genomic and/or proteomic data analyses to assess cellular signaling pathways as disclosed herein.
With continuing reference to exemplary
Measurement of mRNA expression levels of genes that encode for regulatory proteins of the cellular signaling pathway, such as an intermediate protein that is part of a protein cascade forming the cellular signaling pathway, is an indirect measure of the regulatory protein expression level and may or may not correlate strongly with the actual regulatory protein expression level (much less with the overall activity of the cellular signaling pathway). The cellular signaling pathway directly regulates the transcription of the target genes—hence, the expression levels of mRNA transcribed from the target genes is a direct result of this regulatory activity. Hence, the CDS system 10 infers activity of the cellular signaling pathway (e.g., the Wnt, ER, AR and HH pathways) based at least on expression levels of target genes (mRNA or protein level as a surrogate measurement) of the cellular signaling pathway. This ensures that the CDS system 10 infers the activity of the pathway based on direct information provided by the measured expression levels of the target genes.
However, although, as disclosed herein, being effective for assessing activity of the overall pathways, the measured expression levels 20 of target genes of the pathways are not especially informative as to why the pathways are operating abnormally (if indeed that is the case). Said another way, the measured expression levels 20 of target genes of a pathway can indicate that the pathway is operating abnormally, but do not indicate what portion of the pathway is malfunctioning (e.g., lacks sufficient regulation) in order to cause the overall pathway to operate abnormally.
Accordingly, if the CDS system 10 detects abnormal activity of a particular pathway, the CDS system 10 then optionally makes use of other information provided by the medical laboratories 18 for the extracted sample, such as aligned genetic sequences 22 and/or measured expression level(s) for one or more regulatory genes of the pathway 24, or select the diagnostic test to be performed next in order to assess what portion of the pathway is malfunctioning. To maximize efficiency, in some embodiments this optional assessment of why the pathway is malfunctioning is performed only if the analysis of the measured expression levels 20 of target genes of the pathway indicates that the pathway is operating abnormally. In other embodiments, this assessment is integrated into the analysis of the cellular signaling pathway described herein.
In embodiments in which the CDS system 10 assesses what portion of the pathway is malfunctioning, and is successful in doing so, the additional information enables the CDS system 10 to recommend prescribing a drug targeting for the specific malfunction (recommendation 26 shown in
The set of target genes which are found to best indicate specific pathway activity, based on microarray/RNA sequencing based investigation using the (pseudo-)linear model, can be translated into a multiplex quantitative PCR assay to be performed on a tissue or cell sample. To develop such an FDA-approved test for pathway activity, development of a standardized test kit is required, which needs to be clinically validated in clinical trials to obtain regulatory approval.
In general, it is to be understood that while examples pertaining to the Wnt, the ER, the AR and/or the HH pathway(s) are provided as illustrative examples, the approaches for cellular signaling pathway analysis disclosed herein are readily applied to other cellular signaling pathways besides these pathways, such as to intercellular signaling pathways with receptors in the cell membrane (cf above) and intracellular signaling pathways with receptors inside the cell (cf above). In addition: This application describes several preferred embodiments. Modifications and alterations may occur to others upon reading and understanding the preceding detailed description. It is intended that the application be construed as including all such modifications and alterations insofar as they come within the scope of the appended claims or the equivalents thereof.
| Filing Document | Filing Date | Country | Kind |
|---|---|---|---|
| PCT/IB2013/061066 | 12/18/2013 | WO | 00 |
| Number | Date | Country | |
|---|---|---|---|
| 61745839 | Dec 2012 | US |
