Extraction-Empowered machine translation

Abstract

The invention relates to systems and methods for automatically translating documents from a first language to a second language. To carry out the translation of a document, elements of information are extracted from the document and are translated using one or more specialized translation processes. The remainder of the document is separately translated by a statistical translation process. The translated elements of information and the translated remainder are then merged into a final translated document.

Description

BACKGROUND

Machine translation (MT) is the automatic translation of text from one language to another using a computer system. At a basic level, MT provides translations of individual words from one language to another. One type of machine translation is rule-based MT, in which words are translated based on linguistic rules. Generally, rule-based methods first parse an input text and create an intermediate, symbolic representation of the text, and then use this representation to generate the translation of the text in the target language. One drawback of rule-based MT is that it requires large sets of hand-crafted rules and grammars. Another type of machine translation is statistical machine translation, in which the translation is generated based on statistical models. The parameters of the statistical model are derived from bilingual text corpora, and key to the performance of these systems is the availability of massive amounts of translated documents (e.g., in some cases including 80 million words or more) for the major U.N. languages (Arabic, Chinese, English, French, Russian, and Spanish). For less common languages, large amounts of translated documents are not available. Systems and methods are needed for providing MT without requiring large sets of hand-crafted rules or massive amounts of translated documents.

SUMMARY

Presented herein are systems, methods, and devices for machine translation. Machine Translation takes text (e.g., online text, transcribed speech, or OCRed print media) in one language and translates it to a second language. For example, the input text to a machine translator may be Arabic newswire, and the output text may be an equivalent version in English.

BRIEF DESCRIPTION OF THE FIGURES

The foregoing and other objects and advantages of the invention will be appreciated more fully from the following further description thereof, with reference to the accompanying drawings. These depicted embodiments are to be understood as illustrative of the invention and not as limiting in any way.

FIG. 1 is a diagram depicting an exemplary machine translation process, according to an illustrative embodiment of the invention.

FIG. 2 is a diagram depicting an extraction process, according to an illustrative embodiment of the invention.

FIGS. 3A-3C depict translation processes for exemplary elements of information, according to an illustrative embodiment of the invention.

FIG. 4 demonstrates the merging of the outputs from a specialized translation process and a statistical MT process.

FIG. 5 depicts an example of an agreement constraint.

FIG. 6 is a flow chart of a method of training a machine translation process to enforce the agreement constraint described in relation to FIG. 5.

DETAILED DESCRIPTION

To provide an overall understanding of the invention, certain illustrative embodiments will now be described, including systems, methods and devices for machine translation. However, it will be understood by one of ordinary skill in the art that the systems and methods described herein can be adapted and modified for other suitable applications and that such other additions and modifications will not depart from the scope hereof.

An exemplary application is to offer the machine translation within the Broadcast Monitoring System suite, a product provided by BBN Technologies Corp. of Cambridge, Mass., which monitors, transcribes, and translates broadcast media content in multiple languages for near-real time search and analysis. Additionally, it could be an independent component applied to text. There are many languages where there are currently few/no commercial machine translation products (e.g., Farsi, Korean, Pashto, That, Urdu, . . . ). Other applications include commercial applications, e.g., English language web access to non-English sites.

In contrast to traditional rule-based systems, which require massive sets of hand-crafted rules and grammars to function, the approach of the current system uses minimal manual work because of the trainable algorithms at its core. Hand-crafted transfer rules are limited to high-value pieces of information that a pure trainable system cannot handle well without massive amounts of training data. Since such high-value pieces of information (e.g. entities, descriptions and relations) usually have a very simple structure, a little manual work in rule creation can produce a great gain in performance.

FIG. 1 is a diagram depicting an exemplary machine translation system 100 according to one embodiment of the invention. A text 102 in a first language is input into an extraction module 104, which extracts selected elements of information (EIs) 108 from the text, and outputs the EIs 108 and a text remainder 112. The text remainder 112 includes placeholders for the extracted EIs, as described in greater detail with respect to FIG. 2. The text remainder 112 is sent to a statistical machine translation module 114, where it is translated using statistical machine translation. The EIs 108 are translated using specialized translation process modules 110, where the selected translation process depends on the type of EI being translated, as described in greater detail below with respect to FIGS. 3A-3C. The specialized translation process modules 110 and the statistical MT module 114 may output multiple possible translations, as well as the probability of each output translation possibility. The outputs of the statistical MT module 114 and specialized translation process modules 110 are sent to a merge module 122, which determines multiple ways of integrating the translated EIs into the possible text remainder translations, and estimates the probability of each possibility. The merge module 122 selects the most likely integration possibility, and merges the selected EI translations with the selected text remainder. The merge module 112 outputs the translated text 126 in language L2. The various steps of the process shown in FIG. 1 are described in greater detail below with respect to FIGS. 2-4.

According to various embodiments, the specialized translation process 110 and the statistical machine translation process 114 can be carried out serially or in parallel.

According to various embodiments, the machine translation process 100 uses word meaning (the extracted information) and linguistic knowledge (the syntax of, for example, dates/times) to yield high performance translation of important information from the L1 text. That is, for example, using an information extraction process at the extraction module 104, named entities, dates, times, amounts, core noun phrase descriptions, and other classes of information are identified in the native language of the L1 text 102. The extracted elements of information are translated or transliterated as appropriate using a first translation process.

In one implementation, the extraction module 104, the statistical MT module 114, the specialized translation process modules 110, and the merger module 122 are implemented as software. The modules include computer executable instructions, programmed, for example, in C, C++, JAVA, LISP, or other programming language known to those skilled in the art. The computer executed instructions are stored on computer readable media, such as magnetic disks, optical disks, magneto-optical disks, holographic disks, magnetic tapes, or integrated circuits (e.g., flash memory cards). The modules 104, 110, 114, and 122, upon execution, are loaded into a volatile system memory associated with one or more processors. Two or more of the modules 104, 110, 114, and 122, may execute in a memory shared by one or more processors. Similarly, two or more modules may be executed by a single processor. Alternatively, each module 104, 110, 114, and 122 may execute on its own processor with its own memory. The processors may be single or multi-core processors. The processors may be general purpose processors, special purpose processors, digital signal processors, or other form of integrated circuit. In various implementations, one or more of the modules is located on a first computing device, such as a desktop computer, a laptop computers, a personal digital assistant, a cell phone, or other computing device.

In one implementation, the system 100 also includes a user interface module for accepting translation instructions and outputting translated documents. Translated documents may be output to a display screen, to a printer, or to a speaker via a text to speech process. The user interface may also be coupled to a document input device, such as a camera, a speech-to-text-process, a scanner, or a source of digital text information, such as an attached computer readable data store, a remote server, or the Internet. The user interface module may execute on the same computing device as the remaining modules 104, 110, 114, 122, or it may communicate with the remaining modules 104, 110, 114, and 122 over a wired or wireless network.

FIG. 2 is a diagram of an extraction process 200. In the extraction process 200, an extraction module 202 extracts EIs from a text 204. As shown in FIG. 2, words W1-W5 are input to the extraction module 202. The extraction module 202 identifies words w3 and w4 as EIs, and extracts words w3 and w4 from the text 204. The extraction module 202 outputs the EIs 210, and the remainder of the text, 212. The text 212 remainder includes a copy of the input words w1, w2, and w5, with the extracted words w3 and w4 replaced by a placeholder ph1. In this example, words w3, and w4, are clustered to form a single EI 210. In various examples, words w3 and w4 may by a core noun phrase, a name, a date, a time, or a monetary expression. According to some embodiments, the words w3 and w4 may be individual EIs. In other instances words w3 and w4 may be distinct EIs.

Several different types of EIs 210 may be extracted by the extraction module 202, including core noun phrases, dates, times, monetary expressions, and names. A core noun phrase is a noun phrase with no other noun phrases embedded therein (i.e., it is the smallest noun phrase). According to one embodiment, rule-based approaches or trainable chunk parsers may be used by the extraction module to recognize core noun phrases. Various information extraction process known in the art, including, for example, the IDENTIFINDER system offered by BBN Technologies Corp., headquartered in Cambridge, Mass., may be employed by the extraction module 202, to identify EIs in the text.

FIGS. 3A-3C depict translation processes for exemplary elements of information. In FIG. 3A, the illustrative element of information is a core noun phrase 302. The core noun phrase 302 may be translated using one of three possible translation processes know in the art. The first translation process 304 is statistical machine translation, which may be constrained by linguistic and extraction cues. The second translation process 306 is rule-based machine translation. The third translation process 308 is a mixture model including direct translation using manual bilingual lexicons and parallel text lexicons. The translation process may be selected based on the first or second language or the core noun phrase 302. In some embodiments, the selected translation process is predetermined.

The second translation process 306 is rule-based. Many core noun phrase structures in a first language may be translated to a second language word by word with some changes in word order. For many core noun phrase structures, the change in word order, or the mapping of the core noun phrase structure in the first language to a core noun phrase structure in the second language, is predictable. Thus, these core noun phrases may be reliably handled by transfer rules and lexicon lookup. The transfer rules may be provided by bilingual experts. In some embodiments, the transfer rules may be learned by a statistical algorithm using parallel text corpora.

The third translation process 308 is a mixture model combining manual bilingual lexicons and lexicons learned from parallel text corpora. The combination of manual and automatic lexicons can greatly improve performance. Since resources are scarce for Less Commonly Taught Languages (LCTLs), resource combination will be even more useful to good MT performance. Given translation probabilities p(a|e, d_i) for an English word e and a foreign word a in dictionaries d_is, the mixture model combines them into a single probability estimate:

$p\left(a|e\right)=\sum _{\mathrm{all}d_i^{\prime }s}{\lambda }_ip\left(a|e,d_i\right)$

where Σλ_i=1. λ_i is the coefficient given to source d_i. The coefficients can be tuned to maximize translation performance.

In FIG. 3B, the illustrative elements of information are a date 320, a time 322, and a monetary expression 324. Generally, dates 320, times 322, and monetary expressions 324 are translated using rule-based machine translation 326. The rule-based machine translator uses semantic rules of the target language for the corresponding type of information to translate the input EI.

Date, time, and monetary expressions usually follow simple patterns and can be converted to a standard format by an appropriate set of rules. Thus, hand-crafted grammars may be used to handle date and time expressions for many languages. According to one embodiment, date and time expressions in a first language are re-formatted to a numerical representation (e.g., “May 21, 2001” will be represented as “2001-5-21”). In another embodiment, date and time expressions are reformatted using other standard formats, such as the format specified in the TIMEX 2 annotation guideline used by the TIDES program. The re-formatted date and time expressions are then translated directly. Some date and time expressions cannot be normalized (e.g. “during Tom's birthday party”), and for these expressions, statistical machine translation 328 is used. Similar techniques are used to develop grammars for rule-based machine translation of monetary expressions.

In FIG. 3C, the illustrative EI is a name 340. The name 340 may be a proper name, and it may be the name of a person or a place. The translation process 342 used to translate the name 340 is transliteration. The translation process may be a trainable transliteration algorithm, or it may be created manually.

According to one embodiment, a trainable transliteration algorithm is context dependent. Given a foreign name A, the model searches for the most likely English transliteration E. According to Bayes rule, maximizing P(E|A) is the same as maximizing P(E)P(A|E). P(E) is the language model probability, and may be estimated from a large English corpus. P(A|E) is estimated from the training data. The training data may be a list of names transliterated by humans. In one example, E=e₁ e₂ . . . e_n, where e₁ e₂ . . . e_n, are the phonemes in an English word, and A=a₁ a₂ . . . a_n, where a₁ a₂ . . . a_n are the phonemes in the foreign word. The model generates A from E, by probabilistically mapping e_is to a_i's one by one, starting with the leftmost phoneme. The model assumes the probability that e_i generates a_i depends not only on e_i and a_i, but also on e_i-1 and a_i-1, to capture the dependences between adjacent phonemes. For example, the pronunciation of the English grapheme “ch” maps to different phonemes depending on whether it is preceded by “s” or not, e.g. “school” vs “choose”. Using this assumption, the following formula holds:

$P\left(A|E\right)=P\left(a_1|e_1\right)\prod _{1<i\le n}P\left(a_i|e_i,e_{i-1},a_{i-1}\right)$

P(a_i|e_i, e_i-1, a_i-1) is estimated from the training data using the Expectation Maximization (EM) algorithm. The EM algorithm is used because there are often multiple ways to segment a letter sequence into phonemes. Thus, a name pair may have many possible alignments. The EM algorithm can factor in the probabilities of different alignments in parameter estimation. In other embodiments, direct counting is used instead of the EM algorithm.

The number of name pairs used for training the algorithm may be reduced by smoothing the above context-aware model with a simpler model (e.g., a context-independent model):

P _smooth(a_i|e_i, e_i-1, a_i-1)=λP(a_i|e_i, e_i-1, a_i-1)+(1−λ)P(a_i|e_i)

According to one embodiment, a few thousand name pairs are used to train the model. The cost to create the training data for a new language may be about 1-2 weeks by a bilingual speaker. In some embodiments, the model is trained using corpora rich in named entities, such as phonebooks or the Word Wide Web, as well as traditional text or conversational corpora.

In an alternative implementation, the process selected to translate a name depends on the semantic type of the name. For example, names of people are translated using a transliteration process. Names of organizations are translated using a hybrid translation/transliteration process, and names of geopolitical entities are translated using a bilingual lexicon.

Referring back to FIG. 2, the remainder of the text 212 is translated using statistical machine translation. Statistical MT chooses the most likely sentence e in English, given a sentence f in a foreign language, i.e., that maximizes p(e|f). Because of Bayes rule, maximizing p(e|f) is the same as maximizing p(f|e)p(e). P(f|e) estimates how well e preserves the meaning of f and p(e) estimates the fluency of e. In other words, an ideal translation should carry the meaning of the original sentence and be fluent. Models of p(e) known in the art have essentially been limited to literal word sequences (bi-grams, tri-grams, and 4-grams). Estimating p(e|f) is typically factored into two parts: a “translation” probability for a word/word sequence independent of context and a “distortion” probability to model movement, i.e., the difference in word order between f and e. Distortion models have assumed arbitrary words are placed first, and nearby words are moved with respect to them. Key to the performance of these systems is the availability of massive amounts (e.g., 80 million words) of translated documents for the major U.N languages (Arabic, Chinese, English, French, Russian, and Spanish). Translation models have recently moved from assuming each word is translated independently to short word sequences.

FIG. 4 demonstrates the merging of the specialized translation process output 402 with the statistical machine translation output 404 for a sentence being translated. In the illustrative example, for a selected element of information (EI1) 410 extracted from the sentence, the specialized translation process outputs three possible translated words 412a-412c (tw₁₁, tw₁₂, and tw₁₃), each having a corresponding probability 414a-414c (p₁, p₂, and p₃) of being correct. For the remainder 420 of the sentence (f₁ f₂ ph₁ f₃ f₄), the statistical machine translation process outputs four possible translations 422a-422d (e₁ e₂ ph₁ e₃ e₄; e₂ e₁ ph₁ e₃ e₄; e₁ ph₁ e₂ e₃ e₄; and e₁ e₅ ph₁ e₃ e₄) each having a corresponding probability 424a-424d (q₁, q₂, q₃, and q₄) of being correct.

As shown in the illustrative example, the remainder 420 of the sentence includes 4 foreign words f₁, f₂, f₃, and f₄ and a placeholder ph₁. As may occur in actual translation, the illustrative potential translations 422a-422d include different word orders for the translated words e₁, e₂, e₃, and e₄ (e.g., the first translation 422a vs. the second translation 422b), different word orders for the translated words e₁, e₂, e₃, and e₄ with respect to the placeholder ph₁ (e.g., the first translation 422a vs. the third translation 422c), and different translation of the foreign words f₁, f₂, f₃, and f₄ to translated words (e.g. the translation of the second foreign word f₂ in the first translation 422a vs. the fourth translation 422d). Word order determinations and corresponding probabilities can be derived by the statistical MT process based on a supervised or unsupervised training process. In practice, both the specialized translation process 402 and the statistical machine translation process 404 may output any selected number of translations and corresponding probabilities. In some embodiments, the translations processes 402 and 404 may eliminate selected translations 412a-412c and 422a-422d based on the corresponding probabilities 414a-414c and 424a-424d of the selected translation. For example, translations of an EI 410 or a sentence remainder 420 with probabilities that fall below a selected threshold may be eliminated. Alternatively, the system may only maintain a selected number of highest probability potential translations. For example, if the specialized translation process 402 is configured to output the three most likely translations, any potential identified translation with a probability that is lower than three other potential translations is eliminated from contention.

As shown in FIG. 4, the specialized translation process output 402 is merged with the statistical machine translation output 404 resulting in the merged translation output 430 to form a set of potential sentence translations Oct. 22, 2007. According to the illustrative embodiment, the potential sentence translations 432 include combinations of the merger of the statistical machine translations 422a-422d with the placeholder ph₁ replaced with the translated EI 410 possibilities, i.e., words 412a-412c, output by the specialized translation process. Each potential sentence translation 432 has a corresponding overall probability score 434 (x₁-x₁₂). The overall probability score x_i of ith potential sentence translation is calculated as a function of the probability p_i that the EIs in the potential sentence translation are correct, the probabilities q_i that the translation of the remainder of the sentence is correct, and the probability of the potential sentence translation existing in the target language. To obtain this last probability, the entire potential sentence translation is compared to a language model for the target language. In one embodiment, the potential sentence translation with the highest overall probability is selected as the correct translation.

In an alternative implementation, the machine translation system 100 also generates translations of the sentences that leave one or more EIs in the sentence, without replacing them with placeholders. In this embodiment, each sentence is scored with an additional parameters corresponding to the likelihood that the each EI should have been treated as an EI (i.e., be replaced with a place holder), as opposed to being treated as an ordinary word or span of words. The likelihood can be a probability or an unnormalized score, and can be combined with the other scores discussed above to yield a final score.

In another implementation, the translation process occurs in a more iterative fashion. For example, instead of translating entire sentences at a time, individual phrases are translated as described above, yielding a set of potential phrase translations. The number of potential translations of the phrase may be predetermined (i.e., the n most likely translations), or the number may vary based on how many possible phrase translations have probabilities over a predetermined threshold. Next, as consecutive phrases are translated, the probability of ordered combinations of such phrases are evaluated, and low probability combinations are eliminated. The process continues until the translation of sentence is complete. In one implementation, the iteration stops at this point, and the potential combination of phrases with the highest probability is selected as a correct translation. Alternatively, the final selection of a translation is only made after evaluating the probability of an entire paragraph.

After translation is complete, the translated document is saved in memory for later output and analysis, output immediately to a user via a printer or display, and/or fed into another text analysis program for further analysis and processing.

According to one implementation, the statistical MT process module 114 is trained to detect clusters of words that remain together during translation. Such information may be retained within the statistical MT process module 114 in the form of statistics, or as an ancillary rule, referred to as an agreement constraint, that limits the potential outputs of the statistical MT process 114. That is, outputs that fail to meet the agreement constraint may be assigned low probabilities, or they may be eliminated as potential translations, altogether. Similar agreement constraints may also be employed by one or more of the specialized translation process modules 110.

FIG. 5 depicts an example 500 of an agreement constraint that may be applied to the translation process in one embodiment of the invention. In the example 500, a phrase including 5 foreign language words f1, f2, f3, f4, and f5 is translated to an English sentence including 5 English words e1, e2, e3, e4, and e5. Two of the foreign language words (f3 and f4) are clustered together, and the translation process is constrained such that the translation of these two words (i.e., English words e1 and e2) is also clustered together. Including an agreement constraint in the possible word orders of the statistical machine translation output 404 and the merged translations 432 of FIG. 4 restricts the possible translations that are considered, thereby automatically eliminating unlikely translations.

FIG. 6 is a flow chart of a method 600 of training a machine translation process to enforce the agreement constraint described in relation to FIG. 5. For example, the method 600 may be applied to train the statistical machine translation process 114 or the merge module 122 of FIG. 1. The method begins obtaining large parallel corpora in multiple languages, i.e., the corpora include the same content in different languages (step 602). Next, an information extraction module processes each corpus to identify EIs in each corpora (step 604). Next the words and elements in the parallel corpora are aligned to one another (step 606). That is, it is determined which words in a first corpus correspond to particular words in the second corpus. The alignment can be carried out using statistical learning techniques, based, for example, on co-occurrence statistics. A constraint, however, is applied to the alignment process requiring that each element of information identified in the parallel corpora are aligned to contiguous self contained spans of text, as described for example in FIG. 5. In addition, the EIs can only be aligned to EIs of the same type. For example, person names must align to person names, geopolitical entity names must align to geopolitical names, etc.

These constraints may be applied in the at least three possible ways. In one implementation, during the alignment process of training the machine translation process, identified EIs are replaced with placeholders and such place holders can only be aligned to other EIs. In a second implementation, during operation of the process to translate text, the constraints are applied when generating potential translated sentences, by only generating sentences in which EIs in the first language are replaced with EIs in the second language. In a third implementation, also applied during application of the process to text for translation, the constraints are applied when scoring potential sentences. For example, potential sentences that fail to meet the constraints are removed from consideration or have their score reduced based on the failure.

Those skilled in the art will know or be able to ascertain using no more than routine experimentation, many equivalents to the embodiments and practices described herein. Accordingly, it will be understood that the invention is not to be limited to the embodiments disclosed herein, but is to be understood from the following claims, which are to be interpreted as broadly as allowed under the law.

Claims

1. A method for automatically translating a document from a first language to a second language comprising: receiving the document in the first language;processing the document to extract elements of information;translating the extracted elements of information into the second language using a first translation process; andtranslating a remainder of the document into the second language using a second, different translation process.

2. The method of claim 1, wherein the elements of information comprise names of named entities, and the first translation process comprises a transliteration process.

3. The method of claim 1, wherein the first translation process further comprises at least one of a statistical translation process, a transfer rule-based translation process, and an application of a bilingual name lexicon depending on a semantic type of the named entities.

4. The method of claim 1, wherein the elements of information comprise at least of one of dates, times, and amounts, and the first translation process comprises applying semantic rules for the second language associated with a corresponding class of information.

5. The method of claim 1, wherein the elements of information comprise core noun phrase descriptions and the first translation process comprises employing a mixture model to translate the extracted elements of information.

6. The method of claim 6, wherein employing the mixture model includes employing a statistical translation process combined with a dictionary translation process.

7. The method of claim 1, wherein the first process includes applying syntactic and semantic transfer rules.

8. The method of claim 1, wherein the second translation process includes employing a statistical translation process.

9. The method of claim 1, further comprising identifying a span of at least two words that are related, andtranslating the words of the span as a cluster.

10. The method of claim 1, further comprising merging the translated extracted elements of information with the translated remainder of the document.

11. The method of claim 10, wherein the merging results in a translated version of the document in the second language.

12. The method of claim 1, wherein the remainder of the document, prior to translation, includes placeholders for original locations of the extracted elements of information.

13. The method of claim 12, wherein the translated remainder of the document includes placeholders for insertion of the translated elements of information.

14. The method of claim 1, comprising: determining a plurality of potential translations of the elements of information and likelihood values for each respective potential element of information translation;determining a plurality of potential translations of the remainder of the document and likelihood values for each respective remainder translation; anddetermining likelihood values for combinations of respective element of information translations and remainder translations.

15. The method of claim 14, comprising determining the combination likelihood values by comparing the respective combinations to a model of the second language.

16. The method of claim 14, comprising aggregating the combination likelihood values with the respective potential element of information translation likelihood values and remainder translation likelihood values.

17. A method for automatically translating a document from a first language to a second language comprising: receiving the document in the first language,translating a first portion of the document to the second language using a first translation process,translating a second portion of the document to the second language using a second, different translation process, andmerging the translated first portion and translated second portion to form a translated version of the document in the second language.

18. The method of claim 17, comprising: extracting the first portion from the document; andreplacing the first portion in the document with placeholders to form the second portion.

19. The method of claim 17, wherein the first translation process is at least partially rule driven and the second translation process is statistics-based

20. A system for automatically translating a document from a first language to a second language, comprising: a first module for receiving the document in the first language and processing the document to extract elements of information;a second module for translating the extracted elements of information into the second language using a first translation process;a third module for translating a remainder of the document into the second language using a second translation process; anda fourth module for forming a translated version of the document by combining the translated elements of information with the translated remainder.

21. A computer-readable medium containing computer-executable instructions, which when executed by a computer carry out a method for translating a document from a first language to a second language, comprising: receiving the document in the first language;processing the document to extract elements of information;translating the extracted elements of information into the second language using a first translation process;translating a remainder of the document into the second language using a second translation process; andforming a translated version of the document by combining the translated elements of information with the translated remainder.

22. A method of training a machine translation system comprising: receiving a first corpus of text in a first language;receiving a second corpus of text, having the same content as the first corpus of text, in a second language;identifying elements of information in both the first and second corpora, wherein each identified element of information has a semantic type and includes a contiguous span of one or more words;determining an alignment of words between the first and second corpora such that all elements of information in the first corpus align with a contiguous span of words in the second corpus.

23. The method of claim 22, wherein the alignment is determined such that each element of information in the first corpus aligns with an element of information of in the second corpus having a same semantic type.

24. The method of claim 22, wherein, prior to determining the alignment, replacing identified elements of information in each corpus with placeholders.

25. The method of claim 24, wherein determining the alignment comprises aligning a placeholder in the first corpus with a placeholder in the second corpus and determining that the elements of information extracted from the first and second corpora for a given pair of aligned placeholders are aligned to each other.