Embodiments of the present invention relate to a method and system for creating content for bilingual dictionaries in the form of Internet, electronic and paper dictionaries, compiling dictionaries, glossaries, encyclopedias, and other types of reference materials.
One of a dictionary user's basic needs is to find an appropriate translation for a word being looked up (hereinafter, the “lookup word”). The lookup word may be a new word that the user has encountered, for example while reading. Alternatively, the look word may be a word in a source language (e.g. English) that the user wishes to have translated into a corresponding word in a target language (e.g. Russian). Bilingual dictionaries usually comprise dictionary entries, each providing many translations (lexical meanings), examples, synonyms and other information, corresponding to a lookup word,
One of the most challenging tasks for a dictionary producer is to help the dictionary reader/user find a good translation and all the relevant information about a lookup word. For example, the word “file” has several homonyms and several lexical meanings, and depending on context it may be translated into different parts of speech, and each part of speech may have several radically different meanings and different syntactical models of usage. For defining such syntactical models of lexical meanings in the dictionaries an appropriate markup form may be used which is useful when a user wants to translate something into a foreign language. In this case the dictionary can show, for example, in what discourse or subject field a lookup word is usually used, what preposition should be used before a noun, or what preposition should be used after a verb, in what collocations the lookup word is usually used, is a direct object required by the verb and where in the sentence this direct object must be placed, etc. Usage examples that illustrate the lexical meanings can greatly help the user to select the most appropriate translation, when he is reading the text in a foreign language or when he is trying to translate a collocation into a foreign language.
Embodiments of the present invention disclose computer-implemented method for generating examples for electronic dictionaries to serve as an aid to translation between languages, comprising: for at least one dictionary entry comprising a headword Wj in a source language and at least one translation Tj1, Tj2, . . . Tjn for the headword Wj in a target language: generating a first set comprising possible forms for the headword Wj in the source language and a second set comprising possible forms for each translation Tj1, Tj2, . . . Tjn, in the target language; searching a corpus of translations for at least one translation sentence pair that includes the headword Wj, or one of its generated forms, in a first part of the pair, and a translation Tjn or one of its generated forms, in a second part of the pair; and providing each translation sentence pair to a user. In another embodiment, the corpora are preliminarily indexed, and searching on the basis of the index is implemented.
Embodiments of the present invention also disclose a system for performing the aforesaid computer-implemented method.
Advantageously, embodiments of the present invention takes advantage of the large body (corpora) of translations that have been accumulated to (a) automatically retrieve translations for entire sentences or part of sentences, (b) automatically identify the most frequent word combinations, and (c) associate the translations thus obtained with a lexical meaning.
Other advantages of embodiments of the present invention will be apparent from the detailed description below.
Broadly, embodiments of the present invention disclose a content creation method for creating dictionary content based on bilingual or multilingual corpora of translations. Advantageously, in one embodiment, the dictionary content that is created comprises examples that serve as an aid to understanding dictionary entries. Embodiments of the present invention also disclose a content creation system to implement the content creation method of the present invention.
The structure of a dictionary entry in a bilingual dictionary may be very complex, as can be noted from the exemplar dictionary entry for the word “file” provided in
The dictionary entry of
The second homonym II “a line of people or things one behind another” may be general, but if the text being translated with the aid of the dictionary contains terms related to “military” or “chess”, these meanings should be selected. The third homonym II is very specific, and if the translated text contains terms related to “metalwork”, “tools”, “instrument”, this meaning should be selected.
In addition to translations, the dictionary entry of
A bilingual corpus of translations is made up of a multitude of sentences in the source and target languages that are translations of one another. With regard to such a bilingual corpus of translations sentences on the left are assumed to be in the source language, whereas sentences on the right are assumed to be in the target language. These assumptions are intended to be non-limiting as in other embodiments, sentences in the source language may be assumed to be on the right, whereas sentences in the target language may be assumed to be on the left. The same corpus of translations may be used to compile either a Lang1-to-Lang2 dictionary or a Lang2-to-Lang1 dictionary. A corpus of translations may be additionally marked up. There are corpora with grammatical, syntactic, and semantic mark-up, where sentence elements (words) are provided with part-of-speech tags, syntactic function tags (subject, predicate, object, etc.), or semantic tags (the domain of use, etc.). The availability of such mark-up and the ability to use such mark-up can greatly improve the relevance of search results and the quality of the dictionary.
In one embodiment, the content creation method of the present invention includes a concordance search technique wherein a bilingual (N-lingual) corpus of translations may be used to search in the left portion and in right portion of said corpus simultaneously. Additionally, the concordance search method may also be used to search by grammatical meanings, labels, etc.
1. At step 220, in one embodiment, a morphology engine uses morphological descriptions (215) to generate all the possible forms of the headword Wj in the source language and all the forms for each of the Tj1, Tj2, . . . Tjn . . . in the target language. In another embodiment, all lexemes of the corpora are preliminary indexed. For this purposes the index may be specific, for example, the index may contain the number of the pair of sentences and the number of the word.
2. At step 240, for each Tji, the system looks for a corpus item that has the word Wj, possibly in one of the word forms, in the left part, and the Tji, possibly in one of the word forms, in the right part. In one embodiment, searching procedure may be implemented on the basis of any method of search. In another embodiment, searching procedure may be based on the index.
3. The found sentence pairs are placed (260) into the entry Wj immediately after the corresponding lexical meaning Tji.
In one embodiment, the above procedure may be used once to add examples to all entries by consecutively (or simultaneously) going through all the headwords W1, W2, . . . Wj, . . . and their translations Tj1, Tj2, . . . . In another embodiment, the procedure may be used dynamically and interactively, for example, if the user wishes to see all instances where the word Wj is translated with Tji, this may be achieved by the user simply clicking on or selecting Tji.
It may happen that for some entries and translations too many examples are found. In that case, the examples may be filtered and arranged. In one embodiment, they may be arranged, for example, by frequency, by integral rankings supplied by users or experts, by rank of the corpus, or by other criteria. In one embodiment, very infrequent or specific examples may be rejected. In another embodiment examples may be labeled as medical, botanical or navigation terms etc. In one embodiment, the corpora, where examples are searched, may be ranked by authority, by experts estimations (assessment), by volume etc. Accordingly, the examples found in a particular corpus may inherit the rating of that corpus. In addition, a lexicographer or expert may be involved in manually selecting the best dictionary examples.
In one embodiment, the multitude of examples may be filtered (step 250) by labels, for example, by part-of-speech labels, if the corpus includes grammatical mark-up or a very simple context analyzer to deduce the part of speech from the context (e.g. a preceding article) or from the word form specific to a part of speech. For example, the presence of a preposition, article, particle or other specific word before or after the translated word may govern the selection of the part of speech, but “to” may be a preposition, but may indicate an infinitive of a verb. In such indistinct cases the other indications may be used.
In one embodiment, the examples may also be filtered by special labels that indicate that a given lexical meaning belongs to a specific subject domain. For example, referring to
Advantageously, examples in a dictionary entry should differ from one another. To this end, in one embodiment, a very simple analyzer may be used to select unique examples. The analyzer may function by computing a similarity between two examples. In one embodiment, any heuristic methods may be used. If an example is similar to one already selected, then it is not added, as its effect would merely be cumulative. In another embodiment, more complex analysis algorithms may be employed, which analyze the surface syntactic structure of the example sentences or which compare the deep structures of the example sentences. Such analyzer that can determine not only syntactical structure of the sentence, but also its semantic structure. When such analyzer is used, the examples with the same surface syntactic structures may be identified as non-unique.
In one embodiment, under certain conditions, examples added to the dictionary may be semantically expanded. In this case, those sentences may be added to the dictionary which do not contain the translation Tji but which contain a word semantically close to Tji or synonymous with Tji. For example, two examples may have the same deep semantic structures, but a bit different surface syntactic structures. Then, the differing parts of sentences may be considered as possible translations of each other.
In one embodiment, each lexical meaning in the bilingual dictionary may be connected to lexical-semantic dictionary. Each lexical meaning in the lexical-semantic dictionary belongs to some semantic class. The semantic classes are semantic notions (semantic entities) and named semantic classes arranged into semantic hierarchy—hierarchical parent-child relationships similar to a tree. In general, a child semantic class inherits most properties of its direct parent and all ancestral semantic classes. For example, semantic class SUBSTANCE is a child of semantic class ENTITY and the parent of semantic classes GAS, LIQUID, METAL, WOOD_MATERIAL, etc.
The semantic hierarchy is a universal, language-independent structure, and the semantic classes may include lexical meanings of various languages, which have some common semantic properties and may be attributed to the same notion, phenomenon, entity, situation, event, object type, property, action, and so on. Semantic classes may include many lexical meanings of the same language, which differ in some aspects and which are expressed by means of distinguishing semantic characteristics (semantemes). Semantemes express various properties of objects, conditions, and processes, which may be described in the language-independent semantic structure and expressed in natural languages grammatically and syntactically (for example, number, gender, aspect and tense of actions, degree of definiteness, modality, etc.), or lexically. So, lexical meanings are provided with distinguishing semantemes.
The system of semantemes includes language-independent semantic attributes, which express not only semantic characteristics but also stylistic, pragmatic and communicative characteristics. Some semantemes can be used to express an atomic meaning which finds a regular grammatical and/or lexical expression in a language. For example, the semantemes may describe specific properties of objects (for example, “being flat” or “being liquid”) and are used in the descriptions as restriction for deep slot fillers (for example, for the verbs “face (with)” and “flood”, respectively). The other semantemes express the differentiating properties of objects within a single semantic class, for example, in the semantic class HAIRDRESSER the semanteme <<RelatedToMen>> is assigned to the lexical meaning “barber”, unlike other lexical meanings which also belong to this class, such as “hairdresser”, “hairstylist”, etc.
Lexical meanings may be provided by pragmatic description, which allows the system to assign a corresponding theme, style or genre to texts and objects of the semantic hierarchy. For example, “Economic Policy”, “Foreign Policy”, “Justice”, “Legislation”, “Trade”, “Finance”, etc. Pragmatic properties can also be expressed by semantemes.
Each lexical meaning in the lexical-semantic dictionary has its surface (syntactical) model which includes one or more syntforms, as well as, idioms and word combinations with the lexical meaning. Syntforms may be considered as “patterns” or “frames” of usage. Every syntform may include one or more surface slots with their linear order description, one or more grammatical values expressed as a set of grammatical characteristics (grammemes), and one or more semantic restrictions on surface slot fillers. Semantic restrictions on a surface slot filler are a set of semantic classes, whose objects can fill this surface slot.
Thus, the fact that the Wj and Tji belong to the same semantic class and have the same semantemes and pragmatic descriptions may be a significant indication in favor of adding the example into the dictionary. The idioms and word combinations with the lexical meaning which have been assigned to the lexical meanings may also be added as examples into a bilingual dictionary. Lexical meanings which are semantically close to Tji can be found automatically in the same semantic class which contains the W and Tji(for example, derivatives of Tji or meanings of the same part of speech with the same or similar set of semantemes or labeled as synonymous).
In one embodiment, the set of examples thus obtained are added directly into the dictionary. In another embodiment, the set of examples is generated and displayed to the user on demand, for example when the user clicks the corresponding button or clicks on the corresponding lexical meaning. In still another embodiment, the set of examples thus obtained may used as raw material by a lexicographer to compile a new dictionary.
In still another embodiment, the set of examples thus obtained which includes translations semantically close to Tji, may be added directly into the dictionary next to the lexical meaning Tji. In another embodiment, the set of examples, which includes translations semantically close to Tji can be suggested to a lexicographer for further processing. In still another embodiment, a representative set of examples which contain semantically close or completely new yet frequent translations may serve as grounds to add a new lexical meaning into the dictionary, either manually by a lexicographer or automatically. The aforementioned methods of analysis that analyze the surface syntactic structures of example sentences or compare the deep structures of the example sentences can identify such semantically close or completely new translations and add them into the entry automatically.
The numerous large corpora of translations that have been accumulated make it possible to retrieve translations for entire sentences or part of sentences, but they cannot be used to automatically identify the most frequent word combinations and to associate the translations thus obtained with a lexical meaning. Advantageously, embodiments of the present invention solve this problem.
Additionally, a lexicographer or any other user need to make a special query only if they think that these word combinations may be available in the corpora. If there are too many examples, the availability of such corpora is of no help to lexicographers, who need identify useful examples when compiling a dictionary, nor to users who translate texts using a dictionary. Advantageously, embodiments of the present invention automatically generate examples, as described above.
Referring to
The hardware 300 also typically receives a number of inputs and outputs for communicating information externally. For interface with a user or operator, the hardware 300 may include one or more user input devices 306 (e.g., a keyboard, a mouse, imaging device, scanner, etc.) and a one or more output devices 308 (e.g., a Liquid Crystal Display (LCD) panel, a sound playback device (speaker). To embody the present invention, the hardware 300 must include at least one screen device.
For additional storage, the hardware 300 may also include one or more mass storage devices 310, e.g., a floppy or other removable disk drive, a hard disk drive, a Direct Access Storage Device (DASD), an optical drive (e.g. a Compact Disk (CD) drive, a Digital Versatile Disk (DVD) drive, etc.) and/or a tape drive, among others. Furthermore, the hardware 300 may include an interface with one or more networks 312 (e.g., a local area network (LAN), a wide area network (WAN), a wireless network, and/or the Internet among others) to permit the communication of information with other computers coupled to the networks. It should be appreciated that the hardware 300 typically includes, suitable analog and/or digital interfaces between the processor 302 and each of the components 304, 306, 308, and 312 as is well known in the art.
The hardware 300 operates under the control of an operating system 314, and executes various computer software applications, components, programs, objects, modules, etc. to implement the techniques described above. In particular, the computer software applications will include the client dictionary application, in the case of the client user device 102. Moreover, various applications, components, programs, objects, etc., collectively indicated by reference 316 in
In general, the routines executed to implement the embodiments of the invention may be implemented as part of an operating system or a specific application, component, program, object, module or sequence of instructions referred to as “computer programs.” The computer programs typically comprise one or more instructions set at various times in various memory and storage devices in a computer, and that, when read and executed by one or more processors in a computer, cause the computer to perform operations necessary to execute elements involving the various aspects of the invention. Moreover, while the invention has been described in the context of fully functioning computers and computer systems, those skilled in the art will appreciate that the various embodiments of the invention are capable of being distributed as a program product in a variety of forms, and that the invention applies equally regardless of the particular type of computer-readable media used to actually effect the distribution. Examples of computer-readable media include but are not limited to recordable type media such as volatile and non-volatile memory devices, floppy and other removable disks, hard disk drives, optical disks (e.g., Compact Disk Read-Only Memory (CD-ROMs), Digital Versatile Disks (DVDs), flash memory, etc.), among others. Another type of distribution may be implemented as Internet downloads.
While certain exemplary embodiments have been described and shown in the accompanying drawings, it is to be understood that such embodiments are merely illustrative and not restrictive of the broad invention and that this invention is not limited to the specific constructions and arrangements shown and described, since various other modifications may occur to those ordinarily skilled in the art upon studying this disclosure. In an area of technology such as this, where growth is fast and further advancements are not easily foreseen, the disclosed embodiments may be readily modifiable in arrangement and detail as facilitated by enabling technological advancements without departing from the principals of the present disclosure.
This application is a continuation-in-part of U.S. Ser. No. 11/548,214, filed on Oct. 10, 2006, the entire specification of which is incorporated herein by reference.
Number | Name | Date | Kind |
---|---|---|---|
5268839 | Kaji | Dec 1993 | A |
5289376 | Yokogawa | Feb 1994 | A |
5386556 | Hedin et al. | Jan 1995 | A |
5418717 | Su et al. | May 1995 | A |
5426583 | Uribe-Echebarria Diaz De Mendibil | Jun 1995 | A |
5677835 | Carbonell et al. | Oct 1997 | A |
5678051 | Aoyama | Oct 1997 | A |
5687383 | Nakayama et al. | Nov 1997 | A |
5715468 | Budzinski | Feb 1998 | A |
5768603 | Brown et al. | Jun 1998 | A |
5787386 | Kaplan et al. | Jul 1998 | A |
5787410 | McMahon | Jul 1998 | A |
5794050 | Dahlgren et al. | Aug 1998 | A |
5826219 | Kutsumi | Oct 1998 | A |
5884247 | Christy | Mar 1999 | A |
6006221 | Liddy et al. | Dec 1999 | A |
6055528 | Evans | Apr 2000 | A |
6064951 | Park et al. | May 2000 | A |
6076051 | Messerly et al. | Jun 2000 | A |
6081774 | de Hita et al. | Jun 2000 | A |
6182028 | Karaali et al. | Jan 2001 | B1 |
6233544 | Alshawi | May 2001 | B1 |
6233545 | Datig | May 2001 | B1 |
6243670 | Bessho et al. | Jun 2001 | B1 |
6246977 | Messerly et al. | Jun 2001 | B1 |
6275789 | Moser et al. | Aug 2001 | B1 |
6356864 | Foltz et al. | Mar 2002 | B1 |
6381598 | Williamowski et al. | Apr 2002 | B1 |
6393388 | Franz et al. | May 2002 | B1 |
6393389 | Chanod et al. | May 2002 | B1 |
6442524 | Ecker et al. | Aug 2002 | B1 |
6463404 | Appleby | Oct 2002 | B1 |
6490548 | Engel | Dec 2002 | B1 |
6601026 | Appelt et al. | Jul 2003 | B2 |
6604101 | Chan et al. | Aug 2003 | B1 |
6622123 | Chanod et al. | Sep 2003 | B1 |
6651220 | Penteroudakis et al. | Nov 2003 | B1 |
6778949 | Duan et al. | Aug 2004 | B2 |
6871199 | Binnig et al. | Mar 2005 | B1 |
6901402 | Corston-Oliver et al. | May 2005 | B1 |
6917920 | Koizumi et al. | Jul 2005 | B1 |
6928448 | Franz et al. | Aug 2005 | B1 |
6937974 | d'Agostini | Aug 2005 | B1 |
6947923 | Cha et al. | Sep 2005 | B2 |
6965857 | Decary | Nov 2005 | B1 |
6983240 | Ait-Mokhtar et al. | Jan 2006 | B2 |
7200550 | Menezes et al. | Apr 2007 | B2 |
7263488 | Chu et al. | Aug 2007 | B2 |
7403889 | Abe et al. | Jul 2008 | B2 |
7475015 | Epstein et al. | Jan 2009 | B2 |
7483828 | Abir | Jan 2009 | B2 |
7565281 | Appleby | Jul 2009 | B2 |
7664628 | Kojo | Feb 2010 | B2 |
7672831 | Todhunter et al. | Mar 2010 | B2 |
7739102 | Bender | Jun 2010 | B2 |
8041697 | Rayner et al. | Oct 2011 | B2 |
8078450 | Anisimovich et al. | Dec 2011 | B2 |
8145473 | Anisimovich et al. | Mar 2012 | B2 |
8214199 | Anismovich et al. | Jul 2012 | B2 |
8229730 | Van Den Berg et al. | Jul 2012 | B2 |
8229944 | Latzina et al. | Jul 2012 | B2 |
8271453 | Pasca et al. | Sep 2012 | B1 |
8285728 | Rubin | Oct 2012 | B1 |
8301633 | Cheslow | Oct 2012 | B2 |
8402036 | Blair-Goldensohn et al. | Mar 2013 | B2 |
8533188 | Yan et al. | Sep 2013 | B2 |
8548951 | Solmer et al. | Oct 2013 | B2 |
8577907 | Singhal et al. | Nov 2013 | B1 |
20030176999 | Calcagno et al. | Sep 2003 | A1 |
20040122656 | Abir | Jun 2004 | A1 |
20050155017 | Bertis et al. | Jul 2005 | A1 |
20050209844 | Wu et al. | Sep 2005 | A1 |
20090063472 | Pell et al. | Mar 2009 | A1 |
20100057438 | Zhanyi et al. | Mar 2010 | A1 |
20110055188 | Gras | Mar 2011 | A1 |
20110258229 | Ni et al. | Oct 2011 | A1 |
20110301941 | De Vocht | Dec 2011 | A1 |
20120023104 | Johnson et al. | Jan 2012 | A1 |
20120030226 | Holt et al. | Feb 2012 | A1 |
20120131060 | Heidasch et al. | May 2012 | A1 |
20120197885 | Patterson | Aug 2012 | A1 |
20120203777 | Laroco, Jr. et al. | Aug 2012 | A1 |
20120221553 | Wittmer et al. | Aug 2012 | A1 |
20120246153 | Pehle | Sep 2012 | A1 |
20120296897 | Xin-Jing et al. | Nov 2012 | A1 |
20130013291 | Bullock et al. | Jan 2013 | A1 |
20130054589 | Cheslow | Feb 2013 | A1 |
20130091113 | Gras | Apr 2013 | A1 |
20130138696 | Turdakov et al. | May 2013 | A1 |
20130185307 | El-Yaniv et al. | Jul 2013 | A1 |
20130254209 | Kang et al. | Sep 2013 | A1 |
20130282703 | Puterman-Sobe et al. | Oct 2013 | A1 |
20130311487 | Moore et al. | Nov 2013 | A1 |
20130318095 | Harold | Nov 2013 | A1 |
20140012842 | Yan et al. | Jan 2014 | A1 |
Number | Date | Country |
---|---|---|
2400400 | Dec 2001 | EP |
2011160204 | Dec 2011 | WO |
Entry |
---|
Hutchins, Machine Translation: Past, Present, Future, Ellis Horwood, Ltd., Chichester, UK, 1986. |
Bolshakov, I.A. “Co-Ordinative Ellipsis in Russian Texts:Problems of Description and Restoration” Proceedings of the 12th conference on Computational linguistics—vol. 1, pp. 65-67. Association for Computational Linguistics 1988. |
Mitamura, T et al. “An Efficient Interlingua Translation System for Multi-lingual Document Production,” Proceedings of Machine Translation Summit III, Washington DC, Jul. 2-4, 1991. |
Mitamura, T., et al. “An Efficient Interlingua Translation System for Multi-lingual Document Production,” Proceedings of Machine Translation Summit III, Washington, DC, Jul. 2-4, 1991. |
Bolshakov, I.A. “Co-Ordinative Ellipsis in Russian Texts: Problems of Description and Restoration,” Proceedings of the 12th Conference on Computational Linguistics—vol. 1, pp. 65-67. Association for Computational Linguistics. |
Bolshakov, Igor A. , “Co-Ordinative Ellipsis in Russian Texts: Problems of Description and Restoration”, Proceedings of the 12th conference on Computational linguistics, 1988, pp. 65-67, vol. 1, VINITI, Academy of Sciences of USSR, Moscow, 125219, USSR. |
Hutchins, Mashine Translation: past, present, future, 1986, 382 pp, Ellis Horwood:Chichester, UK. |
Mitamura, Teruko et al., “An Efficient Interlingua Translation System for Multi-Lingual Document Production”, Jul. 2-4, 1991, Proceedings of Machine Translation Summit III, Washington DC. |
Number | Date | Country | |
---|---|---|---|
Parent | 11548214 | Oct 2006 | US |
Child | 13004019 | US |