Patent Application
20060009961
1. Technical Field
This invention relates to techniques in machine translation. More particularly, the invention relates to a method for decomposing prose elements in document processing.
2. Description of the Prior Art
As a research in linguistics has reported, length of sentence is of significance in reading comprehension. For example, when reading New York Times articles, the long sentences in the Editorial often cause havoc in processing. Human consciousness in processing language is just like a bird in flight—for a bird the more branches to have to perch on, the farther it can fly, and for human mind the more proper punctuation, the easier it moves on. For example, a five-word segment is easier to process than a ten-word segment, and simple sentences are easier to understand than complex sentences.
As being taught in grammar school, a complex sentence typically consists of main clause, co-ordinate clause(s), participle clause(s) and subordinate clause(s) in a number of combinations. Later in life, human being carefully adapts to parsing complex sentences. It would be advantageous that this internal process for decomposing complex sentences can be articulated so that it may be applied to machine translation techniques.
As the current state of art in machine translation, there is no capability to decompose complex sentences into simple segments which can be easily understood by the reader. The lack of this capability typically results in the translations that are virtually undecipherable.
The Paris MOU consists of 18 participating maritime Administrations and covers the waters of the European coastal States and the North Atlantic basin from North America to Europe. The Paris MOU aims at eliminating the operating of sub-standard ships through a harmonized system of port State control.
This search result consists of a description of Port State Control Paris MOU. As a Chinese reader can see, the machine-translated Chinese sentences 20 and 30 are virtually undecipherable because Chinese words all run into each other with no break. These translations lack the ability to segment the present participle clauses into understandable Chinese modules.
Researches in linguistics find that English and a number of other hypertactic languages are adorned with rich cohesive ties. For example, the follow sentences all have similar meanings but with different cohesive ties:
When the baby cried, the mother picked it up.
If the baby cried, the mother picked up.
Upon hearing the baby cried, the mother picked it up.
Judging from the fact that the baby cried, the mother picked it up.
At the same time, Chinese and other paratactic language emphasize on oral tradition and/or narratives in abundance, or they are highly developed prior to printing technology. Speakers in these paratactic languages can figure out just from the following two simple sentences:
The baby cried. The mother picked it up.
It is therefore desired a technology in machine translation which can pre-process complex sentences into manageable segments for ease of human understanding, preferably such technology should decompose complex sentences into simple sentences.
It is further desired that such technology is able to identify, isolate and strip out cohesive ties in comparatively more hypertactic language for the benefit of people more accustomed to paratactic language.
It is further desired that such technology can be applied to machine translation so that the comparatively independent linguistic components such as clauses and phrases can be translated to a second language, and the translated results in the second language can be easily understand by the speakers of the second language.
The present invention discloses a technology for decomposing prose elements in document processing. Grammar analysis of complex sentences can identify main clauses, embedded clauses, phrases, and cohesive ties linking the embedded clauses to the main clause. Cohesive ties are identified, isolated, and stripped down and a weighted punctuation for segmentation is applied to decompose complex sentences into simple sentences.
In one preferred embodiment of the invention, a language processing system is disclosed for pre-processing complex sentences in a first language into manageable segments for ease of human understanding. The language processing system at least includes: (a) a grammar analysis module for identifying main clause, embedded clauses, and all possible cohesive ties between the main clause and the embedded clauses; (b) a cohesive tie stripping module for filtering or stripping down the possible cohesive ties between the main clause and the embedded clauses; and (c) a punctuation module for applying a weighted punctuation for segmentation to decomposing said complex sentence into simple sentences.
In another preferred embodiment of the invention, a method for pre-processing complex sentences in a first language into manageable segments for ease of human understanding is disclosed. The method includes the steps of:
(a) performing grammar analysis on a complex sentence to identifying main clause, embedded clauses, and all possible cohesive ties between the main clause and the embedded clauses;
(b) stripping down the possible cohesive ties between the main clause and the embedded clauses; and
(c) applying a weighted punctuation for segmentation to decomposing the complex sentence into simple sentences.
With reference to the drawings, the present invention will now be described in detail with regard for the best mode and the preferred embodiments. In its most general form, the invention comprises a program storage medium readable by a computer, tangibly embodying a program of instructions executable by the computer to perform the steps necessary to pre-process complex sentences in a first language into manageable segments for ease of human understanding of the end-result machine translation in a second language.
The language processing system 110 preferably includes the following components: a grammar analysis module 112, a cohesive tie stripping module 114 for filtering or stripping down cohesive ties, and a punctuation module 116. The language processing system 110 is able to decompose complex sentences into simple sentences. A complex sentence typically consists of a main clause, one or more co-ordinate clauses, one or more participle clauses, one or more subordinate clauses, and phrases in a number of combinations. In this application, the term “phrases” may include, but not limited to, verb phrase, noun phrase, adjective phrase, prepositional phrase, postpositional phrase, adverbial phrases, temporal adverb, conjunction, idiom, cohesive tie, spatial adverb, and the like.
When a complex sentence enters the language processing system 110, the grammar analysis module 112 is first invoked. The grammar analysis module 112 identifies the main clause, embedded clauses, and all possible cohesive ties that grammatically connect the embedded clauses to the main clause. The language processing system 110 can optionally include a database of complex sentences 118. This database of complex sentences 118 can be used by the grammar analysis module 112 to identify the main clause, the embedded clauses and the cohesive ties there between.
The cohesive tie stripping module 114 is invoked next. When the cohesive tie stripping module 114 is invoked, all cohesive ties identified by the grammar analysis module 112 are stripped down.
After stripping down all possible cohesive ties between the main clause and the embedded clauses, the punctuation module 116 is invoked. The punctuation module 116 applies a weighted punctuation for segmentation to decompose the complex sentence into simple sentences. The punctuation module 116 can either change one or more commas to a period, or it can supply one or more commas or periods.
The language processing system 110 may optionally contains a translation module 120. When the translation module 120 is present in the language processing system 110, the translation module is invoked to translate the decomposed sentences into a second language.
Step 210: Perform grammar analysis on a complex sentence to identify main clause, embedded clauses, phrases and all possible cohesive ties between the main clause and the embedded clauses.
An optional database of complex sentences can be used in this step.
Step 220: Strip down all possible cohesive ties identified in step 210.
Step 230: Apply a weighted punctuation for segmentation to decompose the complex sentence into simple sentences.
The method can optionally include the step of:
Step 240: Translate the result simple sentences into a second language.
Step 250: Change one or more commas to a period.
Referring to
As/I am unaccustomed/to public speaking,/I/beg your indulgence/for my few inelegant words./
In step 315, the language processing system 110 performs grammar analysis on the complex sentence 310. After performing grammar analysis, the following grammar structures are identified:
The complex sentence also contains a comma 316.
In step 325, the cohesive ties 318 are stripped down. The word “As” is removed from the sentence in this step.
In step 335, a weighted punctuation for segmentation is applied to decompose the complex sentence into simple sentences. The comma 316 is replaced by a period 320.
This sentence after preprocessing will be decomposed into the following:
I am unaccustomed to public speaking. I beg your indulgence for my few inelegant words.
As one can see, the decomposed sentences are easier to comprehend that the original complex sentence.
Now referring back to
Step 260: Supply one or more commas or periods.
Now referring to
The earth contains a large number of metals which are useful to man.
In step 415, the language processing system 110 performs grammar analysis on the complex sentence 410. After performing grammar analysis, the following grammar structures are identified:
In step 425, the cohesive ties “which” 414 are stripped down. The word “which” is replaced with its equivalent part 418 in the main clause: “the metals” or “this”.
In step 435, a weighted punctuation for segmentation is applied to decompose the complex sentence into simple sentences. A period 420 is supplied to separate the main clause 412 and the embedded clause 416.
This sentence after preprocessing will be decomposed into the following:
The earth contains a large number of metals. The metals (that is) are useful to man.
As one can see, the decomposed sentences are easier to comprehend than the original complex sentence.
In the sections below, some additional examples are used to illustrate the operation of invention. These examples are for illustration purpose only. Without departing from the spirit of the invention, those skillful in the art will be able to apply the technology of the invention to pre-process any other kinds of complex sentences into manageable segments for ease of human understanding. Therefore the present invention will by no means be limited by these examples.
When the baby cried, the mother picked it up.
The comma in this sentence will be replaced by a period. This sentence will be decomposed into the following simple sentences:
The baby cried. The mother picked it up.
As much as John wanted to help Mary, given the situation, she would not dream of asking.
Each of the commas in this sentence will be replaced by a period. This sentence will be decomposed into the following simple sentences:
John wanted to help Mary. Mary was given the situation. Mary would not dream of asking.
One common grammatical phenomenon often presents in complex sentences is ellipsis. Ellipsis is a common strategy of rhetoric in English and other hypertactic languages. It is the omission of one or more words that are obviously understood but that must be supplied to make a construction grammatically complete. When ellipsis is present in a complex sentence, the omitted words are typically added back or copied to make the sentence grammatically complete. This is typically performed during the step of grammatical analysis. The completed sentence is then decomposed into simple sentences as described above. The following are some examples illustrating how complex sentences with ellipsis are decomposed according to the method of the invention.
Mary will prepare a dish for the party if John will.
For preprocessing before translation to another language, the words omitted by ellipsis will be added back or copied to make the above sentence complete and a comma will be added to separate the main clause and the subordinate clause. This sentence after preprocessing will be decomposed into the following:
Mary will prepare a dish for the party, if John will prepare a dish for the party.
I am sad if you are.
For preprocessing before translation to another language, the words omitted by ellipsis will be added back or copied to make the above sentence complete and a comma will be added to separate the main clause and the subordinate clause. This sentence after preprocessing will be decomposed into the following:
I am sad, if you are sad.
Jerry will write to the Congressman if Sam will too.
For preprocessing before translation to another language, the words omitted by ellipsis will be added back or copied to make the above sentence complete and a comma will be added to separate the main clause and the subordinate clause. This sentence after preprocessing will be decomposed into the following:
Jerry will write to the Congressman, if Sam will write to the Congressman too.
John will pay for the food if Mary will for the wine.
For preprocessing before translation to another language, the words omitted by ellipsis will be added back or copied to make the above sentence complete and a comma will be added to separate the main clause and the subordinate clause. This sentence after preprocessing will be decomposed into the following:
John will pay for the food, if Mary will pay for the wine.
John took the money because Mary would not take it.
For preprocessing before translation to another language, the words omitted by ellipsis will be added back or copied (through IT regeneration rule) to make the above sentence complete and a comma will be added to separate the main clause and the subordinate clause. This sentence after preprocessing will be decomposed into the following:
John took the money, because Mary would not take the money.
I'll have a soda if you'll have one.
For preprocessing before translation to another language, the words omitted by ellipsis will be added back or copied (through a regeneration rule) to make the above sentence complete and a comma will be added to separate the main clause and the subordinate clause. This sentence after preprocessing will be decomposed into the following:
I'll have a soda, if you'll have a soda.
An adjectival complex sentence consists of a main clause and a dependent adjectival clause. The following are some examples illustrating how adjectival complex sentences are decomposed according to the method of the invention.
Gardens untended by owners will be taken over by the neighbors.
For preprocessing before translation to another language, the words omitted by ellipsis will be added back to make the above sentence complete.
Gardens that are untended by owners will be taken over by the neighbors.
A period will be added to separate this complex sentence into two simple sentences. The original sentence after preprocessing will be decomposed into the following:
There are gardens untended by owners. They will be taken over by the neighbors.
No one decent will accept such money.
For preprocessing before translation to another language, the words omitted by ellipsis will be added back to make the above sentence complete.
No one who is decent will accept such money.
A period will be added to separate this complex sentence into two simple sentences. Notice here a negation transformation rule is applied. The original sentence after preprocessing will be decomposed into the following:
One is decent. One will not accept such money.
The fortunate in this country should share their wealth.
A period will be added to separate this complex sentence into two simple sentences. The original sentence after preprocessing will be decomposed into the following:
There are fortunate people in this country. They should share their wealth.
Complex sentences can also be constructed when two or more sentences are equal importance are connected with coordinating conjunctions. This is called coordination. The common coordinating conjunctions typically comprise the following: “and”, “or”, “but”, “nor”, “yet”, “for”, “so”, “therefore” and “then”.
When translating complex sentences from English into other languages, it is often unnecessary to translate coordinating conjunctions. In fact, it would be awkward to include coordinating conjunctions in a set of sentences. The following are some examples illustrating how complex sentences with coordinating conjunctions are decomposed according to the method of the invention.
She saved so that she could buy candy and so that she could buy clothing.
A period will be added to separate this complex sentence into two simple sentences. The original sentence after preprocessing will be decomposed into the following:
She saved so that she could buy candy. She could buy clothing.
They are living in Tuscany or they are spending a vacation there.
A period will be added to separate this complex sentence into two simple sentences. The original sentence after preprocessing will be decomposed into the following:
They are living in Tuscany. Or they are spending a vacation there.
Alternatively, this sentence can be further decomposed into two simple sentences separated by a comma:
They are living in Tuscany, or they are spending a vacation there.
I may see you tomorrow or may phone later in the day.
A period will be added to separate this complex sentence into two simple sentences. The original sentence after preprocessing will be decomposed into two sentences showing below:
I may see you tomorrow. Or I may phone later in the day.
John never smoked cigarettes or drank alcohol.
A period will be added to separate this complex sentence into two simple sentences. The original sentence after preprocessing will be decomposed into two sentences showing below:
John never smoked cigarettes. John never drank alcohol.
Some complex sentences may include an apposition, which is a grammatical construction in which two usually adjacent nouns having the same referent stand in the same syntactical relation to the rest of a sentence. A complex sentence with an apposition can typically be decomposed to two or more simple sentences.
The following are some examples illustrating how complex sentences with an apposition are decomposed according to the method of the invention.
An unusual present was given to him for his birthday, a book on ethics.
The apposition will be completed to a full sentence and the main clause will become another sentence, and a period will be added to separate these two sentences. The original sentence after preprocessing will be decomposed into two sentences showing below:
An unusual present was given to him for his birthday. It was a book on ethics.
Pauline Kale, the distinguished film critic, died in her sleep last night.
The apposition will be completed to a full sentence and the main clause will become another sentence, and a period will be added to separate these two sentences. The original sentence after preprocessing will be decomposed two sentences showing below:
Pauline Kale was the distinguished film critic.
She died in her sleep last night.
That he didn't notice the car till too late is unsatisfactory.
The apposition will be completed to a full sentence and a period is added to separate the new sentence, and the main clause will become another sentence, and a period will be added to separate these two sentences. The original sentence after preprocessing will be decomposed into two sentences showing below:
He didn't notice the car till too late. That is unsatisfactory.
Some complex sentences may contain a non-restrictive modifier. A non-restrictive modifier is a phrase or clause that does not restrict or limit the meaning of the word it is modifying. It is, in a sense, interrupting material that adds extra information to a sentence. Even though removing the non-restrictive element would result in some loss of meaning, the sentence would still make sense without it.
Non-restrictive modifiers are often preceded by one of these markers: “that is to say”, “that is”, “i.e.”, “namely”, “viz.”, “to wit”, “in other words”, “or”, “or rather”, “or better”, “and”, “as follows”, “for example”, “for instance” , “e.g.”, “say”, “including”, “included”, “such”, “especially”, “particularly”, “in particular”, “notably”, “chiefly”, “mainly”, and “mostly”. Some non-restrictive modifiers may be recognized by the absence of article.
When decomposing complex sentences with non-restrictive modifiers, the non-restrictive modifiers often take weighted segmentation. The following are some examples illustrating how complex sentences with non-restrictive modifiers are decomposed according to the method of the invention.
Johnny, lead singer of the group, just underwent an operation.
For preprocessing before translation to another language, the words omitted by ellipsis will be added back to make the above sentence complete.
Johnny, who was the lead singer of the group, just underwent an operation.
The non-restrictive will be completed to a full sentence and the main clause will become another sentence and a period will be added to separate these two sentences. The original sentence after preprocessing will be decomposed into two sentences showing below:
Johnny was the lead singer of the group. He just underwent an operation.
The two men, one a Norwegian and the other a Dane, were awarded medals.
For preprocessing before translation to another language, the words omitted by ellipsis will be added back, or copied, or regenerated, to make the above sentence complete.
The two men, one being a Norwegian and the other being a Dane, were awarded medals.
The two men, one of whom being a Norwegian, and the other of whom being a Dane, were awarded medals.
The non-restrictive will be completed to two full sentences and the main clause will become another sentence, and two periods will be added to separate these sentences. The original sentence after preprocessing will be decomposed into three sentences showing below:
Two men were awarded medals. One of them was a Norwegian. The other was a Dane.
As one can see from the above examples, the decomposed sentences are easier to comprehend that their corresponding original complex sentences.
In addition of the above, syntactic devices used for connecting sentences like the following can be also stripped down with the method of the invention:
1. Time and place relaters;
2. Logical connectors;
3. Substitution;
4. Discourse reference;
5. Comparison;
6. Ellipsis;
7. Structural parallelism.
Although the invention is described herein with reference to the preferred embodiment, one skilled in the art will readily appreciate that other applications may be substituted for those set forth herein without departing from the spirit and scope of the present invention.
Accordingly, the invention should only be limited by the claims included below.
This application claims benefit to the U.S. patent application Ser. No. 449740 filed on 29 May 2003, and is related to U.S. Pat. No. 6,604,101, the contents of which are incorporated in their entirety by reference herein.
