1. Field of the Invention
The present invention relates to a system and method for converting at least one of a at least one English sentence(s) and English Sign Language (ESL) into American Sign Language (ASL) that when signed could be understood by deaf persons who use or learn to use ASL. The innovation described herein could be utilized in conjunction with the Signtel Interpreter, by its own or in conjunction with other interfaced signing products.
It further enables to convert other languages into ASL.
2. Prior Art
Machine translation of English voice or text to Sign Language is prior art of Liebermann in the Telephone For The Deaf, now Ser. No. 5,982,853 and RE41,002 and also in Liebermann et al. now U.S. Pat. No. 7,774,194. However, in such prior art the translation is into English Sign Language and not American Sign Language, in the sense that the signs utilized are ASL signs and except the ability to ignore the auxiliaries as is done in ASL, the sentence structure is that of ESL. The present disclosure relates to converting said ESL, English text, other sentences, Touch Language or tactile language such as Global Touch Language into ASL.
In accordance with the instant disclosure, there is described a system and method for converting an English sentence into an American Sign Language sentence (ASL) that when so signed may be understood by deaf and other persons—skilled in understanding ASL, who use or learn to use the American Sign Language. It also enables moving on consecutively to the next sentence and the one after that, performing the same process. The innovation described herein may be utilized in conjunction with the Signtel Interpreter, by its own or in conjunction with other interfacing signing products that convert text to ESL, as well as any text that needs to be converted to ASL.
In one embodiment, an English sentence is submitted to the ASL Converter and comes out as an output as an ASL sentence or as close as possible to it, depending on some facial expressions. Namely, the grammar is the same, though some of the images may not, depending on the specific situation. The converter system is divided into two segments: the pre-processing segment that examines the words in the submitted text and the other segment containing all the rest of the process for the conversion and displaying the content in ASL. When said system is utilized with a signing product, such as the Signtel Interpreter that is based on a U.S. Pat. No. 7,774,194 by Liebermann et al. the system emulates the pre-processor to the Signtel Interpreter, with or without additional signing provided. It may also be functioning independently of the Signtel Interpreter, providing separate signing (i.e., film clips) for the system.
Signing may appear as filmed or as animated, though in the preferred embodiment it will appear as film clips. Said film clips are subsequently fully integrated into a seamless sentence.
The same type of processes may be utilized for conversion from English to other Sign Languages, such as in creating a BSL Converter where English sentences are converted to the British Sign Language, abbreviated as BSL. Other possibilities and embodiments as well as those that ensue from this development are to be considered different embodiments and therefore are also embraced here. Such embodiments are not necessarily English based, and may allow for other languages to be converted to or from English, whereas the same principles may be utilized for translation into Sign language.
A sentence is separated into two parts, wherein one part comprises the linguistic part, and the other part contains all the relevant transitional motions of the signing person in the filmed clips. The transitional motions are divided into separate classes wherein one class contains ASL transitional motions and the other class contains operational ASL transitional motions. Said transitional motions are different from those referred to in earlier said work by Liebermann et al.
The process comprising the means for achieving the conversion into ASL in the case of the already existing Signtel Interpreter product can be summarized by the following:
It should be noted that some of the new “transitions” that would need to be added require changes in the existing videos utilized in the patent of Liebermann et al., containing facial features during signing. Any such facial features that do not already exist in the said Signtel Interpreter may need to be added and such changes may be accomplished using the technology of Adobe's “After Effects” software. Further, an interim continuation-in-part for said patent of Liebermann et al. comprising an upgrade of facial features may be appropriate and as such is incorporated herein by reference.
Turning now to
Procedure
The following steps comprise the procedure of the said translation.
1. Turn all passive tense to active tense
2. Identify the person doing the action
3. Identify the action (for the verb)
4. Identify the noun as it will be signed twice
5. Identify the personification
6. Reorganize the sentence to be by ASL grammar
Sentence Structure
Operational ASL Transitions
The following may be instituted:
At least one of the following databases may provide input for the conversion process:
Verbs list
Nouns list
Adjectives list
Adverbs list
Pronouns list
Question List
Yes/No Question list
Time List
Affirmation List
The ASL Sentence:
The ASL sentence may be built from the English sentence by rearranging the components of the English sentence to look as follows:
This arrangement (or mode) of an ASL sentence may constitute the default case, where the user may choose a different form that may be provided by the invention such as one of the following provided options, or any another preferred option:
Further, an option may be provided to the user whereby for example a list of styles are provided by way of examples and allows the user to click on the preferred ASL style, thereby creating automatically a converter suitable for the personal needs o the user. As an example of the foregoing, the user may be presented with the option to click on one of the following sentences
Preliminary Procedure
Starting with the first sentence we compare successively each word against the lists of components as per the specific instructions below. Once it is found it in a particular list, it is positioned in the template slot appropriated for the listed category (e.g., verb, noun, adjective, etc.). Next the next word in the sentence is picked up and the process is repeated. When no additional words are left, the sentence template is declared as complete; the sentence template, now appropriately filled up is moved to the signing engine. Thereupon the process is repeated,
utilizing same procedure for the next sentence. The process is looped until no sentences are left.
The Signing engine will determine as shown below, whether to sign, not to sign or sign multiple times the same sign as appropriate. Finally, the appropriate transitions are appropriately inserted in order to be signed and thereby complete a seamless signed ASL sentence.
The Procedure
The Algorithmic Routine Set for the ASL Converter
Verb Algorithm
Take first word in sentence E and compare with VERB list; If there is a match then it is a verb and then LOCATE the word INTO the verb part of ASL SENTENCE TEMPLATE (i.e., SENTENCE A). If there are two verbs in the sentence then locate the order of their appearance: and
Noun Algorithm
If the word is not a VERB then Check it against the NOUN LIST; if there is a match then declare it as a noun; then LOCATE the word INTO the noun part of the ASL SENTENCE TEMPLATE (SENTENCE A); else
Adverb Algorithm
Check the word against the ADVERB LIST; if there is a match, declare it as an ADVERB and then LOCATE IT INTO the adverb part of the ASL SENTENCE TEMPLATE SENTENCE A); else
Adjective Algorithm
Check the word against the ADJECTIVE LIST; if there is a match then declare it as an adjective then LOCATE IT INTO the adjective part Of the ASL SENTENCE TEMPLATE (SENTENCE A); else
Auxiliary Algorithm
This algorithm is not needed in this version as it was taken care off when SENTENCE E was submitted to the Signtel Interpreter before it arrived at this module; the Signtel Interpreter has an option and once activated it will automatically have the auxiliary removed).
Pronoun Algorithm
Take the word and look for a match in the PRONOUN LIST; if there is a match, it is declared as a pronoun; then assign to the pronoun word a serial number in the sentence to be the serial number of the word preceding it+1. Next keep numbering each word by taking the current word−1 until getting to a word with serial number 1; next continue to provide serial number to the word immediately following the pronoun by adding 1 (i.e., “pronoun=pronoun+1”); now proceed.
Additional Considerations
Second Approach
In one embodiment, the English sentence may be submitted to the said Signtel Interpreter; whereupon the following may take place:
Reading on Electronic Ink Display with Colors
There are electronic reading tablets for books, such as the Kindle by Amazon that even has the technology of providing ability to read outdoors in the Sun without a glare. However, pictures in color are not yet available. While our ASL Converter can operate on a black and while display, the technology disclosed below enables to do it in color as well. It is based on technology developed by the applicant and registered in a dated patent book showing the date of Sep. 28, 1979. It is based on creating the screen panel from fiber optic filaments acting as “wires” wherein varying electrical pulses at one end (the pixel generating end) of a fiber causes varying colors of light to show at the other end, where the viewer can see the pixels in colors. Thus, this invention is incorporated herein by reference.
The Reverse Direction ASL To English Converter When a hearing impaired person who signs ASL arrives at an emergency room in a hospital he or she may understand what the medical staff says because they can either use voice recognition or text in conjunction with the ASL converter that will enable the person to see what is said or the text to him or her in ASL sign language. However, when that person responds, such as entering text into the Signtel Interpreter, it is in ESL and as such will be meaningless or at best difficult to comprehend for the medical staff. The ASL to English converter resolves that problem and the text entered by the person utilizing ASL may appear in English to the medical staff or voiced to them that way.
In essence the process outlined above is reversed, whereby the construct of the ASL converter (English to ASL) proceeds sentence by sentence however, the task here is complicated in the sense that in the English to ASL we go from a maximal full sentence to a reduced sentence, while in the reverse we start from a minimalist sentence and have to account for proper times and tense that do not exist in ASL. Further, while in ASL the locations for the subject, the time, etc is fixed, this is not the case in the English sentence and requires a different approach for the construction of the English sentence from the ASL sentence and is disclosed herein.
Cognitive Disability
Same conversion procedure as disclosed herein may also be useful for persons with cognitive disability, or persons versed in another language altogether who need translation in a simple and understandable form. As such an additional database is provided with images, wherein at least one of a pictures, animations, or cartoons constitute the material in said database. In that regards, said images would either be added or provide the complete articulated voice or text.
Lip Reading
Right now the signer enunciates the words that enable lip reading. So a person (say late deaf) who reads lips can also consult the text from time to time. However, under the conversion to ASL there are words that are not mouthed (in ASL), putting the late deaf at a disadvantage (as they will be missing words for comprehension). We advance the following solution in the converter.
The embodiment for persons who read lips departs from identifying what needs to put in certain locations of the disclosed template and instead the system identifies what is not needed in a sentence and takes it out. Namely, only essentials of each sentence are kept rather than selecting components and positioning them. For example, let us look at the following:
When said sentence is repeated with the essentials underlined, we have:
“I WILL GO TO THE STORE TO BUY FOOD; namely, “I GO STORE BUY FOOD”
Let's take another sentence:
Underlining only the essentials the sentence becomes:
“THE JUDGE DECIDED JEREMY DID NOT DO ANYTHING WRONG AND LET HIM GO;” namely
We note that in ASL the sentence should be:
Thus some transposition may be needed.
Let's look at another case:
Underlining the important elements we get:
It is noted that in ASL, same sentence would appear as:
Thus in the embodiment for lip reader repositioning may be chosen as the procedural route.
As can be seen from the foregoing description, an ASL conversion system has been provided, as well as a new type of color screen. While the present invention has been described in the context of specific embodiments thereof, other alternatives, modifications, and variations may become apparent to those skilled in the art having read the foregoing description. Accordingly, it is intended to embrace those alternatives, modifications, and variations.
This application claims the benefit of U.S. provisional patent application No. 61/589,391, filed Jan. 22, 2012, entitled “THE ASL CONVERTER.”