Patent Application
20100057436
The present invention relates generally to a language translation system, and more particularly, to a Phonetic Language Translation System capable of translating any speech within the audio output of the audible program to the native language of user who is listening to the audible program. The present invention performs translation of the spoken words from the audio output of the audible program to a language the users' brain language area can comprehend.
In recent days, people prefer to hear event/programs in their native language either while watching the public events like indoor area for housing dramatic presentations, stage entertainments, surgical demonstrations or motion-picture. People always look for translators or subtitles in events and programs to comprehend the foreign language. There is a need for a system to translate audio output of any audible program to a user's native language in a fast, easy, reliable and cost effective manner. Also, there is a need for a phonetic language translation system that may substitute interpreters and subtitles.
A number of hand-held language translators are available in the market, capable of translating an audible speech only to a specific set of languages. This predetermined set is a combination of popular languages spoken in the world. But, there are more than 6,700 native languages being used in the world. People are forced to buy multiple language translators to cover a broader range of language translation. There is not a single system capable of performing audible speech translation from any of the 6,700 native languages to any of the other languages spoken in the world.
In today's language translators, the user always has a need to select their native languages as target language. If translators are unavailable for their native languages they have to settle for a translator that has the closet language they are familiar with as target language. But in settling for secondary target language translators there is a possibility that users may experience loss in understanding some of the translations. This can happen because of cross cultural differences.
U.S. patent application Ser. No. 12/543,054, filed Aug. 18, 2009, and U.S. patent application Ser. No. 12/563,123, filed Sep. 19, 2009, assigned to the same assignee as the instant application, incorporated herein by reference.
The following U.S. patents are hereby incorporated by reference for their teaching of language translation systems and methods: U.S. Pat. No. 6,356,865, issued to Franz et al., entitled “Method and system for performing spoken language translation”; U.S. Pat. No. 5,758,023, issued to Bordeaux, entitled “Multi-language speech recognition system”; U.S. Pat. No. 5,293,584, issued to Brown et al., entitled “Speech recognition system for natural language translation”; U.S. Pat. No. 5,963,892, issued to Tanaka et al., entitled “Translation apparatus and method for facilitating speech input operation and obtaining correct translation thereof”; U.S. Pat. No. 7,162,412, issued to Yamada et al., entitled “Multilingual conversation assist system”; U.S. Pat. No. 6,917,920, issued to Koizumi et al., entitled “Speech translation device and computer readable medium”; U.S. Pat. No. 4,984,177/issued to Rondel et al., entitled “Voice language translators; and U.S. Pat. No. 4,507,750, issued to Frantz et al., entitled “Electronic apparatus from a host language”.
According to U.S. Pat. No. 5,615,301 issued to Rivers et al., entitled “Automated Language Translation System”, each sentence is translated into a universal language and then the sentences are translated from universal language to the preferred language of the user as identified by the user. Such a system disclosed in U.S. Pat. No. 5,615,301, is capable of translating an audible speech only to a specific set of native languages. This predetermined set is a combination of popular native languages spoken in the world. But, there are more than 6,700 native languages being used in the world. Such a system disclosed in U.S. Pat. No. 5,615,301 is not capable of performing language translation for audio speech in any of the 6,700 native languages to any of the other languages spoken in the world.
Although there have been many advances in system and software for providing phonetic language translation for users who are interested to hear an audible program in a language other than their native language; there has not been an system or method that facilitate to identify user's native language using language area of brain of user and use the identified native language for translation. Accordingly, the present inventor has developed a system that can identify the native language of user by his/her brain language area as target language for audio speech translation.
In view of the foregoing disadvantages inherent in the prior art, the general purpose of the present invention is to provide a native language translation system configured to include all the advantages of the prior art, and to overcome the drawbacks inherent therein.
The present invention translates the spoken dialog in audio output to user's native language. In other words, the present invention performs language translation of the audible program presented to the user; to a language that is directly comprehended by the language area of the listener's brain. Thus, audience enjoys the audible program without having language books, interpreters, or closely reading the subtitles.
The present invention allows a user to hear a program in his or her native language either while watching the audible program which is presented in a foreign language. The present invention includes a speech recognition module to recognize phonemes of speech from the audio program. These phonemes are then combined in word groups to form recognizable words in one of the native languages spoken in the world. The user's brain language area activity is recorded by using electrodes in the cap. The recorded brain language area activity signals are then analyzed and compared with brain language area activity knowledge base. If the characteristics of received brain language area activity signal are identical to any one of the entry present in the brain language area activity knowledge base the present invention selects the equivalent native language information from the entry and then the selected native language is used as target language for language translation. Further the present invention automatically translates the speech in audible program into an audible speech of user's native language and then each translated sentence is broadcast with a voice synthesizer to the user.
Accordingly, it is a principal object of the present invention to provide a language translation, to translate the audio of an audible program into a native language of the user.
It is another object of the present invention to identify the native language of the user without selecting any language preference. The present invention uses the brain language area activity signals to identify the native language of the user. The brain language area activity signals are acquired using the electrodes which are presented in the cap and then these signals are compared with brain language area activity knowledge base to determine the native language of the user.
It is an object of the present invention to provide improved elements and arrangements thereof in a system for the purposes described which is inexpensive, dependable and fully effective in accomplishing its intended purposes.
In another aspect, the present invention provides a phonetic language translation system for use as a portable apparatus, thereby making the phonetic language translation system handy and comfortable to use.
These and other objects of the present invention will become readily apparent upon further review of the following specification and drawings.
Therefore, an object of the present invention is to provide the phonetic language translation system that is capable of providing a translation of audio output of an audible program from one language to a native language of user which his/her brain language area can comprehend, thereby user does not need to select the target language but is able to listen to the audible speech of foreign language program without using language translator books or closely reading the subtitles of foreign language program.
These together with other aspects of the present invention, along with the various features of novelty that characterize the present invention, are pointed out with particularity in the claims annexed hereto and form a part of the present invention. For a better understanding of the present invention, its operating advantages, and the specific objects attained by its uses, reference should be made to the accompanying drawings and descriptive matter in which there are illustrated exemplary embodiments of the present invention.
FIG. 1.a illustrates a first embodiment of prior art of a Phonetic Language Translation System of the present invention.
FIG. 1.b illustrates a second embodiment of prior art of a Phonetic Language Translation System of present invention.
FIG. 5.a is a front-side elevation of the cap;
FIG. 5.b is a back-side elevation of the cap;
FIG. 5.c is a left-side elevation of the cap;
FIG. 5.d is a right-side elevation of the cap.
The exemplary embodiments described herein detail for illustrative purposes and are subject to many variations in structure and design. It should be emphasized, however, that the present invention is not limited to a phonetic language translation system, as shown and described. It is understood that various omissions and substitutions of equivalents are contemplated as circumstances may suggest or render expedient, but these are intended to cover the application or implementation without departing from the spirit or scope of the claims of the present invention. Also, it is to be understood that the phraseology and terminology used herein is for the purpose of description and should not be regarded as limiting.
The present invention allows a user to hear a program in his or her native language while watching the public events like indoor area for housing dramatic presentations, stage entertainments, surgical demonstrations or motion-picture, which is presented in a foreign language. The speech in the audible program is reproduced as an analog audio signal transferred on a physical cabling audio distribution network (shown in
The one end of the male plug connector 102 has one terminal connected to provide a left audio signal AL and a second terminal connected to provide the right audio signal AR. The three-slot female connector 20 of present invention has a terminal that is connected through the cabling audio distribution network of theater/auditorium to a power supply voltage source either integrated in or associated with phonetic language translation system (in dashed lines). The three-pin male connector 102 has a terminal that engages the terminal of the three-slot female connector 20 to conduct the power supply voltage VPS to the power conditioner. The power conditioner conditions the power supply voltage VPS to generate the voltage VAA(not shown) to provide the necessary energy to power the system of present invention. Alternately, in connector structures, where there are no connections to the power supply voltage source VPS, the power conditioner may be connected to a battery.
The phonetic language translation system of present invention receives the audio signal of a program presented to the user. The speech recognition module 104 is capable of receiving continuous speech information and converts the speech into machine recognizable phonemes. The speech recognition module 104 also includes a spectrum analyzer to remove background noise from the audio signal.
The phonetic language translation system of present invention discloses a translation module (shown in FIG. 1.a) which has parsing 106 and generation 108 module. The translation module is capable of interpreting the elliptical and ill-formed sentences that appear in audio output of the audible program. An interface is made between speech recognition module 104 and parser 106 in terms of phoneme hypothesis and word hypothesis levels, so that prediction made by the parser 106 can be immediately fed back to the speech recognition module 104. Thus, phoneme and word hypotheses given to the parser 106 consists of several competitive phoneme or word hypotheses each of which are assigned the probability of being correct. With this mechanism, the accuracy of recognition can be improved because it filters out false first choices of the speech recognition 104 and selects grammatically and semantically plausible second or third best hypotheses. The parser 106 is capable of handling multiple hypotheses in a parallel rather than a single word sequence as seen in machine translation systems. A generation module 108 is capable of generating appropriate sentences with correct articulation control. The phonetic language translation system of present invention employs a parallel marker-passing algorithm as the basic architecture. A parallel incremental generation scheme is employed, where a generation process and the parsing processing run almost concurrently. Thus, a part of the utterance is generated while parsing is in progress. Unlike most machine translation systems, where parsing and generation operate by different principles, this invention adopts common computation principles in both parsing and generation, and thus allows integration of these processes.
Various systems use different methods to extract the users' intentions from her/his brain electrical activity. The present invention discloses a new method to identify the native language of user by using these brain signals and translate the audio speech to identified native language. The present invention includes a signal processing module as shown in FIG. 1.a which has data acquisition module 110, signal preprocessing with online blind-source separation 112 to reduce artifacts and improve signal to noise ratio, a features extraction system 114 and classifiers i.e. pattern recognition 116.
In an exemplary embodiment, the first task of phonetic language translation system of the present invention is signal acquisition. The phonetic language translation system of present invention relies on measurements of brain language area activity signals collected via electrodes in the cap. As shown in FIG. 1.b, the electrode arrays 60 consists of sterile, disposable stainless steel, carbon tip electrodes each mounted on a cap 50 (as shown in FIG. 1.b) and closely joint with 66-pin male connector 80 for ease in positioning. These electrodes are transparent, flexible, numbered at each electrode contact and the standard spacing between electrodes is 1 cm. The electrodes of the cap 50 (as shown in FIG. 1.b) sit lightly on the language areas (Left, Right hemispheres and frontal lobes) of user's brain and are designed with enough flexibility to ensure that normal movements of the head do not cause injury to the user.
As shown in
The second task of phonetic language translation system of the present invention is signal processing as shown in FIG. 1.a, which includes signal preprocessing online blind-source separation 112, features extraction system 114, pattern recognition 116. Language comprehension features are isolated from the brain language area activity signals and translated into machine readable code.
The third task of the present invention is native language identification 118. The native language identification module 118 uses an algorithm to determine the native language of user by comparing the recorded signals characteristics with “brain language area activity knowledge base;” (as shown in FIG. 1.a).
The “brain language area activity knowledge base” is an exhaustive, comprehensive, obsessively massive list of brain signal samples of language areas activity information; where the list of samples are collected information from experimental test results data of brain's language area activities and collected information from neurologists about brain's language areas comprehension. The “brain language area activity knowledge base” comprises of millions and millions of brain signals collected by recording the language area activity of the human brains. People from all cultures around the world are surveyed; while listening to the audible program in their native language, brain activity signals from the language area of their brain are recorded. These signals act as raw translations that indicate how the brain perceives the audible program in their native language. The recorded “brain language area activity signals” are then analyzed and the characteristics of the “brain language area activity signals” are stored in the “language area brain activity knowledge base” along with the name of corresponding native language.
For example, for building the “brain language area activity signal” sample for French language, a French audible program is presented to a person for whom French is the native language. During this experiment the electrodes are connected to the language areas (i.e., Left and Right hemispheres and frontal lobes) of his/her brain. While listening to a French audible program, his/her brain language area activity is being recorded. The recorded “brain language area activity signals” are then sent to a translator that uses special algorithms to decode the neural language to determine the characteristics of the French language. The test results along with name of the native language (i.e., French) information are being stored in the “brain language area activity knowledge base”.
The “brain language area activity knowledge base” thus built contains a massive store house of characterizations of “brain language area activity signals” for over 6,700 native languages spoken across the world. This massive repository of language characteristics is later used by the present invention to identify the native language of the user.
The first language area is called Wernicke's area 310. Wernicke's area 310 is an area in the posterior temporal lobe of the left hemisphere of the brain involved in the recognition of spoken words. Wernicke's area 310 one of the two parts of the cerebral cortex linked since the late nineteenth century to speech. It is traditionally considered to consist of the posterior section of the superior temporal gyrus in the dominant cerebral hemisphere (which is the left hemisphere in about 90% of people). The second language area within the left hemisphere is called Broca's area 308. The Broca's area 308 is an area located in the frontal lobe usually of the left cerebral hemisphere and associated with the motor control of speech. The Broca's area 308 doesn't just handle getting language out in a motor sense it is more generally involved in the ability to deal with grammar itself, at least the more complex aspects of grammar.
By analyzing data from numerous brain-imaging experiments, researchers have now distinguished three sub-areas within Wernicke's area 310. The first sub-area responds to spoken words (including the individual's own) and other sounds. The second sub-area responds only to words spoken by someone else but is also activated when the individual recalls a list of words. The third sub-area is more closely associated with producing speech than with perceiving it. All of these findings are still compatible, however, the general role of Wernicke's area 310, relates to the representation of phonetic sequences, regardless of whether the individual hears them, generates them, or recalls them from memory.
In operation, as illustrated in FIG. 1.a, three-pin male connector 102 is connectable to three-slot female connector 20 of user's seat 10 armrest in the auditorium/theater. While hearing the audible program users wear a cap 30 (as shown in FIG. 1.a) and the activity of language area of user's brain is being recorded using electrodes 50 (as shown in FIG. 1.a) in the cap 30. The recorded “brain language area activity signals” are decoded in signal processing module (as shown in FIG. 1.a) to identify the user's native language. The native language identification module 118 receives the decoded brain signals and runs a program routine to determine the native language of user, by comparing with “brain language area activity knowledge base”. The native language identification module 118 program looks for the identical characterizations in the “brain language area activity knowledge base” for the decoded brain signals. If any data characterizations match with the decoded brain signals then the corresponding native language information is retrieved and fed into the generation module 108 for translation.
Simultaneously, the audio output of the broadcast program is transmitted through the three-pin male connector 102 to the speech recognition module 104. Speech recognition module 104 identifies the phoneme-level sequences from the audio output and builds the information content from best bet hypotheses of phoneme-level sequence using the parser module 106 and language dictionaries. The language dictionaries is a knowledge base which contains all possible word presented in more than 6,700 native languages being used in the world and provides lexical, phrase, syntactic fragment to generation module 108 while generating the equivalent sentence of native language of user for the audible speech from audio output. The language dictionaries are also operatively coupled to the parser 106 where speech recognition module 104 receives the feedback of phoneme hypothesis and word hypothesis prediction from the parser 106.
After determining the native language of speech in the audible program, the consecutively received phonemes are grouped to form consecutive words and these words are then combined into recognizable sentences in accordance with the grammatical rules of that native language. These recognizable sentences are then translated into an identified user's native language and each translated sentence is broadcast using voice synthesizer 120 to earphones 40 (as shown in FIG. 1.a) of the cap 30, so that the user's brain can comprehend the audible program in his/her native language.
Although the description above contains much specificity, these should not be construed as limiting the scope of the invention but as merely providing illustrations of some of the presently preferred embodiments of the invention. For example, the phonetic language translation system of the present invention could be utilized in any public event place where audible program is being broadcasted to each of user's native language. Such public event place could be, a theater, an auditorium, a flight, and such audible program could be, an indoor area for housing dramatic presentation, a stage entertainment, a surgical demonstration, in-flight entertainment, in-flight announcements or a motion-picture show having a sound track.
