BRIEF DESCRIPTION OF DRAWINGS
FIG. 1 shows a user making use of a language training system constructed according to the present invention.
FIG. 2 shows a display screen of the FIG. 1 system prompting a user to speak several words.
FIG. 3 shows a display screen of the FIG. 1 system, after all words were recorded by the user, offering analysis of user pronunciation errors (adding Analyze button at the center bottom of the screen).
FIG. 4 shows the display screen of the FIG. 1 system providing pronunciation error analysis of the words recorded as in FIG. 3.
FIG. 5 shows the display screen of the FIG. 1 system prompting a user to speak several expressions.
FIG. 6 shows the display screen of the FIG. 1 system providing pronunciation error analysis of the expressions recorded as in FIG. 5.
FIG. 7 shows a display screen of an exercise training a user with the proper language required for dialogue.
FIG. 8 shows a display screen of Mini Dialogue after the user has recorded all the responses and they were analyzed in accordance with communication criteria, thus providing overall speech grade and pronunciation Help.
FIG. 9 shows a display screen of a Dialogue conducted between the user and the system/PC. The user is selecting to play Speaker A or B roll. Then he/she is triggered to record the speaker roll in response to the PC “speaking” the other speaker roll.
FIG. 10 shows a display screen of the FIG. 1 system providing communication performance result and offering pronunciation error analysis of the dialogue recorded according to the application described in FIG. 9.
DETAILED DESCRIPTION
FIG. 1 is a representation of a user 102 using the Spoken Language System constructed according to the current invention. The system shown in FIG. 1 includes a PC 106 with a Sound Card, speakers or headset 122, and a microphone 126. The PC plays multiple roles in the system. Its CPU runs the application, its display 120 presents the application screens, and its audio interface plays the application prompts through the speakers or headset 122. In addition, the PC Audio input is being used to record (via the microphone 126) the user produced utterances. These utterances are recorded to the PC memory to be later played back to the user and/or analyzed according to pronunciation or communication analysis criteria.
FIG. 2 shows a visual display of the screen 120 that prompts or triggers the user to speak multiple words. In the current application software, the user first produces (speaks) all the words. Each word is displayed on the screen and the user can listen to it being spoken by clicking on the play button located on the left side of each word. The user clicks on the microphone button and then records the user's pronunciation of the word. During recording, a record level indicator is displayed in the recorded word row. If recording is rejected because the speech was too soft, too loud etc., an error message is immediately displayed on the pronounced word row. If the word was properly recorded (regardless of pronunciation errors), a signal symbol is presented on the display and a user play button is added on the right side of the microphone display icon. The Student Play button enables the user to play his/her recorded word. Each word translation is also displayed on the right side of the word row. The user has to finish recording all the prompted words in order to continue with the application. The words can be recorded in any order as long as, at the end, all the prompted words are recorded. The user may also, after listening to his/her recordings, elect to re-record a certain word. The user can do so, and the last recording of each word is taken into account for the following parts of the application.
FIG. 3 shows a visual display of the screen described in FIG. 2 above, after all words were successfully recorded. Some words may have been recorded several times, but there is no external indication to the number of times each word was recorded. Only the last recording will be analyzed in the following part of the application software. After all words are recorded, a new button is presented at the center bottom of the display—shown in FIG. 3 as “Analyze Results”. This button enables the user to run the application software analysis program, and analyze user recordings of the presented words to find pronunciation errors.
FIG. 4 shows a visual display of a feedback of pronunciation error analysis performed on the words presented in FIG. 3 above, after the user had clicked on the Analyze Results display button. Up to five pronunciation errors are displayed in the pronunciation feedback window. Each pronunciation error is identified by English letters (e.g. IH) symbolizing the phoneme that was not pronounced properly, and/or another text that provides the user indication on the error phoneme (e.g. sheep). This kind of simplified text may be required, since most users of such systems are not familiar with the phonetic alphabet. When one of these error phoneme buttons is clicked, the system displays all words where the error was found, and indicates the exact location of the error within the word. This is done by displaying the “spelling” of the word, and adding a red triangle below the part of the text that represents the phoneme that was identified as pronounced incorrectly. The user is also offered additional training and practice for the specific sound that was mispronounced. By clicking on the “Train Me” button shown in FIG. 4, that appears below the mispronounced phoneme, the user is being introduced to another part of the application that teaches and practices the student how to properly produce the sound.
FIG. 5 shows a visual display of a similar screen as in FIG. 2, which triggers the user to speak. In FIG. 2, the recorded utterances were words, whereas in FIG. 5 these are expressions composed of multiple words. The application is also similar to the one described in FIG. 2 above, that encourages the user to record all expressions before offering Pronunciation analysis.
FIG. 6 shows the computer system display screen providing feedback on the user's production of the inputted expressions. As in FIG. 4 above, where analysis results are displayed for words, the FIG. 5 screen provides feedback on the analysis results for the recorded expressions. Up to five phonemes that were mispronounced are displayed. When a user selects any of them, the application presents the expressions and exact location within each of the expressions where this error was identified. The user may also click on the newly appeared button—“Train Me”—that will offer additional teaching, training, and exercises on the proper production of the mispronounced sound (phoneme).
FIG. 7 shows a visual display of the system teaching the user the correct language required to conduct a dialogue. There are multiple questions and multiple answers for each of them. The user is requested to select the appropriate answer to each statement in the question. This exercise trains the user in dialogue language prior to the oral dialogue that follows this part of the application. A score is given to the overall student performance in this exercise.
FIG. 8 shows a display screen of the computer system that practices the user in dialogues. This part of the application software is called “Mini Dialogue” since the system/PC represents one of two speakers, where the user is the other one. These are short dialogues, one phrase for each speaker. The system prompts the user and he/she is requested to orally complete the other speaker role in the dialogue. After all recordings have been completed, the system analyzes the user utterances and provides a grade on the user overall speech performance as well as providing pronunciation help. The Speech Recognition engine being used in this application is the communication one, where only a subset of the pronunciation rules are active and the system emphasizes more on the communication skills than on the pronunciation skills.
FIG. 9 shows a display screen of the computer system that practices a more complete dialogue (compared to the Mini Dialogues presented in FIG. 8 above). In this case the user selects to be either speaker A or speaker B and then orally interacts with the PC that plays the other speaker role. The exercise goal is to improve and practice the user fluency in spiking the language while conducting a dialogue. Unless the user makes a “significant” mistake, the system will not comment and let the user record his/her part of the dialogue without interference.
FIG. 10 shows a display screen of the computer system that practices dialogues as presented in FIG. 9 above, where all user utterances were successfully recorded and are analyzed for fluency, intelligibility and pronunciation errors. The speech score is immediately presented, where in order to receive the pronunciation feedback the user should click on the Pronunciation Help button (“See your errors”), and then the pronunciation errors are presented (in a similar way as for the words and expressions). This part of the application uses the Communication Engine, which is the same Speech Recognition Engine that operates with sub set of the Pronunciation Errors rules, and thus enables (skips) certain pronunciation errors that are not effecting the intelligibility of the utterance, and indicate others that are unacceptable by an average teacher in a classroom.
Patent Application
20080027731
