The present invention relates to computer-assisted language learning and, more particularly, to the incorporation of contextual cues in an interactive interface for language learning.
Current techniques for Computer-Assisted Language Learning (CALL) and Technology Enabled Language Learning (TELL) include approaches such as translation and transcription exercises, simulated dialogue, reading in the target language, or reading parallel language texts. Generally speaking, these techniques present some sort of pure or combined audio, graphic, textual, or video stimulus to which the learner is to respond using speech, writing, or menu selections.
However, contemporary linguistics research shows that language learning is strongly facilitated by the use of the target language in interactions where the learner can negotiate the meaning of vocabulary and that the use of words in new contexts stimulates a deeper understanding of their meaning. Current TELL and CALL technologies lack the ability to give the learner an opportunity to linguistically interact within his or her current problem-solving context.
Pocket translators allow users to quickly translate text but do not provide contextual or cultural background. Furthermore, pocket translators are not interactive and do not allow the user to practice a new language in conversational situations. Hand-held translation devices also require the user to provide input for translation which limits the user's ability to interact with their environment if they have to type or speak into a translator. Hand-held translation devices act only as a tool to assist in language learning and have very limited function as an interactive instructional device.
Although museums and exhibitions often provide hand-held translation devices that can utilize user input regarding physical location to translate location-specific content, such technologies do not provide the important conversational aspect that is necessary in learning a new language. These hand-held translation devices are functionally limited within the location involving a specific set of exhibits or demonstrations and require pre-programming of data regarding each location.
While computer-enabled video interactions can present engaging situations that provide opportunities to model and practice language, the most successful of them must resort to dramatic excess to maintain learner engagement. Their focus on rare or contrived situations leads to learners hearing and using unusual or infrequent expressions which would be not be useful in everyday situations. Furthermore, the learner does not link language use to his or her actions and goals; instead, language use relates to the portrayed actors' actions and goals.
Principles of the invention provide improved techniques for language acquisition through the incorporation of data concerning the context in which acquisition is occurring.
By way of example, in one aspect of the present invention, a language learning system includes an interface for communicating with at least one user, at least one sensor for collecting at least one form of data regarding the context in which the system is being used, and a processing device capable of making at least one adjustment to the communication with the user based on analysis of at least a portion of the data collected by the at least one sensor.
The data may include audio data, visual information, biometric data, location, or velocity and the sensors may include a microphone, a camera, a biometric sensor, a global positioning system (GPS) device, or a velocimeter. The system may also use this data, alone or in combination with schedule data obtained from an external source, to determine the attention level of the user and to make corresponding adjustments to the communication. The system may further be capable of tracking changes to the data collected by the sensor and/or the number and/or type of errors made by the user and making corresponding adjustments to the communication.
In another aspect of the present invention, a method for facilitating language acquisition includes the steps of collecting at least one form of data regarding the context in which acquisition is occurring and communicating with at least one user wherein the communication is based at least in part on analysis of at least a portion of data collected by at least one sensor.
Advantageously, principles of the invention provide enhanced techniques for utilizing contextual information to facilitate enhanced language acquisition. Principles of the invention provide for incorporating contextual cues into a conversation in order to facilitate deeper understanding of a target language. Principles of the invention also permit adjusting the pace of the conversation in response to the user's attention level and/or errors.
These and other objects, features and advantages of the present invention will become apparent from the following detailed description of illustrative embodiments thereof, which is to be read in connection with the accompanying drawings.
The system contains user input devices 120, which may include, for example, speech/audio or point-and-click menus. The system also contains user output devices 130, such as speakers, headphones, and/or visual display. System 110 may acquire audio data 103, visual information 104, biometric data 105, global positioning system (GPS) data 107, and velocity data 108. This GPS data can be used to identify the user's location and allow the module to isolate a set of questions and conversation topics related to that specific area. For example, if a user is learning Italian and the module, using GPS, recognizes that the use is in a grocery store, the module may ask questions related to items in a grocery store in Italian. Additionally, velocity data 108, either alone or in conjunction with GPS data 107, can be used to determine whether the user is stationery, walking, running, or driving, and to thus determine an appropriate pace of questioning. For example, rapid questioning of a user who is operating a vehicle may distract the user and result in a dangerous situation.
The speech, audio, visual, and biometric recognition modules may be used to identify the user's surroundings to provide appropriate questions for the user. For example, if a camera identifies a dog and an audio recognition system recognizes a dog's bark, the system may prompt the user to answer questions about a dog. The system may also incorporate simple games based on the recognition systems that will improve the user's vocabulary. For example, “I Spy” is a popular game that involves the identification of objects of a certain shape or color. The system can isolate an object and then request the user to identify it through questions in a particular language.
The system may also be synchronized to the user's home computer 106 to update daily activities and to-do list in order to help the system adapt to the user's activities and pace of life. The system may also synchronize to, for example, a personal digital assistant (PDA) (e.g., Palm or Blackberry), mobile phone, smart watch, or any other electronic repository of scheduling information. Biometrics may also be used to measure the user's heart rate to determine if the user is doing exercise, nervous, or under strain. If, for example, if the user is traveling at a fast pace, the system may ask fewer questions so not to distract the user or ask questions related to the user's current activities. The system will be able to recognize activities based on the user's responses to a question “what are you doing?” or the module can sync with the user's planner and follow the user through their daily scheduled activities. If the user is moving slowly, the module may ask more questions and process more information related to the surroundings. Depending on the user's preference and their pace settings, the system may determine that it should refrain from interacting with the user.
For example, OPS coordinate 306 may indicate that the user is in a museum and the keyword search 303 may identify words such as “Picasso” and “Dali.” Accordingly, the system may choose to engage the user in conversation regarding 20th century Spanish art or merely ask the user what he thinks of the works he is viewing. By tailoring the conversation to the context, the system can provide more relevant and engaging exercises, which in turn will facilitate more effective learning.
For example, object recognition module 400, through the use of object identification database 401, may detect the presence of bats, helmets, and balls. Accordingly, video cultural information compiler 402 may conclude that the user is at a baseball game. Therefore, the system may initiate dialogue in the target language about what the user's favorite team or players are. If the user is keenly interested in baseball, learning words which are relevant to baseball may be more useful to the user than rote examples, which may cover subjects in which the user lacks interest and will therefore find irrelevant and uninteresting (and probably useless as well).
For example, a user who is stressed or tired may be less able to engage in faster-paced learning then one who is calm and focused. If biometric data identification module 500 detects, for example, that the user's heartbeat is significantly faster than user profile 502 would indicate and biometric profile database 503 shows that this increased heart rate is likely to indicate that the user is stressed and distracted, attention compiler 502 may choose to decrease the pace of language learning or perhaps even pause until the user is calmer and better able to focus on his studies.
The temporary profile created by 702 is stored in user language history profile 804, which also contains the user's basic language history and comprehension information. Pace-mediated question module 801 selects questions based on the temporary profile of the user's current attention level from the question database 802, which lies within language database 805. The questions within the database are also compiled in a hierarchical system based on the results from error-statistic module 800, which indicates in which areas of the language the user has the highest number of errors. Error-statistic module 800 receives information on errors from error detection module 807, which detects errors in pronunciation and incorrect language use via microphone 203. User interface compiler prepares the information processed by language teaching processing module 230 and also prepares games to executed from the game database 803 which is connected to microphone 203 and video camera 204.
The methodologies of embodiments of the invention may be particularly well-suited for use in an electronic device or alternative system. For example,
It is to be appreciated that the term “processor” as used herein is intended to include any processing device, such as, for example, one that includes a central processing unit (CPU) and/or other processing circuitry (e.g., digital signal processor (DSP), microprocessor, etc.). Additionally, it is to be understood that the term “processor” may refer to more than one processing device, and that various elements associated with a processing device may be shared by other processing devices. The term “memory” as used herein is intended to include memory and other computer-readable media associated with a processor or CPU, such as, for example, random access memory (RAM), read only memory (ROM), fixed storage media (e.g., a hard drive), removable storage media (e.g., a diskette), flash memory, etc. Furthermore, the term “I/O circuitry” as used herein is intended to include, for example, one or more input devices (e.g., keyboard, mouse, etc.) for entering data to the processor, and/or one or more output devices (e.g., printer, monitor, etc.) for presenting the results associated with the processor.
Accordingly, an application program, or software components thereof, including instructions or code for performing the methodologies of the invention, as described herein, may be stored in one or more of the associated storage media (e.g., ROM, fixed or removable storage) and, when ready to be utilized, loaded in whole or in part (e.g., into RAM) and executed by the processor 1010. In any case, it is to be appreciated that at least a portion of the components shown in the above figures may be implemented in various forms of hardware, software, or combinations thereof, e.g., one or more DSPs with associated memory, application-specific integrated circuit(s), functional circuitry, one or more operatively programmed general purpose digital computers with associated memory, etc. Given the teachings of the invention provided herein, one of ordinary skill in the art will be able to contemplate other implementations of the components of the invention.
Although illustrative embodiments of the present invention have been described herein with reference to the accompanying drawings, it is to be understood that the invention is not limited to those precise embodiments, and that various other changes and modifications may be made by one skilled in the art without departing from the scope or spirit of the invention.