The use of wireless devices is widespread and growing. Many wireless devices are capable of operating in multiple languages. The language setting may be changed by following traditional device menu commands. After a language setting is changed, all further operation by the device is typically in the newly selected language until and unless the language is once again changed by the user.
For a more complete understanding of the present disclosure, reference is now made to the following description taken in conjunction with the accompanying drawings.
Setting a language on an computing device can be challenging, especially for users who are not familiar with computing devices. Further, returning the computing device to a preferred language setting may become even more challenging when the language setting of the computing device is changed, which may happen either accidentally or intentionally, such as by an earlier user who preferred the device operate in a different language from that preferred by a later user. If a language of a device is somehow set to a language that a user does not understand, the use may experience frustration with the device and be unable to reset the language without difficulty. A number of implementations exist to navigate computing devices to reset a preferred language, however those implementations all involve following standard menu device commands that may be in a language a user does not understand.
Even communications to traditional customer services agents are unlikely to solve the problem as such agents may have limited ability to remotely operate the device or assist the user, particularly if the device is operating in a language the user does not understand, rendering the user incapable of precisely describing the device operation to a remote agent. As the customer service agent is unable see the computing device, he/she may be limited to virtually describing a solution to the user for setting the computing device to the user's preferred language.
Offered is a method for setting a preferred language on a computing device based on communications from a user in a preferred language. The device may include speech recognition capability that may recognize when a user is speaking a language different from a currently indicated language in a device setting. The communication may be, for example, repeated speech from a user in a language recognized to be a different from a currently selected language of the device. Such speech may indicate frustration with the device operation. The communications from the user may be received by the computing device directly from the user or may be received by a remote device (such as a server) configured to process communications from the user. In one aspect the language setting method cross references other information associated with the user or the computing device to determine and/or confirm the preferred language. The cross referenced information may be other configurations, settings, and/or information associated with the computing device.
At block 112, the language setting module 110 may receive a communication of the user 102. The communication may be in the form of speech, device navigation, or other communication. The language setting module 110 will then identify a language uncertainty trigger at block 112. The language uncertainty trigger may be some indication that the user 102 is struggling to navigate a feature, e.g., language setting, of the computing device or is generally struggling to operate the device 104. Such struggles may be evidenced by randomly operating the device 104.
At block 116, the language setting module may determine a preferred language of the user based on the communication of the user. At block 118, the language setting module 110 may select the preferred language of the computing device based on the communication of the user 102. Determining and/or Setting the preferred language may be implemented in conjunction with a speech recognition technique which recognizes the language spoken by the user. In one aspect, upon identification of the language uncertainty trigger in block 114 the user may be prompted by the device 104 to further communicate with the device in the user's preferred language so the language setting module 110 may confirm the preferred language. In some aspects, the preferred language may be set based on the user communication in conjunction with other information associated with the device 104 and/or the user 102, such as settings, configurations, and/or content which may indicate a language preference.
Aspects of the present disclosure may be implemented as a computer implemented method in a computing device or computer system.
The input device interface 208, sometimes also embodied as an input/output interface, enables the computing device 200 to obtain data input/speech from a user 102 and/or a variety of devices including, but not limited to, a digital pen, a touch screen, a keyboard, a mouse, a scanner, and the like. In addition to the exemplary components described above, a display interface 210 may be used for outputting display information to a computer user. Typically, the display information is output by the display interface 210 via a display device (e.g., a monitor or similar device). While not illustrated, one skilled in the art should appreciate that a display device may be incorporated as an integral element within a computing device 200 or may be separate therefrom.
The processor 202 may be configured to operate in accordance with programming instructions stored in a memory 214. The memory 214 generally comprises RAM, ROM, and/or other memory. Thus, in addition to storage in read/write memory (RAM), programming instructions may also be embodied in read-only format, such as those found in ROM or other non-volatile memory. The memory 214 may store an operating system 216 for controlling the operation of the computing device 200. The operating system may be a general purpose operating system such as a Microsoft Windows operating system, a UNIX operating system, a Linux operating system, or an operating system specifically written for and tailored to the computing device 200. Similarly, the memory 214 may also store user-executable applications 218, or programs, for conducting various functions on the computing device 200. For example, the application 218 in memory 214 may be configured according to aspects of the present disclosure to process or analyze communications (voice and data), settings and configurations stored in the storage database 212.
The language uncertainty trigger identification module 302 may be configured to identify a language uncertainty trigger, which initiates the preferred language setting implementation. The language uncertainty trigger is an indication that the device may be operating in a language that is not understood (or desired) by the user. The language uncertainty trigger may take a number of forms. In one aspect the language uncertainty trigger may be when the device the user is speaking in a language other than the preferred language setting of the device (or of an application running on the device). The language uncertainty trigger may also be evidence of frustration in the speech of the user, such as the user expressing verbal displeasure with the device operation. In one aspect, the language uncertainty trigger may be an input of commands to a device that are illogical or indicate that a user does not understand the commands that he/she is inputting. A number of other potential triggers are also possible. In one aspect, the user speaking in a language different from the operating language of the device may be sufficient for to institute a change of the operating language.
The computing device may be configured to listen to speech from the user or monitor operation of the device to detect various language uncertainty triggers. In one aspect, the input/output module 310 may include a listening feature to listen for communication of the user. The device may be configured to detect for language uncertainty triggers at all times, such as in a passive mode, or during certain prescribed or predetermined windows. The device may also be configured for heightened sensitivity for language uncertainty triggers during prescribed windows. For example, in one aspect, the listening feature may be activated (or have its sensitivity increased) after a preferred language setting is changed on the computing device. For example, the listening feature may be activated when a user changes a language configuration or setting of the computing device to determine if such a change was accidental or intentional. For example, following a language change setting the computing device may listen to the user to identify any inquiries or indications of confusion/frustration from the user. For example, a user may inquire about getting back to a previous preferred setting, or to resetting the computing device to a preferred language, or the user may generally attempt to communicate with the device in a language other than the current preferred language setting. The listening period may also be activated during device reset, initialization, or during other windows.
The user speech determining module 304 may determine a language being spoken by a user at a time a language uncertainty trigger is identified. If a user is expressing dissatisfaction with the language operation of a device, the device may determine the language being spoken by the user and consider that language when determining a new language to set the device to. In one aspect, the language setting module 308 may simply set the new language of the device to the language being spoken by the user. In another aspect, the device may prompt the user for further confirmation, such as a verbal response, prior to reconfiguring a language setting of the computing device. The user may also be prompted to select a further configuration based on the identification of the language uncertainty trigger. For example, a user struggling to set a preferred language may be prompted to select the preferred language and/or other settings/configurations of the computing device, such as language changing permissions.
In another aspect, a choice of language may be cross referenced by the cross referencing module 306 against other information associated with the user or the computing device. Such other information may include previous settings/configurations of the device or content associated with the user and/or the computing device. For example, other information may include media stored on the computing device (or stored elsewhere but associated with the user and/or device) such as electronic books, music, videos, movies, contacts and/or other content. Other information may also include other settings/configurations such as language settings of other applications. For example, the computing device may base a preferred language setting of the device on whether the user explicitly set the language for some applications or software associated with the computing device. Based at least in part on these settings, the language setting module 308 may determine the preferred language or suggest one or more languages to the user when the device determines the user may be unfamiliar with a presently set language of the device or application.
The computing device may have communication capabilities including short message service (SMS), emails and the like. A language setting associated with these communications capabilities may provide an indication of the preferred language of the user with respect to the computing device. An explicitly set language and/or language associated with one or more communication capabilities of the computing device may assist the determination of a user preferred language for the computing device. In some aspects, the current language setting and/or the language associated with one or more communication capabilities may be used in conjunction with communications from the user in determining and/or selecting the preferred language.
In one aspect of the disclosure, setting the preferred language of the computing device may be based on settings/configuration associated with other computing devices currently or previously associated with the user. For example, a service provider, e.g., Amazon, may automatically apply language settings of a previously owned computing device, e.g., Kindle, to a current computing device owned by the user. The settings/configuration of current and previously owned computing devices may be stored in a remote memory associated with a remote device, such as a server. Each stored setting/configuration may be associated with a user such that the settings/configuration of a new device associated with the user may be set or configured based on one or more stored settings/configurations. In some aspects, the new device may be configured to prompt the user to confirm the settings/configurations of the new device. For example, the user may be prompted for feedback in the language associated with the new configuration/setting. The preferred language may be set for the computing device when the user responds with a confirmation. Otherwise the new device may execute a language preference initiation sequence as described below.
In addition to processing a user's speech to correct a language setting after identification of a language uncertainty trigger, the user's speech may be processed to identify a preferred language without prompting by the user in the form of the trigger. That is, the user may be prompted by the device to speak to the computing device upon an initial power up. The initial user communications received by the input/output module 310 during set up may serve as a training set with which the preferred language of the user is identified. Alternatively, the computing device may be configured with a communication trigger or button, by which the user may initiate a communication with the computing device for the purpose of preferred language setting. The communication trigger or button may be engaged by the user independent of the identification of the language uncertainty trigger. In this case, engaging the communication trigger may serve to activate a language setting process similar to the process activated by identification of the language uncertainty trigger.
In one aspect, identification of the communication trigger or language uncertainty trigger may initiate contact between a user and a customer service agent either through the device or through other means (such as a telephone call to another device.) The device may be configured so that a remote agent may reset a preferred language remotely by sending a command to the device to select a certain preferred language.
Multiple devices may be connected over a network and may coordinate in language selection. As shown in
In certain language selection system configurations, a combination of devices may be used. For example, one device may receive a speech input from a user, another device may process that speech to determine the language of the speech, and still another device may output an indication of preferred language to a user's device. For example, a user's speech input may be received by a wireless device 404 and sent to a computer 414 or server 416 for language identification. The resulting language identification may be returned to the wireless device 404. Or computer 412 may partially process the speech before sending it over the network 402. Because such processing may involve significant computational resources, in terms of both storage and processing power, such split configurations may be employed where the device receiving the speech may have lower processing capabilities than a remote device and more rapid or higher quality results are desired. The processing may thus occur remotely from the device whose language settings are to be configured.
Accordingly, the computing device may utilize a speech/language recognition technique implemented at the computing device and/or remotely. For example, the computing device may include a speech recognition device to identify the language in which the user is speaking. Alternatively, the communication of the user may be deciphered at a remote device. The remote device may be a server hosting a speech recognition device. In this case, the user's speech may be forwarded from the local device to a remote device where the language is deciphered. The remote device may then indicate the spoken language to the local device for purposes of determining a language setting on the local device. Alternatively, the remote device may perform the language setting procedure and simply notify the local device of the determined preferred language.
In some aspects of the disclosure, identifying the language may be performed independent of understanding the language. For example, the language may be identified based on phoneme (sound of a language) recognition techniques, text recognition techniques or other language identification techniques that may be implemented on the computing device (e.g., speech recognition device). The phoneme recognition technique may be based on sounds associated with different languages. For example, a language may be identified by different sounds that follow each other in certain sequences. In some aspects, the speech/language recognition technique may apply a broad set of phonemes across different languages. The speech/language recognition technique may be based on models built for each language. In some aspects, the models may be customized to a particular user. For example, the models may be based on sound and text from a particular language as well as communications from the particular user. Models for identifying a language may be significantly simpler than models used for full speech recognition. Language identification models may be stored either on a local device or a remote device. When the computing device identifies one or more languages having similar sounding words the computing device may prompt the user to select a preferred language from the one or more identified languages.
To avoid toggling between two or more languages, the computing device may be configured to deactivate the preferred language setting implementation on the computing device. For example, if the language is explicitly set to English, and the user is detected conversing with a friend in French, the user may not want the computing device to reset the preferred language or to prompt the user to select a preferred language based on the French conversation. In this case, the preferred language setting implementation may be deactivated. In some aspects, the computing device may automatically prompt the user to activate or deactivate the computing device to avoid toggling between two or more languages.
When selecting a preferred language, the device may perform an audible check of the language selection with the user. For example, the device may output audio such as “I hear you speaking English. Is English your preferred language?” The device may process a verbal response from the user using language recognition techniques or speech recognition techniques to confirm the language choice. The device may also perform audible checks in other circumstances, such as when a user's spoken language may differ with a language detected in other content/configurations. For example, the device may output audio such as “I hear you speaking English, however your emails are in French. Would you prefer your device be set to English or French?”
In another aspect, the device may be configured to divide language preference settings, such as displaying all text in a first language but engaging in speech communications with a user in a second language.
In one aspect of the disclosure, setting the preferred language of the computing device may be based at least in part on the location of the computing device. The location of the computing device may be identified based on global position systems (GPS) or any other computing device location system. For example, a preferred language of a computing device may be set to English, when the GPS coordinates indicate that the computing device is in North Dakota. In some aspects of the disclosure, the user may be prompted to select between two or more languages corresponding to the location of the computing device. For example, when the GPS coordinates identify a multilingual region/state/country the user may be prompted to select between the languages associated with the multilingual region/state/country.
Although aspects of the present disclosure describes setting a preferred language of the computing device, the present disclosure is not so limited. In some aspects, other setting and/or configurations of the computing device may be set or activated using a communication of the user. For example, speech from the user may be processed in conjunction with the speech recognition/language technique to initially set up features of the computing device, such as naming the computing device and setting up wireless local area network. Other applications of the present teachings are also possible.
The above aspects of the present disclosure are meant to be illustrative. They were chosen to explain the principles and application of the disclosure and are not intended to be exhaustive or to limit the disclosure. Many modifications and variations of the disclosed aspects may be apparent to those of skill in the art. For example, the detection techniques described herein may be applied to many different languages. Persons having ordinary skill in the field of computers, digital imaging, and/or content conversion, should recognize that components and process steps described herein may be interchangeable with other components or steps, or combinations of components or steps, and still achieve the benefits and advantages of the present disclosure. Moreover, it should be apparent to one skilled in the art, that the disclosure may be practiced without some or all of the specific details and steps disclosed herein.
Aspects of the present disclosure may be implemented as a computer implemented method, a system, or as an article of manufacture such as a memory device or non-transitory computer readable storage medium. The computer readable storage medium may be readable by a computer and may comprise instructions for causing a computer or other device to perform processes described in the present disclosure. The computer readable storage medium may be implemented by a volatile computer memory, non-volatile computer memory, hard drive, solid state memory, flash drive, removable disk, and/or other media.
Aspects of the present disclosure may be performed in different forms of software, firmware, and/or hardware. Further, the teachings of the disclosure may be performed by an application specific integrated circuit (ASIC), field programmable gate array (FPGA), or other component, for example.
Aspects of the present disclosure may be performed on a single device or may be performed on multiple devices. For example, program modules including one or more components described herein may be located in different devices and may each perform one or more aspects of the present disclosure. As used in this disclosure, the term “a” or “one” may include one or more items unless specifically stated otherwise. Further, the phrase “based on” is intended to mean “based at least in part on” unless specifically stated otherwise.
Number | Name | Date | Kind |
---|---|---|---|
5440615 | Caccuro | Aug 1995 | A |
5526268 | Tkacs et al. | Jun 1996 | A |
5758023 | Bordeaux | May 1998 | A |
6069939 | Fung et al. | May 2000 | A |
6424935 | Taylor | Jul 2002 | B1 |
H0002098 | Morin | Mar 2004 | H |
6785654 | Cyr | Aug 2004 | B2 |
6963836 | Van Gestel | Nov 2005 | B2 |
7133830 | Hoban et al. | Nov 2006 | B1 |
7185289 | Taima | Feb 2007 | B1 |
7277732 | Chen et al. | Oct 2007 | B2 |
7286990 | Edmonds et al. | Oct 2007 | B1 |
7340390 | Hurst | Mar 2008 | B2 |
7343290 | Breuer | Mar 2008 | B2 |
7447635 | Konopka | Nov 2008 | B1 |
7475390 | Berstis et al. | Jan 2009 | B2 |
7822434 | Scott | Oct 2010 | B2 |
7835901 | Muramatsu et al. | Nov 2010 | B2 |
7843330 | Chamberlain et al. | Nov 2010 | B2 |
7850640 | Williams et al. | Dec 2010 | B2 |
7852416 | Bennett et al. | Dec 2010 | B2 |
7949517 | Eckert et al. | May 2011 | B2 |
7953879 | Nagaralu | May 2011 | B1 |
7979442 | Hyman | Jul 2011 | B2 |
8005680 | Kommer | Aug 2011 | B2 |
8041372 | Minns | Oct 2011 | B1 |
8065134 | Naito | Nov 2011 | B2 |
8069030 | Iso-Sipila | Nov 2011 | B2 |
8099289 | Mozer et al. | Jan 2012 | B2 |
8099290 | Suzuki et al. | Jan 2012 | B2 |
8126703 | Runge et al. | Feb 2012 | B2 |
8161073 | Connor | Apr 2012 | B2 |
8170868 | Gamon | May 2012 | B2 |
8229743 | Carter et al. | Jul 2012 | B2 |
8234111 | Lloyd et al. | Jul 2012 | B2 |
8238525 | Zhang et al. | Aug 2012 | B2 |
8279861 | Chao-Suren et al. | Oct 2012 | B2 |
8285537 | Tanner et al. | Oct 2012 | B2 |
8290701 | Mason et al. | Oct 2012 | B2 |
8359234 | Vieri | Jan 2013 | B2 |
8364134 | Rang | Jan 2013 | B2 |
8380505 | Konig | Feb 2013 | B2 |
8446901 | Merino Gonzalez et al. | May 2013 | B2 |
8479112 | Clow | Jul 2013 | B2 |
8498725 | Petit-Huguenin | Jul 2013 | B2 |
8510103 | Angott | Aug 2013 | B2 |
8515728 | Boyd | Aug 2013 | B2 |
8515762 | Krumel | Aug 2013 | B2 |
8560326 | Gao | Oct 2013 | B2 |
8612224 | Breslin et al. | Dec 2013 | B2 |
8635058 | Patel | Jan 2014 | B2 |
8635068 | Pulz et al. | Jan 2014 | B2 |
8666014 | Suzuki | Mar 2014 | B2 |
8666750 | Buck et al. | Mar 2014 | B2 |
8677463 | Piepenbrink et al. | Mar 2014 | B2 |
8793118 | Srinivasa | Jul 2014 | B2 |
8813059 | Sasaki | Aug 2014 | B2 |
8837687 | Odinak et al. | Sep 2014 | B2 |
8874429 | Crosley | Oct 2014 | B1 |
8879701 | Phadnis et al. | Nov 2014 | B2 |
8880631 | Logan | Nov 2014 | B2 |
8886453 | Cerecke et al. | Nov 2014 | B2 |
8909532 | Cross, Jr. | Dec 2014 | B2 |
8913193 | Fukunaka | Dec 2014 | B2 |
8924215 | Pulz et al. | Dec 2014 | B2 |
8924853 | Weng et al. | Dec 2014 | B2 |
8929853 | Butler | Jan 2015 | B2 |
8942974 | Leblang | Jan 2015 | B1 |
8972268 | Waibel | Mar 2015 | B2 |
8972967 | Sannidhanam | Mar 2015 | B2 |
8976944 | Doulton | Mar 2015 | B2 |
8977974 | Kraut | Mar 2015 | B2 |
8983038 | Boutcher et al. | Mar 2015 | B1 |
9002699 | Qin | Apr 2015 | B2 |
9020760 | Fryer et al. | Apr 2015 | B2 |
9020824 | Govil | Apr 2015 | B1 |
9021245 | Guidotti | Apr 2015 | B2 |
9031293 | Kalinli-Akbacak | May 2015 | B2 |
9031839 | Thorsen et al. | May 2015 | B2 |
9043205 | Mengibar et al. | May 2015 | B2 |
9094702 | Bhogal | Jul 2015 | B2 |
9098197 | Suese | Aug 2015 | B2 |
9129591 | Sung | Sep 2015 | B2 |
9138591 | Abbenhouse | Sep 2015 | B2 |
9160967 | Wang | Oct 2015 | B2 |
9165071 | Myslinski | Oct 2015 | B2 |
9177545 | Maruta | Nov 2015 | B2 |
9251144 | Du | Feb 2016 | B2 |
9275635 | Beaufays | Mar 2016 | B1 |
9368104 | Eller | Jun 2016 | B2 |
9414004 | Tran | Aug 2016 | B2 |
9471567 | Duyaguit | Oct 2016 | B2 |
9483461 | Fleizach | Nov 2016 | B2 |
9489375 | Koll | Nov 2016 | B2 |
9502025 | Kennewick | Nov 2016 | B2 |
9697822 | Naik | Jul 2017 | B1 |
9747510 | Jordan | Aug 2017 | B2 |
9754589 | Mathias | Sep 2017 | B2 |
20020186328 | Nishida | Dec 2002 | A1 |
20100223050 | Kelly | Sep 2010 | A1 |
20110153325 | Ballinger et al. | Jun 2011 | A1 |
20120010886 | Razavilar | Jan 2012 | A1 |