Patent Application
20190026266
The present disclosure relates to a translation device and a translation system.
PTL 1 discloses a communication system equipped with a translation unit that translates an input voice. This communication system is equipped with: a microphone to input a speech voice, spoken by a person concerned, in a video-conference device or the like; and an image memory in which an image of an image signal taken by a television camera is temporarily accumulated. In this communication system, when a voice signal is communicated, the voice is output after being translated. In addition, an image having been input corresponding to the input voice is displayed in synchronism with outputting of the translation of the voice. This operation makes it possible to read a facial expression and the like of a communication partner from the image corresponding to the translated voice in a video-conference and the like.
PTL 2 discloses a technique to enable a smooth conversation between users talking with each other through a speech translation device. This speech translation device is equipped with a speech recognition unit that performs speech recognition of a voice signal in a first language; a first machine translation unit that translates the recognized first language into a second language; and a second machine translation unit that reversely translates the translated second language into the first language. This speech translation device voice-outputs a synthetic voice of a result of the machine translation and a synthetic voice of a result of the reverse translation in synchronism with each other. These outputs voices enable one user to confirm, while the other user is listening to the synthetic voice of the result of the machine translation of the voce emitted by the one user, whether the content of the speech spoken by the one user is correctly translated.
PTL 1: Japanese Patent No. 2664679
PTL 2: Unexamined Japanese Patent Publication No. 2015-060332
The present disclosure provides a translation device and a translation system that make it possible for a user speaking words as a translation object to easily communicate a content of the user's intention.
A translation device in one aspect of the present disclosure includes: an operation input unit; a voice input unit; a translation information obtaining unit; an information output unit; an execution unit; and a controller. A user's operation is input to the operation input unit. A voice is input to the voice input unit. The translation information obtaining unit obtains a translation result of the voice having been input to the voice input unit. The information output unit outputs the translation result. The controller causes the execution unit to perform, in synchronization with outputting of the translation result by the information output unit, a processing based on an operation content of the user's operation having been input to the operation input unit while the voice corresponding to the translation result was being input.
A translation system in another aspect of the present disclosure includes a translation device and a translation server. The translation server receives voice information representing a voice having been input to a voice input unit from the translation device, and performs a translation process on the voice information.
With the translation device and the translation system in the present disclosure, when a user's operation is input while a voice as a translation object was being input, a processing based on a content of the user's operation is output in synchronization with outputting of a translation result. This operation can make it easy for the user speaking a speech as a translation object to communicate the content of the user's intention.
Hereinafter, exemplary embodiments will be described in detail with reference to the drawings as appropriate. However, unnecessarily detailed description will not be given in some cases. For example, a detailed description of a well-known matter and a duplicated description of substantially the same configuration are omitted in some cases. This is to avoid the following description from being unnecessarily redundant and thus to help those skilled in the art to easily understand the description.
Note that the applicant provides the attached drawings and the following description to help those skilled in the art to sufficiently understand the present disclosure, and the applicant does not intend to use the drawings or the description to limit the subject matter of the claims.
A configuration of a translation system according to the present exemplary embodiment is described with reference to
Translation system 20 according to the present exemplary embodiment includes translation device 1 and translation server 2 as shown in
Referring back to
Operation input unit 11 accepts various user's operations provided by the user operating translation device 1. Operation input unit 11 includes, for example, a touch sensor on touch panel 11A. When a touch operation is input with a display surface on touch panel 11A being touched, operation input unit 11 generates a signal representing, for example, a touch position by coordinates and outputs the signal to display controller 141 or the like.
Operation input unit 11 is not limited to touch panel 11A and may include various input devices such as a mouse and a keyboard. Further, touch panel 11A may be configured to be able to input, for example, a gesture operation in the vicinity of the display surface. The user's operation includes various operations such as a touch operation, a gesture operation, a mouse operation, and a keyboard operation. Operation input unit 11 obtains an operation content of the user's operation expressed by, for example, a history of the touch position.
Voice input unit 12 is configured with, for example, microphone 12A. Voice input unit 12 receives a voice input to generate voice information (voice data) representing the input voice. The input voice includes, for example, a voice spoken by the user.
Display 13 is, for example, a liquid crystal display, an organic electroluminescence (EL) display, or other displays, which constitute the display surface of touch panel 11A. Display 13 displays various information, under control of display controller 141. For example, display 13 displays a screen of a predetermined application software and an image corresponding to the content of the user's operation. Display 13 is an example of an execution unit of translation device 1 in the present exemplary embodiment.
Controller 14 wholly controls the operations of various parts in translation device 1. Controller 14 is configured with, for example, CPU 14A implementing a predetermined function in cooperation with software. Controller 14 reads out data and programs stored in storage 15 to perform various arithmetic processing and thus implements various functions of display controller 141, registration unit 142, operation playback unit 143, and the like. Further, controller 14 has an internal memory configured with, for example, a static random access memory (SRAM), a dynamic random access memory (DRAM), or the like.
Display controller 141 controls a display operation on display 13, on the basis of a signal from operation input unit 11. For example, display controller 141 determines, on touch panel 11A, a touch position at which a touch operation is being performed, and performs control to display an image corresponding to a trajectory of the touch position.
Registration unit 142 writes, in storage 15, information temporarily stored in the internal memory of controller 14 or other information to register the information in history database D1. History database D1 is a database that associates an operation content of a user's operation with an input voice to manage a history of the operation content. History database D1 will be described later in detail.
Operation playback unit 143 generates, for example, a predetermined command on the basis of the information stored in history database D1. By this operation, operation playback unit 143 sets an operation for playing back the operation content of the user's operation.
Note that, controller 14 may be configured with a hardware circuit such as a dedicated electronic circuit designed to realize a predetermined function or a reconfigurable electronic circuit. Controller 14 may be configured with various semiconductor integrated circuits such as a CPU, a micro processing unit (MPU), a microcontroller, a digital signal processor (DSP), a field programmable gate array (FPGA), or an application specific integrated circuit (ASIC).
Storage 15 is a storage medium that stores a program and data necessary to realize a function of translation device 1. For example, storage 15 stores history database D1 described above. Storage 15 includes memory 15A that is a high-speed memory such as an SRAM or a DRAM or is a flash memory.
In the present exemplary embodiment, translation information obtaining unit 16 obtains translation information containing a translation result of a specific input voice through data communication with translation server 2 via network 3. Translation information obtaining unit 16 includes communication module 16A that performs communication based on, for example, a predetermined communication standard. The predetermined communication standard includes communication standards such as IEEE 802.3 and IEEE 802.11a/11b/11g/11ac. Translation information obtaining unit 16 may be realized in cooperation with software such as a control program on CPU 14A.
Voice output unit 17 is configured with, for example, speaker 17A. Voice output unit 17 voice-outputs a translation result represented by, for example, the translation information obtained by translation information obtaining unit 16. Voice output unit 17 is an example of an information output unit of translation device 1 in the present exemplary embodiment.
Translation server 2 is configured with one or a plurality of server devices such as an application service provider (ASP) server, for example. Translation server 2 transmits and receives voice information to and from translation device 1 via network 3. Translation server 2 performs processes such as a speech recognition process, a translation process, and a voice synthesizing process on the basis of the received voice information. In translation server 2, the server devices performing the processes may be integrally configured or separately configured. Alternatively, translation server 2 may perform cloud computing.
Hereinafter, operations of the translation system and the translation device according to the present exemplary embodiment are described.
An operation of translation system 20 according to the present exemplary embodiment is generally described with reference to
The usage examples of
For example, in the state of
Further, after the voice is output as
It is supposed that in the case that the host uses translation device 1 to give directions as described above, if the host explains while pointing a finger at a road on a map, the host can easily transfer a content of what the host wants to explain to the guest. Therefore, in translation system 20 according to the present exemplary embodiment, when the host gives various guiding to the guest while using machine translation, translation device 1 voice-outputs the translation results and at the same time displays auxiliary information such as a map, a photograph, how to use something, and where something is. The usage examples of
In the example of
For example, as shown in
Further, in the example of
Further, in the examples of
Hereinafter, the operations of the translation system and the translation device according to the present exemplary embodiment are described in detail.
The operation of translation system 20 according to the present exemplary embodiment is described with reference to
Part (a) of
In the sequence of
In the example of
If translation server 2 receives the voice information and the identification information from translation device 1, translation server 2 performs a speech recognition process for speech recognition of the speech sentence represented by the received voice information (step S21). In this process, translation server 2 generates, for example, text data in Japanese as information representing the speech sentence having undergone speech recognition. Further, translation server 2 manages the generated text data in association with the identification information (t1) received together with the voice information.
Next, translation server 2 performs a translation process on the basis of the generated text data (step S22). In the examples of
Next, translation server 2 performs the voice synthesizing process to synthesize a voice representing the translation sentence represented by the text data on the basis of the generated text data of the translation sentence (step S23). In the examples of
Upon receiving the translation information from translation server 2, translation device 1 performs a playback process for simultaneously playing back the voice information of the translation result and the operation content of the corresponding user's operation (step S3).
In the examples of
When the process of step S3 is performed by translation device 1, the process of translation system 20 in this sequence is finished.
By the above processes, in translation system 20, translation server 2 performs machine translation, and translation device 1 can input a speech of a user and can output a translation result in synchronization with inputting and outputting of a user's operation.
In the above processes, each of the processes of steps S21, S22, and S23 may be performed on different server devices included in translation server 2, for example, by adding identification information when each process inputs and outputs.
Transmission and reception of data among steps S21 to S23 may be performed within translation server 2. Alternatively, every time when each process is completed, translation server 2 may return data back to translation device 1, and translation device 1 may transmit the data to the next processing server. This arrangement makes it possible to use speech recognition and translation of different service companies, for example.
Alternatively, part or all of the speech recognition process (step S21), the translation process (step S22), and the voice synthesizing process (step S23) may be performed by translation device 1 instead of translation server 2.
The input process on translation device 1 in step S1 of
The process according to the flowchart of
With respect to the flowchart of
If controller 14 detects that a speech is started (step S11: Yes), controller 14 obtains a start time at which the speech is started (step S12). For example, in the example of
Next, controller 14 records, for example, in the internal memory, an input voice being input from voice input unit 12 and an operation content of a user's operation being input, together with the input voice, from operation input unit 11 (step S13). The process of step S13 is a process in which, in order to register the input voice and the operation content collectively in history database D1, the input voice and the operation content are temporarily stored while the user is speaking.
Next, controller 14 detects whether the user has finished speaking (step S14). In the present exemplary embodiment, the detection process of step S14 is performed on the basis of a signal generated by operation input unit 11 in response to an operation of pushing down the speech end button. Controller 14 repeats (step S14: No) the processes of steps S13 and S14 with a predetermined cycle, for example, 1/60 seconds until the speech is detected to be finished.
In the example of
If controller 14 detects that the speech is finished (step S14: Yes), controller 14 obtains an end time at which the speech is finished (step S15). For example, when the user pushes down the speech end button at time t2 after speaking one speech sentence (part (a) of
Next, controller 14 functions as registration unit 142 and registers the temporarily stored input voice and operation content of the user's operation in history database D1 in association with each other (step S16). In step S16, controller 14 writes the information temporarily stored in the internal memory in storage 15. In the present exemplary embodiment, registration unit 142 registers information in history database D1, using a start time and an end time obtained. History database D1 is described with reference to
In the present exemplary embodiment, history database D1 manages a “start time,” an “end time,” an “input voice,” and an “operation content” in association with one another as shown in
By using history database D1 exemplified in
Next, controller 14 transmits the voice information of the input voice registered in history database D1 to translation server 2 via translation information obtaining unit 16 (step S17). In the present exemplary embodiment, controller 14 refers to history database D1 and adds, as identification information, the start time associated with the input voice to the voice information of the input voice. Then, controller 14 transmits the voice information to translation server 2 (see
By performing the process of step S17, controller 14 of translation device 1 completes the process according to the flowchart. Then, the process on translation system 20 goes to step S21 of
The above process enables the user to speak a content of a translation object and, at the same time, to input, in translation device 1, a user's operation corresponding to the content of the speech. Translation device 1 can accumulate the operation content of the user's operation performed simultaneously with the speech, in association with the input voice of the speech.
Further, for example, in the examples of
In the above description, history database D1 exemplified in
A description will be given to a case in which an operation is performed in the user's operation from time t1 to time t2 of part (c) of
In this case, controller 14 records the display magnification at time t1 in history database D1 as the “application state.” Therefore, controller 14 obtains the display magnification at time t1 when the playback display is started to be performed at time t21, and controller 14 changes the display magnification of the map application to the obtained value.
In the above description, the start time of a speech is used as the identification information by which an input voice and a user's operation are associated with each other; however, the present disclosure is not limited to above example, and another time information may be used, for example, an end time. Further, the identification information may be various identification information other than time information, and, for example, an identification data (ID) may be added to each single speech sentence, where the ID is incremented in order of speech.
The detection process for the start of a speech on step S11 is performed in response to the detection of the pushing down of the speech start button; however, the present disclosure is not limited to this example. The detection process for the start of a speech may be performed depending on, for example, a speech analysis result of a voice being input by voice input unit 12. For example, the detection process for the start of a speech is performed when translation device 1 detects a predetermined phrase meaning the start of a speech.
The detection process for the end of a speech on step S14 is also performed in response to the detection of the pushing down of the speech end button; however, the present disclosure is not limited to this example. The detection process for the end of a speech may be performed depending on, for example, a speech analysis result. For example, the detection process for the end of a speech may be performed in response to the detection, of a conjunction, by translation device 1 via voice input unit 12, or in response to the detection of a silence state for a predetermined period or longer.
Further, in the process of step S17, translation device 1 does not need to transmit all of the information registered in history database D1 to translation server 2. For example, translation device 1 may transmit only the information of the input voice to translation server 2. At this time, for example, translation device 1 may be configured not to accept a new input voice until receiving a translation result of the transmitted input voice.
In the above description, the process of step S13 of
The playback process of translation device 1 in step S3 of
First, controller 14 determines whether translation information obtaining unit 16 has received translation information from translation server 2 (step S31). For example, in the examples of
If translation information obtaining unit 16 receives the translation information (step S31: Yes), controller 14 obtains the time, which is the identification information contained in the received translation information, as a key for searching in history database D1 (step S32). In the above example, controller 14 obtains, as a key, start time t1 of the input voice of the translation source.
Next, controller 14 functions as operation playback unit 143 and searches history database D1 to obtain the operation content associated with the time of the obtained key (step S33). In the examples of
Further, controller 14 functions as operation playback unit 143 and sets a display operation (playback operation), of playback display, based on the obtained operation content (step S34). In the examples of
Further, in the above example, operation playback unit 143 issues, to display controller 141, a command representing the same content as the signal generated by operation input unit 11 in step S13, for example (step S34). On the basis of the command, display controller 141 causes display 13 to display an arrow image so as to reproduce the arrow image displayed from time t1, at which the user started a touch operation (see parts (c) and (f) of
Next, controller 14 generates a synchronization signal for synchronizing, for example, the operation of voice output unit 17 and the operation of display controller 141. Then, on the basis of the synchronization signal, controller 14 causes the voice output of the voice information of the translation result included in the received translation information and the playback display corresponding to the operation content to be displayed in synchronization with each other (step S35). Note that controller 14 does not necessarily have to cause the playback display to be output simultaneously with the voice output. Controller 14 may cause the voice output and the playback display in synchronization with each other such that the playback display is output a predetermined period after the voice output.
In the examples of
Controller 14 completes the process according to the flowchart by performing the process of step S35.
By the above process, the operation content of the user's operation associated with the input voice is played back and displayed in synchronization with the voice output of the translation result of the input voice (step S35). By this operation, for example, the operation content when the host spoke is played back and displayed in synchronization with the voice output of the translation result. Therefore, the guest listening to the translation result of the host's speech can more easily understand the host's intention.
In step S35 described above, for example, when the input period of the input voice is longer than the voice output period of the translation result, translation device 1 may extend or repeatedly play back the playback display of the operation content, depending on the length of the voice output period. Alternatively, when the period of the input voice is shorter than the voice output period of the translation result, translation device 1 may thin out or play quicker the playback display.
For example, in step S34, operation playback unit 143 of translation device 1 calculates the input period of the input voice on the basis of the start time and the end time registered in history database D1. Further, operation playback unit 143 obtains the voice output period of the translation result on the basis of the translation information obtained by translation information obtaining unit 16. Operation playback unit 143 compares the input period and the voice output period and sets a setting of, for example, extending or thinning out of the playback display. By this operation, operation playback unit 143 performs setting of the playback operation of display 13 such that the voice output period of the translation result and the playback display of the operation content are in synchronization with each other.
Note that in the case that the operation based on the operation content is set without particularly comparing the input period of the input voice and the voice output period of the translation result, history database D1 does not need to manage both of the start time and the end time. In this case, the process of step S12 or step S15 may be omitted in the input process (
As described above, in the present exemplary embodiment, translation device 1 includes operation input unit 11, voice input unit 12, translation information obtaining unit 16, voice output unit 17 as an example of an information output unit, display 13 as an example of an execution unit, and controller 14. A user's operation is input to operation input unit 11. A voice is input to voice input unit 12. Translation information obtaining unit 16 obtains a translation result of the voice having been input to voice input unit 12. Voice output unit 17 outputs the translation result. Display 13 performs a display operation as an output operation corresponding to the user's operation. Controller 14 controls the processing by display 13. Controller 14 causes display 13 to perform, in synchronization with outputting of the translation result by voice output unit 17, a processing based on an operation content of the user's operation having been input to operation input unit 11 while the voice corresponding to the translation result was being input.
Further, in the present exemplary embodiment, voice output unit 17 outputs, as a voice, the translation result.
With translation device 1 described above, a user's operation corresponding to an intended content is input to translation device 1 while a voice as a translation object is being input, and the operation content of the user's operation having been input is output in synchronization with the voice output of the translation result. By this operation, the user, who speaks the translation object, can easily communicate the host's intended content to a partner, who listens to the voice output of the translation result.
In the present exemplary embodiment, controller 14 sets the operation based on the operation content such that the period of the translation result being output and the period of the operation based on the operation content of the user's operation being performed are in synchronization with each other. By this setting, the partner listening to the voice output of the translation result can easily confirm the operation content being output.
Further, in the present exemplary embodiment, controller 14 causes display 13 to perform a processing based on the operation content of the user's operation in synchronization with the input of the user's operation to operation input unit 11. By this operation, for example, the user can confirm the processing being output by display 13 while performing a user's operation like the arrow image shown in
Further, in the present exemplary embodiment, the processing synchronized with outputting of the translation result includes an operation for reproducing a processing synchronized with inputting of the user's operation to operation input unit 11. By this operation, for example, like the arrow image in
Further, in the present exemplary embodiment, the processing synchronized with outputting of the translation result includes a processing expressing that a user's operation is being performed. This operation can give an impression to the guest as if the operation content of the user's operation having been input is being actually performed during the voice output of the translation result, for example, like the icon displayed in
In addition, in the present exemplary embodiment, translation device 1 further includes storage 15. Storage 15 records therein history database D1 that manages the voice information representing the voice having been input to voice input unit 12 and the user's operation having been input to operation input unit 11 while a voice was being input in association with each other. Controller 14 refers to history database D1 recorded in storage 15 and causes display 13 to perform a processing based on the operation content of the user's operation in synchronization with the voice output of the translation result of the voice represented by the voice information. By this operation, translation device 1 can manage the operation content synchronized with the translation information, on the basis of history database D1 recorded in storage 15.
Further, in the present exemplary embodiment, the execution unit of translation device 1 includes display 13 that displays information. Controller 14 causes display 13 to display the information corresponding to the operation content of the user's operation in synchronization with the voice output of the translation result. By this operation, the user can visually confirm the information corresponding to the operation content.
Further, in the present exemplary embodiment, translation system 20 includes translation device 1 and translation server 2. Translation server 2 receives, from translation device 1, voice information of a voice having been input to voice input unit 12 and performs a translation process on the voice information. With this arrangement, translation device 1 can obtain translation information from translation server 2 and can voice-output the translation information.
With reference to
In this example, as shown in
In the example of
At this time, controller 14 of translation device 1 detects a break between two voices V3 and V4 by, for example, detecting silence or a conjunction on the basis of the input voice from voice input unit 12, detecting a change of the touch position on the basis of the signal form operation input unit 11, or the like. By this operation, in translation device 1, voices V3 and V4 are sequentially registered in history database D1 in association with the first and second coordinates (10, 20), (30, 50), respectively. Translation device 1 transmits, to translation server 2, the voice information of each of the input voices at appropriate times together with identification information. Then, translation device 1 sequentially obtains the translation information of each of the translation results.
Translation device 1 receives the translation information of translation result T3 “This is post office” corresponding to voice V3 and then voice-outputs translation result T3. As shown in
Further, the above-described playback display is sequentially performed at the time of the voice outputting of the translation result regardless of the state of host-use screen 13a. Therefore, the host can perform a user's operation and speak without paying any attention to the playback display, and can easily use translation device 1. Further, by using real time translation as described above, for example, in the case that instead of a host, an attendant of a conference uses translation device 1, the attendant can perform a user's operation and speak without disturbing the flow of the conference. Therefore, the attendant of the conference can easily use translation device 1.
Hereinafter, a second exemplary embodiment is described with reference to
Hereinafter, translation device 1A and translation system 20A according to the present exemplary embodiment are described while omitting description of the same configuration and operation as in translation device 1 and translation system 20 according to the first exemplary embodiment. Note that in
Control signal transmitter 18 transmits a control signal for controlling an operation of predetermined device 4, for example, in infrared communication. Control signal transmitter 18 includes, for example, an infrared light emitting diode (LED) and a transmitter circuit. Control signal transmitter 18 is an example of a signal transmitter according to the present exemplary embodiment and an example of the execution unit in translation device 1A.
Device 4 may be a display that performs screen display when translation information in translation device 1A is output. Further, device 4 may be various electronic devices, home appliances such as an air conditioner and a television set, and working devices such as a manipulator of a robot. Further, an operation object, of device 4, based on the above control signal may be, for example, a button, a lever, and a remote panel.
Translation device 1A according to the present exemplary embodiment can be used, for example, for a host to instruct a guest or the like how to operate device 4. For example, display 13 displays a graphical user interface (GUI) representing device 4 and other graphics, and the host operates the GUI representing device 4 via operation input unit 11 while speaking. For example, the host operates the GUI, speaking an instruction of a series of processing procedures, for example, “First, turn on the power button of device 4; next, flip down the left lever to the far side; further, flip down the right lever to the right.”
At this time, translation device 1A obtains translation information representing a translation result of an input voice of a speech in the same way as in the first exemplary embodiment. Next, translation device 1A according to the present exemplary embodiment transmits a control signal for controlling such that device 4 performs, in synchronization with a voice output of the translation information, an operation corresponding to an operation content during a speech of the user. Then, device 4 performs, at the time of the voice outputting of the translation result, the same processing operation as the processing procedures performed on the GUI of device 4. By this operation, the guest can confirm the actual processing of device 4 at the time of the voice outputting of the translation result and can thus easily understand the content of the translation result.
As described above, in the present exemplary embodiment, the execution unit of translation device 1A includes control signal transmitter 18 that transmits a control signal to device 4. Controller 14 causes control signal transmitter 18 to transmit, to device 4, the control signal for controlling the operation of device 4, in synchronization with the voice output of the translation result.
By this operation, the actual processing of device 4 can be confirmed in synchronization with the voice output of the translation result, and the content intended by the user who performed the operation can be easily communicated.
In the above description, the operation object of the host is the GUI, on display 13, representing device 4; however, the operation object does not have to be the GUI and may by actual device 4, for example. For example, when the host operates device 4, translation device 1A transmits a control signal from controller 14 to device 4. In this case, control signal transmitter 18 functions as the execution unit also at the time of the operation of the user, and display 13 may be appropriately omitted in translation device 1A.
Hereinafter, a third exemplary embodiment is described with reference to
Hereinafter, translation device 1B and translation system 20B according to the present exemplary embodiment are described while omitting a description of the same configuration and operation as in translation devices 1, 1A and translation systems 20, 20A according to the first and second exemplary embodiments. Note that in
Before a control signal is transmitted, controller 14 of translation device 1B according to the present exemplary embodiment starts a translation operation based on an input voice from voice input unit 12 in response to a pressing down operation of the speech start button by the user in the same manner as in the second exemplary embodiment (see
In the display state of
In the present exemplary embodiment, when device 4 has completed the operation based on the control signal received from translation device 1B, device 4 transmits the completion signal to translation device 1B. This arrangement can prevent a new control signal from being transmitted to device 4 before device 4 finishes operation. Therefore, in translation device 1B, it is possible to adjust timing for accepting a new input voice as a translation object.
As described above, in the present exemplary embodiment, translation device 1B further includes completion signal receiver 19 that receives an operation completion signal from device 4. Controller 14 restricts inputting of a voice to voice input unit 12 in a period from transmission of a control signal to reception of a completion signal from device 4 via completion signal receiver 19, where the completion signal indicates completion of the operation controlled by the control signal.
This operation realizes a function that restricts an input process of translation device 1B, depending on the operation state of device 4, and the user can thus easily explain the content corresponding to the operation
In the above description, an example is described in which the speech start button displayed on display 13 is used to restrict the input process on the basis of the operation completion signal; however, the input process may be restricted on the basis of the operation completion signal without using the displayed speech start button. Controller 14 may restrict the input process from being performed and restrict inputting of a voice to voice input unit 12, for example, after the control signal is transmitted from control signal transmitter 18 to device 4 until the completion signal is received from device 4.
In the above, the first to third exemplary embodiments are described as examples of the technique disclosed in the present application. However, the technique in the present disclosure is not limited to the above exemplary embodiments and can also be applied to an exemplary embodiment in which modification, replacement, addition, removal, or the like is performed appropriately. Additionally, the components described in the above exemplary embodiments can be combined to configure a new exemplary embodiment. Therefore, other exemplary embodiments are described below as examples.
In the above exemplary embodiments, an example is described in which an input voice and an operation content of a user's operation are associated with each other for each speech sentence, in translation device 1. It is also possible to associate an input voice and an operation content of a user's operation with each other by a predetermined unit such as a phrase or a word into which a speech sentence can be divided. This modified example is described with reference to
To address this issue, translation device 1 changes the order of playback of operation contents A5, A6, and A7, corresponding to the word order of the translation results on the basis of operation contents A5, A6, and A7 associated as described above. For example, at a time of voice outputting of the translation result in the first pattern, as shown in
In this case, translation device 1 previously obtains a relationship between words before and after the translation such as the relationship between translation result T7 “Please turn” and voice V7. The relationship between words may be stored in storage 15 of translation device 1 or may be issued by translation server 2, for example.
As described above, by changing the order of playback of the operation contents, depending on the word order of the translation result, the operation contents during voice outputting are played back corresponding to the word order of the translation result. Therefore, this operation facilitates understanding of the user listening to the voice of the translation result.
In the above exemplary embodiments, examples are described in each of which the translation source language and the translation target language are respectively Japanese and English; however, the translation source language and the translation target language are not particularly limited, and various natural languages may be used.
Further, in the above exemplary embodiments, examples are described in each of which translation information obtaining unit 16 of translation device 1 obtains translation information from translation server 2 that performs a translation process; however, translation device 1 (controller 14) may perform a translation process on an input voice. In this case, translation information obtaining unit 16 obtains, as translation information, a translation result translated by the translation process having been performed as an internal process of translation device 1.
Further, in the above exemplary embodiments, a description is given to translation system 20 that include a single information processing terminal as translation device 1; however, translation system 20 may include a plurality of information processing terminals that functions as translation device 1. For example, the host and the gest each have the above information processing terminals. In this case, translation server 2 and network 3 may be used to transmit and receive various information between the information processing terminals.
Further, in the above exemplary embodiments, a description is given taking translation device 1 as an example; however, the spirit of the present disclosure may be applied to a system or a device that performs predetermined speech processing such as noise reduction or speech conversion instead of translation. That is, a system or an electronic device may be configured by replacing translation information obtaining unit 16 of translation device 1 with an obtaining unit that obtains the result of predetermined speech processing on the voice having been input to the voice input unit. In that electronic device, a controller causes the execution unit to perform a processing based on the operation content of the user's operation having been input to the operation input unit while the voice corresponding to the result of the speech processing was being input, in synchronization with the voice output of the speech processing result, which is performed by the voice output unit. This arrangement can provide an electronic device or a system in which it is easy to communicate the content intended by the user who is speaking a speech as a target of the above speech processing.
The above speech processing may be performed in the electronic device or may be performed in a server within the system. Further, the output of the translation result based on an input voice is not limited to a voice output. For example, in the case that a translation result is displayed in text form, controller 14 may cause display 13 to perform a processing based on the operation content of the user's operation having been input to the operation input unit while the voice corresponding to the translation result was being input, in synchronization with the display of texts.
For example, display 13 may output the translation information as text information. More specifically, instead of the voice output of the translation result shown in
As described above, the exemplary embodiments are described as examples of the technique in the present disclosure. For this purpose, the attached drawings and the detailed description are provided.
Therefore, the components illustrated in the attached drawings and described in the detailed description may include, for illustration of the above-described technique, not only components essential for the solution to the problem but also components not essential for the solution to the problem. Thus, it should not be immediately deemed that, merely based on the fact that the components that are not essential are illustrated in the attached drawings and described in the detailed description, the components that are not essential are essential.
In addition, because the above exemplary embodiments are for illustrating the technique in the present disclosure as examples, various modifications, replacements, additions, removals, or the like can be made without departing from the scope of the claims or the equivalent thereto.
The present disclosure can be used as a translation device that voice-outputs a translation result of a speech of a user who treats customers, guides, or gives explanations, for example.
| Number | Date | Country | Kind |
|---|---|---|---|
| 2016-148859 | Jul 2016 | JP | national |
| Number | Date | Country | |
|---|---|---|---|
| Parent | PCT/JP2017/020457 | Jun 2017 | US |
| Child | 16144380 | US |
