This application is based on and claims priority under 35 USC 119 from Japanese Patent Application No. 2016-180318 filed Sep. 15, 2016.
The present invention relates to a dialogue apparatus and method.
According to an aspect of the invention, there is provided a dialogue apparatus including a memory, an estimation unit, and a dialogue unit. The memory associatively stores a certain topic and a change in an affective state of each user before and after a dialogue on that topic. The estimation unit estimates an affective state of a user using information obtained from a detector that detects a sign that expresses the affective state of the user. The dialogue unit extracts, from the memory, a topic where the affective state obtained by the estimation unit matches or is similar to a pre-dialogue affective state and where a target affective state matches or is similar to a post-dialogue affective state, and has a dialogue on the extracted topic with the user.
An exemplary embodiment of the present invention will be described in detail based on the following figures, wherein:
A dialogue system 10 according to an exemplary embodiment of the present invention will be described with reference to
The CPU 201 controls the overall operation of the components of the dialogue-type robot 20 on the basis of a control program stored in the storage device 203. The memory 202 temporarily stores dialogue speeches in a dialogue spoken by the dialogue-type robot 20 with the user 30, dialogue information including the details of the dialogue, a face image of the user, and images of the expression, behavior, and physical state of the user 30 captured by the camera 204. The memory 202 further stores biometrics information, such as the heart rate and the skin resistance, of the user 30, detected by the biometrics sensor 207. The storage device 203 stores a control program for controlling the components of the dialogue-type robot 20. The camera 204 captures changes in the face image, expression, behavior, and physical state of the user 30, and stores these captured changes in the memory 202.
Upon a dialogue with the user, the microphone 205 detects the voice of the user 30, and stores, that is, records, the voice in the memory 202. The memory 202 may alternatively store the details of the dialogue after the details of the voice are analyzed, instead of directly recording the voice. The loudspeaker 206 outputs voice generated by a later-described dialogue controller 212 of the dialogue-type robot 20. The biometrics sensor 207 measures biometrics information, such as the heart rate, skin resistance (skin conductivity), and temperature, of the user 30, and stores the measured data in the memory 202. Sensors according to the exemplary embodiment of the present invention include the camera 204 and the microphone 205 in addition to the biometrics sensor 207, and detect signs that express the affective state of the user 30. The movement device 208 includes wheels and a drive device such as a motor necessary for moving the dialogue-type robot 20 to an arbitrary place, and a current position detector such as a Global Positioning System (GPS) receiver. The camera 204, the microphone 205, and the biometrics sensor 207 function as a detector that detects signs that express the affective state of the user 30.
The person authenticator 211 analyzes the face image of the user 30, captured by the camera 204 and temporarily stored in the memory 202, and compares the face image with the face image of each user 30 stored in the personal information database 217, thereby identifying who the user 30 is. The person authenticator 211 may identify the user 30 by using another authentication method other than the face authentication method. For example, the following biometrics may be adopted: iris authentication that extracts and uses a partial image of the eyes of the user 30 captured by the camera 204, vein authentication and fingerprint authentication that use biometrics information of the user 30 detected by the biometrics sensor 207, and voiceprint authentication that analyzes and uses the voice of the user 30 captured by the microphone 205. In this case, it is necessary to store, in the personal information database 217, iris pattern information, vein pattern information, fingerprint pattern information, and voiceprint pattern information corresponding to each user 30 in accordance with the authentication method to adopt.
The dialogue controller 212 controls a dialogue of the dialogue-type robot 20 with the user 30. Specifically, the dialogue controller 212 applies control to have a dialogue with the user 30 on a topic extracted by the later-described topic extractor 216. The dialogue controller 212 generates a response message to the user 30 in accordance with the extracted topic, and outputs the response message to the loudspeaker 206. The storage device 203 of the dialogue-type robot 20 stores various conversation patterns and speeches in accordance with various topics (not illustrated), and a dialogue with the user 30 is advanced using these conversation patterns in accordance with the dialogue with the user 30. The dialogue-type robot 20 may include a communication function, and the dialogue controller 212 may obtain appropriate conversation patterns and speeches in accordance with the above-mentioned topic from a server connected to the dialogue-type robot 20 and generate response messages.
The affective estimator 213 estimates the current affective state of the user 30 using information on signs that express the affective state of the user 30, detected by the detector, that is, the camera 204, the microphone 205, and the biometrics sensor 207. Specifically, the affective estimator 213 estimates the affective state of the user 30 on the basis of one or more signs that express the affective state of the user 30, which are configured by at least one or a combination of the behavior of the user 30, the physical state such as the face color, expression, heart rate, temperature, and skin conductivity, the voice tone, the speed of the words (speed of the speech), and details of the dialogue in a dialogue between the user 30 and the dialogue-type robot 20.
For example, a change in the face color is detectable from a change in the proportions of red, green, and blue (RGB) of a face image of the user 30, captured by the camera 204. The affective estimator 213 estimates the affective state of the user 30 such that the user 30 is “happy” from a change in the face color, and how greatly the user 30 opens his/her mouth in the face image, captured by the camera 204. The affective estimator 213 estimates the affective state of the user 30 such that the user is “nervous” from changes in the heart rate, temperature, and skin conductivity of the user 30, detected by the biometrics sensor 207, or the user is “irritated” on the basis of changes in the voice tone and the speed of the words of the user 30.
The situation obtainer 214 obtains a situation where the dialogue-type robot 20 is having a dialogue with the user 30, on the basis of the current position information where the dialogue-type robot 20 and the user 30 are having this dialogue, identified by the current position detector of the movement device 208. This situation may be one of large categories such as “public situation” and “private situation”, or of small categories such as “meeting”, “office”, “rest area”, “home”, and “bar”. The situation obtainer 214 compares the identified current position information with spot information registered in advance in the storage device 203, and obtains a situation where the dialogue-type robot 20 and the user 30 are having the dialogue, on the basis of the spot information corresponding to the current position information.
The affective change determiner 215 refers to the affective conversion table 219 on the basis of the situation where the user 30 and the dialogue-type robot 20 are having the dialogue, obtained by the situation obtainer 214, the normal character (original character) of the user 30, stored in the later-described personal information database 217, and the current affective state of the user 30, estimated by the affective estimator 213, and determines a target affective state different from the current affective state of the user 30. That is, the affective change determiner 215 determines what kind of affective state the dialogue-type robot 20 wants to produce in the user 30. Furthermore, the affective change determiner 215 may make the target affective state different in accordance with the intensity of the current affective state estimated by the affective estimator 213.
The topic extractor 216 extracts, from the conversation result database 218, a topic proven to have changed the affective state of the user 30 from the current affective state to the target affective state, on the basis of the current affective state of the user 30, obtained by the affective estimator 213, the target affective state after the change, determined by the affective change determiner 215, and the situation where the dialogue-type robot 20 and the user 30 are having the dialogue. Specifically, the topic extractor 216 extracts, from the conversation result database 218, a topic where the current affective state of the user 30, obtained by the affective estimator 213, matches a pre-dialogue affective state in the conversation result database 218, and where the target affective state matches a post-dialogue affective state in the conversation result database 218.
The personal information database 217 stores information on the face image and the normal character of each user 30 in association with each other.
The conversation result database 218 is a database that associatively stores, in each certain situation, a certain topic and a change in the affective state of each user 30 before and after a dialogue on that topic. In other words, the conversation result database 218 accumulates the record of how each user's affective state has changed when having a dialogue on what topic in what situation.
The affective conversion table 219 associatively stores, for each user 30, the normal character, the current affective state, the intensity of the current affective state, and a target affective state different from the current affective state.
Next, the flow of the operation of the dialogue-type robot 20 according to the exemplary embodiment will be described with reference to
Next in step S702, the affective estimator 213 estimates the affective state of the user 30 using information obtained by a detector that detects signs that express the affective state of the user 30. Specifically, the affective estimator 213 estimates the current affective state of the user 30 and its intensity on the basis of the behavior, face color, and expression of the user 30, captured by the camera 204, the physical states such as the heart rate, temperature, and skin conductivity of the user 30, detected by the biometrics sensor 207, and the voice tone, the speed of the words, and details of the dialogue of the user 30, detected by the microphone 205.
Next in step S703, the affective change determiner 215 determines whether to change the affective state of the user 30. Specifically, the affective change determiner 215 refers whether an affective conversion pattern identified by a combination of the normal character of the user 30, stored in the personal information database 217, and the current affective state of the user 30, estimated in step S702 described above, is included in the affective conversion table 219, and, if there is such an affective conversion pattern, the affective change determiner 215 determines to change the affective state of the user 30, and proceeds to step S704. If there is no such affective conversion pattern, the affective change determiner 215 determines not to change the affective state, and the operation ends.
For example, it is assumed that the user 30 identified in step S701 described above is “Mr. A”, and the current affective state of “Mr. A” estimated in step S702 described above is “depressed”, and its intensity is “moderate”. In that case, the affective change determiner 215 refers to the personal information database 217, identifies that the normal character of “Mr. A” is “active”, and determines whether there is an affective conversion pattern corresponding to the normal character (“active”) of “Mr. A” and the current affective state (“depressed”) of “Mr. A” identified in step S702 described above. Because there is a conversion pattern that includes the normal character “active” and the current affective state “depressed” in the affective conversion table 219, the affective change determiner 215 determines to change the feeing of “Mr. A”, and proceeds to step S704.
In step S704, the affective change determiner 215 refers to the affective conversion table 219, and determines a target affective state, different from the current affective state, corresponding to the normal character of the user 30, the current affective state of the user 30, and its intensity. For example, when the user 30 is “Mr. A”, the affective change determiner 215 refers to the affective conversion table 219 and, because the target affective state after the change in the case where the intensity of the current affective state “depressed” is “moderate” is “calm”, the affective change determiner 215 determines “calm” as the affective state.
In step S705, the situation obtainer 214 identifies a situation where the user 30 and the dialogue-type robot 20 are having the dialogue, on the basis of the current position information detected by the current position detector of the movement device 208. Specifically, the situation obtainer 214 identifies to which of the large categories such as “public situation” and “private situation”, and further of the small categories such as “meeting”, “office”, “rest area”, “home”, and “bar” the situation where the user 30 and the dialogue-type robot 20 are having the dialogue correspond.
In step S706, the topic extractor 216 extracts, from the conversation result database 218, a topic where the affective state of the user 30, estimated by the affective estimator 213, matches a pre-dialogue affective state in the conversation result database 218, and where the target affective state, determined by the affective change determiner 215, matches a post-dialogue affective state in the conversation result database 218, on the basis of the situation where the dialogue is taking place. Specifically, the topic extractor 216 extracts a topic where the current affective state of the user 30 matches a “pre-dialogue affective state” in the conversation result database 218 and where the target affective state after the change matches a “affective state after the change” in the conversation result database 218. For example, it is assumed that, in the above-mentioned example, a situation where “Mr. A” is having a dialogue with the dialogue-type robot 20 is a “public” place and that place is a “rest area”. In this case, reference to the conversation result database 218 clarifies that there has been an actual conversation where, in the “public” situation of the “rest area”, when a dialogue took place on the topics “children” and “school”, the pre-dialogue affective state “depressed” changed to the post-dialogue affective state “calm”. Thus, the topic extractor 216 extracts, from the conversation result database 218, the topics “children” and “school” in order to change the mood of the user 30.
In step S707, the dialogue controller 212 generates dialogue details for having a dialogue with the user 30 on the basis of the extracted topics and outputs the dialogue voice using the loudspeaker 206, thereby having a dialogue with the user 30. In the above-described example, the dialogue controller 212 applies control to have a dialogue with “Mr. A”, who is the user 30, on the topics “children” and “school” extracted in step S706. Next in step S708, the affective estimator 213 monitors the affective state of the user 30, who is the dialogue partner, and estimates the affective state of the user 30 at the time of the dialogue or after the dialogue using the above-mentioned topics.
In step S709, the affective change determiner 215 determines whether the user 30 has changed his affective state to the target affective state, on the basis of the affective state of the user 30 estimated by the affective estimator 213. If the user 30 has changed his affective state to the target affective state, the operation ends. If it is determined that the user 30 has not changed his affective state to the target affective state, the operation proceeds to step S710. Specifically, the affective change determiner 215 determines whether “Mr. A”, who is the user 30, has changed his affective state to “calm”, which is the target affective state, when he had a dialogue with the dialogue-type robot 20 on the topics “children” and “school”. If “Mr. A” has become “calm”, the operation ends. If it is determined that “Mr. A” has not become “calm” yet, the operation proceeds to step S710.
In step S710, the affective change determiner 215 determines the number of times the above-described processing from step S703 to step S709 is performed, that is, the number of dialogues with the user 30 using the topics for changing the affective state of the user 30. If it is determined that the number of times is less than a certain number of times, the operation returns to step S703, repeats the processing from step S703 to step S709, and retries to change the affective state of the user 30. If it is determined in step S710 that the number of dialogues on the topics for changing the affective state of the user 30 is already the certain number, the operation ends.
The operation of the dialogue-type robot 20 for having a dialogue(s) with the user 30 according to the exemplary embodiment has been described as above. In the exemplary embodiment, the case where there is only one user 30 with which the dialogue-type robot 20 has a dialogue has been described. However, the number of dialogue partners of the dialogue-type robot 20 according to the exemplary embodiment of the present invention is not limited to one, and multiple users 30 may serve as dialogue partners. For example, when multiple users 30 gather at one place in order to hold a meeting or the like, the affective change determiner 215 of the dialogue-type robot 20 determines a user 30 whose affective state is to-be changed and a target affective state different from the current affective state of that user 30 of interest, extracts a topic(s) for changing the affective state of that user 30, and has a dialogue(s) with the user 30 on that topic(s) to change the affective state of the user 30.
For example, it is assumed that, in the affective conversion table 219, the affective state of a person whose normal character is “active” and current affective state is “depressed” and “much” is to be changed in preference to the others. In this case, the affective change determiner 215 refers to the affective conversion table 219, gives priority to the affective state of “Mr. A”, and determines to change the affective state from “depressed” and “much” to “happy”. The topic extractor 216 extracts, from the conversation result database 218, a topic where the current affective state of the user 30 whose affective state is determined to be changed matches a pre-dialogue affective state in the conversation result database 218, and where the target affective state after the change matches a post-dialogue affective state in the conversation result database 218, on the basis of a context where the dialogue is taking place. In reference to the conversation result database 218 illustrated in
After trying to change the affective state of “Mr. A”, the dialogue-type robot 20 again refers to the affective conversion table 219 to determine whether there is a user 30 whose affective state is to be changed next among the other users 30. If there is such a user 30, the dialogue-type robot 20 performs processing that is the same as or similar to the above-described processing for “Mr. A”.
In the example illustrated in
In this case, the affective estimator 213 estimates the overall affective state or the average affective state of the users 30 who are there, and the affective change determiner 215 determines whether to change the overall affective state, and, if it is determined to change the overall affective state, to what affective state the overall affective state is to be changed. The topic extractor 216 extracts, from the conversation result database 218, a topic where the overall affective state of the users 30 matches a pre-dialogue affective state in the conversation result database 218, and where the target affective state after changing the overall affective state of the users 30 matches a post-dialogue affective state in the conversation result database 218, and the dialogue controller 212 has a dialogue with the multiple users 30 on the extracted topic to change the overall atmosphere. For example, as illustrated in part (C) of
Although the case where the dialogue-type robot 20 includes the personal information database 217, the conversation result database 218, and the affective conversion table 219 has been described as above, the exemplary embodiment of the present invention is not limited to this case, and these components may be arranged in a server connected through a communication line to the dialogue-type robot 20. The biometrics sensor 207 may be located not only in the dialogue-type robot 20, but also in other places, such as in an office. In this case, a motion sensor located on the ceiling or wall of the office may be adopted as the biometrics sensor 207.
Although the appearance of the dialogue-type robot 20 is illustrated in a shape that imitates a person in the exemplary embodiment, the appearance need not be in the shape of a person as long as the dialogue-type robot 20 is a device that is capable of having a dialogue with the user 30.
Although an example where the topic extractor 216 extracts, from the conversation result database 218, a topic where the current affective state of the user 30, obtained by the affective estimator 213, matches a pre-dialogue affective state in the conversation result database 218, and where the target affective state, determined by the affective change determiner 215, matches a post-dialogue affective state in the conversation result database 218 has been described in the above-described embodiment, the exemplary embodiment of the present invention is not limited to the above-described example in which a topic where the affective states “match” is extracted, and a topic where the affective states are “similar” may be extracted.
For example, the topic extractor 216 may extract, from the conversation result database 218, a topic where the current affective state of the user 30 matches a pre-dialogue affective state in the conversation result database 218, and where the target affective state is similar to a post-dialogue affective state in the conversation result database 218. Alternatively, the topic extractor 216 may extract, from the conversation result database 218, a topic where the current affective state of the user 30 is similar to a pre-dialogue affective state in the conversation result database 218, and where the target affective state matches a post-dialogue affective state in the conversation result database 218. Alternatively, the topic extractor 216 may extract, from the conversation result database 218, a topic where the current affective state of the user 30 is similar to a pre-dialogue affective state in the conversation result database 218, and where the target affective state is similar to a post-dialogue affective state in the conversation result database 218.
In the above-described exemplary embodiment, the case has been described in which the topic extractor 216 extracts a topic where the current affective state of the user 30 matches or is similar to a pre-dialogue affective state in the conversation result database 218, and where the target affective state matches or is similar to a post-dialogue affective state in the conversation result database 218. However, the exemplary embodiment of the present invention is not limited to this case, and, for example, a topic where a change from the current affective state to the target affective state of the user 30 matches or is similar to a change from a pre-dialogue affective state to a post-dialogue affective state in the conversation result database 218 may be extracted from the conversation result database 218.
If the current affective state of the user 30 is “nervous” and “afraid” and the target affective state is “satisfied” and “peaceful”, a change in the affective state that the user 30 is requested to have is expressed by a vector 1000A in
A change in the affective state from the current affective state to the target affective state (vector 1000A) matches a change in the affective state before and after a dialogue on the topics “children” and “school” (vector 1000B), stored in the conversation result database 218, in the direction and length though differs in the start point and the end point. Thus, the topic extractor 216 extracts the topics “children” and “school” in order to change the mood of the user 30. Not only in the case where a vector that expresses a change from the current affective state to the target affective state matches a vector that expresses a change in the affective state before and after a dialogue on a certain topic, stored in the conversation result database 218, but also in the case where the direction and length are within predetermined thresholds or in the case where the deviations of the direction, length, and barycenter are within predetermined thresholds, the topic extractor 216 may regard that the vectors (such as 1000A and 1000B) are similar, and may extract a topic that produces an affective change expressed by one of the vectors (1000B).
The foregoing description of the exemplary embodiment of the present invention has been provided for the purposes of illustration and description. It is not intended to be exhaustive or to limit the invention to the precise forms disclosed. Obviously, many modifications and variations will be apparent to practitioners skilled in the art. The embodiment was chosen and described in order to best explain the principles of the invention and its practical applications, thereby enabling others skilled in the art to understand the invention for various embodiments and with the various modifications as are suited to the particular use contemplated. It is intended that the scope of the invention be defined by the following claims and their equivalents.
Number | Date | Country | Kind |
---|---|---|---|
2016-180318 | Sep 2016 | JP | national |