Patent Grant
7076334
This invention relates to a robot apparatus, an action controlling method for controlling the action of the robot apparatus and an action controlling system for controlling the action of plural robot apparatus. More specifically, the present invention relates to a robot apparatus convenient for performing an autonomous action, an action controlling method and an action controlling system for such robot apparatus
Recently, a robot apparatus, formed after the shape of an animal, such as a dog or a cat, has been presented to the market. Among these robot apparatus, there is such an apparatus which acts autonomously responsive to the information from outside or to an internal state of the robot apparatus. For example, the robot apparatus is designed to act under a command from a user, as the information from outside, or depending on the level of the emotion, indicating its internal state.
The following are the functions of the robot apparatus operating responsive to the level of the emotion indicating the internal state of the robot apparatus.
The robot apparatus has variable states A to D, as shown for example in
For example, when the robot apparatus is ‘angry’ when in the state A, it correspondingly utters, e.g., ‘pi-ro-li-ro-li’ and, when the robot apparatus is ‘happy’ when in the state B, it correspondingly utters ‘pi-po-pa’.
Moreover, by the functions of the robot apparatus, operating under a command from a user, the interaction between the robot apparatus and the user takes place. These functions may be enumerated by, for example, those reacting to contact by the user from outside, and those in action under a command from outside.
For example, by providing the robot apparatus with a touch sensor detecting the contact thereof with outside, and by causing the action of the robot apparatus responsive to the contact thereof with the touch sensor, there is realized the function of the robot apparatus reacting to contact from outside, such as by the user. The touch sensor may, for example, be mounted on, e.g., a head of the robot apparatus.
As for the function of the robot apparatus, operating under an external command, the following technique may be used.
For example, an equipment outputting a scale command, as an external command, may be exploited. The scale command forms a command for causing the robot apparatus to express a preset action by the scale sound. The robot apparatus recognizes the scale command, output by the external control equipment, and performs the action corresponding to the scale command. The external communication equipment, outputting this scale command, may be enumerated by, for example, a sound commander. Meanwhile, the technique of the control system of the apparatus, exploiting the scale command, has recently come to be established.
By such contact as the information from outside, or the scale command, the robot apparatus in the standby state detects ‘being patted’ by soft touch at a touch sensor at the scalp or ‘being hit’ by strong touch at the touch sensor as shown in
By the stimuli from outside by contact, or by the sound scale command, the action of the robot apparatus is affected, such as to enable the interaction between the user and the robot apparatus.
Meanwhile, the actions of the conventional robot apparatus are merely determined by its own emotion or by the information afforded by the user. If plural robot apparatus can interact with one another by way of communication, and if the plural robot apparatus can react to one another and operate accordingly, the entertainment character of the robot apparatus will be improved further.
Moreover, if it is desirable for the plural robot apparatus to have a dialog with one another, it may be an occurrence that the plural robot apparatus are not of the same machine type. For example, one of the robot apparatus may be apparatus of an original model, while another may be the product of a generic model, or the plural robot apparatus may be furnished by different manufacturers. In such case, it is difficult for the robot apparatus to react to one another.
The robot apparatus of the generic model may be afforded with the functions of the robot apparatus of the original model. In other words, there are occasions where the robot apparatus of the original model are not afforded with the function owned by the robot apparatus of the generic model. Thus, there are occasions where the interaction function is limited to the robot apparatus of the same machine type.
However, the entertainment character of the robot apparatus can be enhanced by enabling the interaction between plural robot apparatus having different interaction functions. Additionally, it may be said that such interaction among various robot apparatus lends itself to effective utilization of the robot apparatus as resources.
In view of the above-described status of the art, it is an object of the present invention to provide a robot apparatus that can have a dialog with other robot apparatus, a method for controlling the action of the robot apparatus and a system for controlling the robot apparatus.
For accomplishing the above object, the robot apparatus according to the present invention includes an action part, action detection means for detecting the information contained in an action output by another robot apparatus, action part control means for controlling the action part, based on the information detected by the action detection means, and means for measuring the number of times of reactions of the other robot apparatus corresponding to the actions output by the action part. The action part control means outputs an action by the action part depending on the number of times of reactions of the other robot apparatus.
The robot apparatus, constructed as described above, detects the information contained in the action output by other robot apparatus, by action detection means, and controls the action part by action part controlling means to output an action. At this time, the action part controlling means controls the action part responsive to the number of times of reactions of the other robot apparatus as measured by the means for measuring the number of times of reactions. This permits the robot apparatus to act responsive to the action of the other robot apparatus.
For accomplishing the above object, the robot apparatus according to the present invention also includes action detection means for detecting the information contained in an action output by another robot apparatus and action outputting means for outputting an action corresponding to the information as detected by the action detection means.
The robot apparatus, constructed as described above, detects the information contained in the action output by the other robot apparatus to output the action corresponding to the information detected by the action detection means to output the action corresponding to the information as detected by the action detection means by the action outputting means. This permits the robot apparatus to act responsive to the action of the other robot apparatus.
For accomplishing the above object, an action controlling method for the robot apparatus including an action part according to the present invention includes an action detection step of detecting the information contained in an action output by another robot apparatus, an action part controlling step of controlling the action part, based on the information detected by the action detections step, and a step of measuring the number of times of reactions of the other robot apparatus based on the actions output by the action part. The action part controlling step outputs an action by the action part depending on the number of times of reactions of the other robot apparatus. This permits the robot apparatus to act responsive to the action of the other robot apparatus.
For accomplishing the above object, the action controlling method for the robot apparatus according to the present invention includes an action detection step of detecting the information contained in an action output by another robot apparatus and an action outputting step of causing one of the robot apparatus to output an action corresponding to the information as detected by the action detection step.
For accomplishing the above object, an action controlling system for the robot apparatus according to the present invention includes a plurality of robot apparatus including action detection means for detecting the information contained in an action output by the counterpart robot apparatus, and action outputting means for outputting an action corresponding to the information detected by the action detection means.
With this action controlling system for the robot apparatus, the robot apparatus detects the information contained in the action output by the counterpart robot apparatus by action detection means to output the action corresponding to the information as detected by the action detection means by the action outputting means. This action controlling system for the robot apparatus permits the robot apparatus to act responsive to the action of the robot apparatus.
For accomplishing the above object, an action controlling method for the robot apparatus according to the present invention includes an action outputting step of one of the robot apparatus outputting a preset action when one of the robot apparatus is at a preset state, and a reactive action outputting step of the other robot apparatus outputting an action corresponding to the preset action output by the one robot apparatus. This action controlling method for the robot apparatus permits the robot apparatus to act responsive to the action of the robot apparatus.
Other objects, features and advantages of the present invention will become more apparent from reading the embodiments of the present invention as shown in the drawings.
Referring to the drawings, a preferred embodiment of the present invention will be explained in detail. The present embodiment is directed to an autonomous robot apparatus which acts autonomously responsive to the external information, ambient environment (or extraneous factors) or to internal states (or internal factors).
In the present embodiment, the structure of the robot apparatus is first explained and subsequently applying part of the present invention in the robot apparatus is explained in detail.
As shown in
Referring to
On the head unit 4, there are mounted, in position, a CCD (charge coupled device) camera 20 for imaging an outside state, a touch sensor 21, for detecting the pressure resulting from a physical action, such as ‘stroking’ or ‘patting’ from the user, a distance sensor 22 for measuring the distance to an object positioned ahead, a microphone 23 for collecting the external sound, a loudspeaker 24 for outputting the sound, like whining, and LEDs (light emitting diodes)equivalent to the ‘eyes’ of the robot apparatus 1.
The joint portions of the leg units 3A to 3D, connecting portions of the leg units 3A to 3D and the trunk unit 2, connecting portions of the head unit 4 and the trunk unit 2 and the connecting portion of a tail 5A of the tail unit 5 are provided with a number of actuators 251 to 25n and potentiometers 261 to 26n corresponding to the number of the degrees of freedom. For example, the actuators 251 to 25n include servo motors. The leg units 3A to 3D are controlled by the driving of the servo motors to transfer to a targeted posture or movement.
The sensors, such as the angular velocity sensor 18, acceleration sensor 19, touch sensor 21, floor contact sensors 23R/L, posture sensor 24, distance sensor 25, microphone 26, distance sensor 22, microphone 23, loudspeaker 24 and the potentiometers 251 to 25n are connected via associated hubs 271 to 27n to the signal processing circuit 14 of the controller 16, while the CCD camera 20 and the battery 17 are connected directly to the signal processing circuit 14.
The signal processing circuit 14 sequentially captures sensor data, picture data or speech data, furnished from the above-mentioned respective sensors, to cause the data to be sequentially stored over internal bus 15 in preset locations in the DRAM 11. In addition, the signal processing circuit 14 sequentially captures residual battery capacity data indicating the residual battery capacity supplied from the battery 17 to store the data thus captured in preset locations in the DRAM 11.
The respective sensor data, picture data, speech data and the residual battery capacity data, thus stored in the DRAM 11, are subsequently utilized when the CPU 10 performs actional control of the robot apparatus 1.
In actuality, in an initial stage of power up of the robot apparatus 1, the CPU 10 reads out a memory card 28 loaded in a PC card slot, not shown, of the trunk unit 2, or a control program stored in the flash ROM 12, either directly or through a PC card interface circuit 13, for storage in the DRAM 11.
The CPU 10 then checks its own status and surrounding statuses, and the possible presence of commands or actions from the user, based on the sensor data, picture data, speech data or residual battery capacity data, sequentially stored from the signal processing circuit 14 to the DRAM 11.
The CPU 10 also determines the next ensuing actions, based on the verified results and on the control program stored in the DRAM 11, while driving the actuators 251 to 25n, as necessary, based on the so determined results, to produce behaviors, such as swinging the head unit 4 in the up-and-down direction or in the left-and-right direction, or moving the leg units 3A to 3D for walking or jumping.
The CPU 10 generates speech data as necessary and sends the so generated data through the signal processing circuit 14 as speech signals to the loudspeaker 24 to output the speech derived from the speech signals to outside or turns on/off or flicker the LEDs.
In this manner, the present robot apparatus 1 is able to behave autonomously responsive to its own status and surrounding statuses, or to commands or actions from the user.
A robotics server object 32 is made up by a virtual robot 33, a power manager 34, comprised of a set of software items responsible for switching between power sources, a device driver manager 35, comprised of a set of software items, supervising various other device drivers, and a designed robot 36, comprised of a set of software items supervising the mechanism of the robot apparatus 1. The virtual robot 33, located in the lowermost layer of the device driver layer 30, is comprised of a set of software items furnishing an interface for accessing the hardware items, including the above-mentioned various sensors and actuators 251 to 25n.
A manager object 37 is made up by an object manager 38 and a service manager 39. The object manager 38 is a set of software items supervising the booting and the end of respective software items included in the robotics server object 32, a middle ware layer 40 and an application layer 41, while the service manager 39 is a set of software items supervising the connection to respective objects based on the information on the connection among respective objects stated in a connection file stored in a memory card 28 (
The middle ware layer 40 is positioned as an upper layer of the robotics server object 32, and is made up by a set of software items providing basic functions of the robot apparatus 1, such as picture processing or speech processing. The application layer 41 is located as an upper layer of the middle ware layer 40, and is a set of software items for deciding on the behavior of the robot apparatus 1 based on the results of the processing by the software items making up the middle ware layer 40.
Referring to
The signal processing modules 50 to 58 of the recognition system 60 captures relevant data from the sensor data, picture data and the speech data, read out by the virtual robot 33 of the robotics server object 32 from the DRAM 11 (
The input semantics converter module 59 recognizes the own status, the surrounding status, user's commands or actions, such as ‘annoying’, ‘sultry’, ‘light’, ‘a ball has been detected’, ‘falldown is detected’, ‘stroked’, ‘patted’, ‘do-mi-so scale has been heard’, ‘a moving object has been detected’, or ‘an obstacle has been detected’ to output the results of recognition to the application layer 41 (
The application layer 41 is made up by five modules, namely a behavioral model library 70, a behavioral switching module 71, a learning module 72, a emotional model 73 and an instinct model 74, as shown in
In the behavioral model library 70 there are provided respective independent behavioral models 701 to 70n in association with plural pre-selected condition items, such as ‘residual battery capacity is small’, ‘restoration from the falldown state’, ‘an obstacle is to be evaded’, ‘the emotion is to be expressed’ or ‘a ball has been detected’, as shown in
When the results of recognition are provided from the input semantics converter module 59 or a preset time has elapsed as from the time the last results of recognition were provided, the behavioral models 701 to 70n decide on the next behaviors, as they refer to parameter values of the emotion as held by the emotional model 73 and to parameter values of the corresponding desires as held by the instinct model 74, to send the results of decision to the behavioral switching module 71.
In the present embodiment, the behavioral models 701 to 70n use an algorithm, termed finite probability automaton, as a technique of deciding on the next behavior. This algorithm probabilistically determines from which one of the nodes (states) NODE0 to NODEn to which one of these nodes NODE0 to NODEn transition is to be made, based on the values of the transition probability P1 to Pn as set for the arcs ARC1 to ARCn interconnecting the respective nodes NODE0 to NODEn.
Specifically, each of the behavioral models 701 to 70n includes a status transition table 80, shown in
In the status transition table 80, input events (results of recognition), as the conditions for transition in the nodes NODE0 to NODEn, are listed in the column of the ‘input event name’, in the priority order, and further conditions for the transition condition are stated in associated rows of the columns ‘data name’ and ‘data range’.
Thus, in the node NODE100, shown in the status transition table 80 of
Also, in the present node NODE100, if no results of recognition are input, but any one of the parameter values ‘joy’ (JOY), surprise (SURPRISE) or ‘sadness’ (SADNESS) among the parameter values of the respective emotion and desires, as held in the emotional model 73, among the parameter values periodically referenced by the behavioral models 701 to 70n, is in a range between ‘50 and 100’, transition may be made to another node.
Moreover, in the status transition table 80, the node names to which transition can be made from the node NODE0 to NODEn are shown in the row ‘nodes of destination of transition’ in the column ‘probability of transition to other nodes’. Additionally, the probability of the transition to other nodes NODE0 to NODEn, enabled when all conditions stated in the columns ‘input event name’, ‘data name’ and ‘data range’ are met, is entered in corresponding locations in the column ‘probability of transition to other nodes’. The behaviors to be output on the occasion of transition to the nodes NODE0 to NODEn are indicated in the row ‘output behavior’ in the column ‘probability of transition to other nodes’. Meanwhile, the sum of the probability values of each row in the column ‘probability of transition to other nodes’ is 100%.
Thus, in the node NODE100 represented by the status transition table 80 of
In each of the behavioral models 701 to 70n, a plural number of the sets of the nodes NODE0 to NODEn, each stated as this status transition table 80, are concatenated together, such that, given the results of recognition from the input semantics converter module 59, the next behavior is probabilistically determined by exploiting the status transition tables of the NODE0 to NODEn and the results of the decision are output to the behavioral switching module 71.
The behavioral switching module 71, shown in
The behavioral switching module 71 notifies the learning module 72, emotional model 73 and the instinct model 74 of the effect of the termination of the behavior, based on the behavior completion information afforded from the output semantics converter module 68 after the end of the behavior.
The learning module 72 is fed with the results of recognition of the instructions received as an action from a user, such as ‘patting’ or ‘stroking’, from among the results of recognition provided from the input semantics converter module 59.
The learning module 72 changes the probability of transition of the behavioral models 701 to 70n in the behavioral model library 70, based on the results of recognition and on the notification from the behavioral switching module 71, such that, when the action is ‘patting’ (‘scolding’) or ‘stroking’ (‘praising’), the probability of occurrence of the behavior in question will be increased or decreased, respectively.
On the other hand, the emotional model 73 is holding parameters representing the intensity of each of the six emotion types, namely joy (JOY), sadness (SADNESS), anger (ANGER), surprise (SURPRISE), disgust (DISGUST) and fear (FEAR). The emotional model 73 periodically updates the parameter values of these emotion types, based on the particular results of recognition provided by the input semantics converter module 59, such as ‘patted’ or ‘stroked’, time elapsed and on the notification from the behavioral switching module 71.
Specifically, the emotional model 73 calculates a parameter value E[t+1] of the current emotion type for the next period in accordance with the following equation (1):
E[t+1]=E[t]+ke×ΔE[t] (1)
where ΔE[t] in the amount of variation of the emotion type as calculated by a preset equation based on, for example, the results of recognition provided by the input semantics converter module 59, the behavior of the robot apparatus 1 at the pertinent time or on the time elapsed as from the previous updating event, E[t] is the current parameter value of the emotional type and ke is a coefficient representing the sensitivity of the emotion type. The emotional model 73 substitutes the so calculated value for the current parameter value E[t] of the emotion type to update the parameter value of the emotion type. In similar manner, the emotional model 73 updates the parameter values of the totality of the emotion types.
Which effect the respective results of recognition and the notification from the output semantics converter module 68 will have on the variation of the parameter values of the respective emotion types ΔE[t] is predetermined, such that the results of recognition ‘patted’ significantly affects the amount of variation ΔE[t] of the parameter value of the emotion type ‘anger’, while the results of recognition ‘patted’ significantly affects the amount of variation ΔE[t] of the parameter value of the emotion type ‘joy’.
The notification from the output semantics converter module 68 is the so-called behavior feedback information (behavior end information) and the information concerning the results of occurrence of the behavior. The emotional model 73 also changes the emotion based on this information. For example, the level of the emotion of anger may be lowered by the act of ‘barking’. Meanwhile, the notification from the output semantics converter module 68 is also input to the learning module 72, which then changes the corresponding transition probability of the behavioral models 701 to 70n based on this notification.
Meanwhile, the feedback of the results of the behavior may be made by an output of the behavioral switching module 71 (behavior seasoned with the emotion).
On the other hand, the instinct model 74 holds the parameters, representing the intensity of four reciprocally independent desires, namely ‘desire for exercise’ ‘desire for affection’, ‘appetite’ and ‘curiosity’. The instinct model 74 periodically updates the parameter values of these desires, based on the results of recognition provided from the input semantics converter module 59, time elapsed and on the notification from the behavioral switching module 71.
Specifically, as concerns the ‘desire for exercise’, ‘desire for affection’ and ‘curiosity’, the instinct model 74 calculates, at a preset period, the parameter value I [k+1] of these desires at the next period, using the following equation (2):
I[k+1]=I[k]+ki×ΔI[k] (2)
where ΔI[k] is the amount of variation of the desire in question at a pertinent time as calculated by a preset equation based on the results of recognition, time elapsed and the notification of the output semantics converter module 68, I[k] is the current parameter value of the desire and ki is the coefficient representing the sensitivity of the desire in question, and substitutes the calculated results for the current parameter value I[k] to update the parameter value of the desire. The instinct model 74 updates the parameter values of the respective desires except the ‘appetite’.
The effect of the results of recognition and the notification from the output semantics converter module 68 on the amount of variation ΔI[k] of the parameter values of the respective desires is predetermined, such that, for example, the notification from the output semantics converter module 68 significantly affects the amount of variation ΔI[k] of the parameter values of ‘fatigue’.
In the present embodiment, the parameters of the respective emotion types and the respective desires (instincts) are varied in a range from 0 to 100, while the values of the coefficients ke and ki are also set individually for the respective emotion types and for respective desires.
The output semantics converter module 68 of the middle ware layer 40 sends abstract behavioral commands, such as ‘go ahead’, ‘joy’, ‘cry’, or ‘tracking (track a ball)’, provided by the behavioral switching module 71 of the application layer 41, as described above, to the signal processing modules 61 to 67 of the output system 69, as shown in
Given a command for a behavior, the signal processing modules 61 to 67 generates servo command values to be supplied to the associated actuators 251 to 25n (
Thus, the robot apparatus 1 is able to perform an autonomous behavior, based on the control program, responsive to its own internal state, surrounding state (exterior state) or to the command or action from the user.
In the following explanation, the structure of the robot apparatus is first explained, and the portion thereof pertinent to the present invention is then explained.
By the application of the present invention, the robot apparatus, having the above structure, is able to operate responsive to the action of another robot apparatus. This enables the plural robot apparatus to have dialog with one another. Moreover, the robot apparatus may have a dialog with another robot apparatus having a different dialog function. First, the robot apparatus of the present invention, according to the present invention, capable of having a dialog with the robot apparatus of the different dialog function, is first explained, and the robot apparatus according to the present invention, capable of having a dialog based on the same dialog function, is subsequently explained.
(3-1) Robot Apparatus Having a Dialog with a Robot Apparatus Having a Different Dialog Function
The robot apparatus according to the present invention is able to have a dialog with another robot apparatus having a different dialog function.
The robot apparatus having a different dialog function may be exemplified by a robot apparatus of a different machine type, a robot apparatus furnished from a different manufacturer and a robot apparatus produced by the same manufacturer but different in type number or model. In the present embodiment, explanation is made under an assumption that a dialog is to be had with another robot apparatus which has its dialog function controlled by the sound scale command. In this case, the other robot apparatus, as a counterpart of the dialog, utters, e.g., ‘pi-ro-lin’, in its standby state, as a preset action, on recognition of the scale command composed of scale sounds.
By the robot apparatus of the present embodiment of the present invention having such sound scale command outputting function, and reacting, such as by a synchronous action, to the action of another robot apparatus, having its action controlled by the sound scale command, the user is in a position of appreciating the actions, produced as a result of reactions between the robot apparatus, as the dialog being had by the robot apparatus.
The robot apparatus of the present embodiment realizes the dialog with the other robot apparatus in a manner which will be clarified in the following explanation. In this explanation, the robot apparatus of the present embodiment is termed a main robot apparatus and the robot apparatus having the dialog function different from that of the main robot apparatus is termed a slave robot apparatus.
The main robot apparatus includes action detection means for detecting the information contained in the actions output by the slave robot apparatus as the other robot apparatus, and action outputting means for outputting an action corresponding to the information as detected by the action detection means. Specifically, the main robot apparatus includes a CPU 10, as described above. Specifically, the CPU 10 implements the above means as an action detecting function of detecting the information contained in the action output by the slave robot apparatus and an action outputting function of outputting the action corresponding to the information as detected by the action detecting function. That is, the CPU 10 implements the above functions by a variety of signal processing modules.
The main robot apparatus also includes communication means for exchanging the information with the slave robot apparatus. The information here is that by the scale sound. Thus, the communication means realizes the information transmission/reception by a loudspeaker 24. The information is also the meaning information indicating the meaning of an action. For example, the information may be the information for the meaning of an action itself or the emotion contained in the action.
The main robot apparatus, having the above-described structure, realizes the dialog function with the slave robot apparatus by a sequence of processing actions such as a so-called dialog mode. Specifically, the main robot apparatus realizes the dialog function by one of the behavior models shown in
The main robot apparatus, having such function, recognizes the ‘emotion expressing sound’, such as ‘pi-ro-lin’, which is the sound signal issued from the slave robot apparatus in an idling state ST1, as shown in
The idling state ST1 is such a state in which the main robot apparatus is acting autonomously, that is a state on which the robot apparatus is acting autonomously depending on the external environment or responsive to its own emotional states.
On recognition of the emotion expressing sound, uttered by the slave robot apparatus, the main robot apparatus causes the slave robot apparatus to transfer to a ‘sit’ posture. It should be noted that the main robot apparatus does not necessarily have to cause the slave robot apparatus to transfer to the ‘sit’ posture but may cause the slave robot apparatus to transfer to other postures or to continue its autonomous behavior.
The main robot apparatus then causes the transfer to the command producing state ST2 to produce a sound scale command comprised of scale sounds. For example, the main robot apparatus generates the scale command as random.
Specifically, the main robot apparatus includes a database comprised of various sound scale commands. It should be noted that the various sound scale commands are associated with respective different action contents to be executed by the slave robot apparatus, with the scale sounds corresponding to different combinations of the sound names. The sound scale commands, forming the database owned by the main robot apparatus, are the sound scale commands owned by a so-called sound commander in order to control, e.g., the slave robot apparatus.
For example, the main robot apparatus includes a database made up by patterns of scale sounds of the sound scale command and by the command contents, as shown in
The sound scale commands, owned in this manner by the main robot apparatus, may be registered during manufacture, but may also be subsequently registered anew or updated. The main robot apparatus selects the sound scale commands, owned as, e.g., a database, at random, and outputs the so selected sound scale command.
The main robot apparatus may also output the sound scale commands in association with actions. Usually, the sound scale commands are retained to be signal patterns for controlling the robot apparatus, and are handled as being meaningful for the robot apparatus. However, these commands are not meaningful, insofar as the human being is concerned. Thus, by causing the main robot apparatus to output the actions associated with the sound scale commands, the user is able to comprehend the meaning of the sound scale commands.
On outputting the sound scale command, the main robot apparatus enters into a state ST3 of waiting for reactions of a slave robot apparatus. The main robot apparatus maintains its reaction awaiting state ST3 for a preset time.
On recognition of the sound scale command, issued by the main robot apparatus, the slave robot apparatus outputs a emotion expressing sound, depending on the sound scale command. However, it is indefinite whether or not the slave robot apparatus necessarily responds to the sound scale command. The reason is that the slave robot apparatus may have gone away to some distant place by its own autonomous behavior or cannot detect the sound scale command due to, for example, ambient noise. In order to cope with this, the main robot apparatus provides for a variety of actions, responsive to reactions of the slave robot apparatus.
When the slave robot apparatus has output a emotion expressing sound, responsive to a sound scale command issued by the main robot apparatus, the latter recognizes this emotion expressing sound and outputs the same emotion expressing sound. For example, the main robot apparatus reads out and outputs the emotion expressing sound of the slave robot apparatus it holds as a database from the outset.
The user is able to appreciate the action performed between the main robot apparatus and the slave robot apparatus as the dialog between these robot apparatus.
On recognition of the emotion expressing sound (sound scale reaction sound), the main robot apparatus increments the variable NumResponse by one, while setting a variable NumTimup to zero. It should be noted that the variable NumResponse is a variable indicating the number of times of the reaction of the slave robot apparatus. This variable NumResponse is afforded with zero as an initial value when entering into the processing or mode of the dialog function. On the other hand, the variable NumTimup is a value indicating the number of times of failure of the slave robot apparatus to react to the sound scale command issued by the main robot apparatus. This value is referred to below as the number of times of non-reaction. For example, the failure of the slave robot apparatus to react to the command is decided on by the absence of the reaction from the slave robot apparatus for a preset time duration. This variable NumTimup is also afforded with zero as an initial value when entering into the processing or mode of the dialog function.
The emotion recognition by the slave robot apparatus is by providing a table showing the sound of the sound scale reaction and the corresponding emotion and by referring the detected sound scale reaction sound to the table. It is also possible not to provide such tables, in which case the emotion is to be recognized based on analyses of the pitch, power etc. of the detected sound of the sound scale reaction.
When the slave robot apparatus 1 has made reactions, the main robot apparatus performs preset actions as the next following reaction action.
The main robot apparatus performs the actions depending on the value of the above-mentioned NumResponse. The main robot apparatus uses this variable NumResponse as a variable for deciding on an action. Specifically, the main robot apparatus has various actions as action (1), action (2), action (3), . . . , and decides on one of them by the action (NumResponse) as an action on the slave robot apparatus.
The variable NumResponse indicates the number of times of reaction of the slave robot apparatus as recognized by the main robot apparatus. Thus, the larger this number, the larger is the number of times of reaction of the slave robot apparatus. By and large, a reaction to a person's action expressing his or her happy state, if any, gives him or her the happy emotion, as in case of chat or conversation. In this consideration, the actions of the action (1), action (2), action (3), . . . are defined so that the action indicating ‘happiness’ will be increasingly garish in this order.
In the state of reaction action ST4, the main robot apparatus selects the action (NumResponse), from the action as defined above, depending on the current variable NumResponse, to perform the action. That is, if the variable NumResponse is 1 to 5, the main robot apparatus causes the corresponding action (NumResponse) to be performed to transfer to the command issuing state ST2 again to select the sound scale command at random to output the so selected sound scale command. In the reaction awaiting state ST3, the main robot apparatus waits for the reaction from the slave robot apparatus. If the slave robot apparatus makes a reaction, the main robot apparatus causes a preset action to be performed again, responsive to the reactive action of the slave robot apparatus, to increment the variable NumResponse by 1, as an example. The main robot apparatus then performs an action indicating happiness which has become more garish responsive to the variable NumResponse incremented by 1.
When the variable NumResponse exceeds 5, the main robot apparatus performs the action of ‘banzai’ (‘hurrah!’), as an action (6), to then revert to the idling state ST1. That is, the main robot apparatus completes the processing or mode for having dialog with the slave robot apparatus to then revert to a mode for autonomous behavior.
Thus, the main robot apparatus is responsive to the reaction from the slave robot apparatus to perform variable actions depending on the number of times of the reactions. The user is able to appreciate the actions exchanged between the main robot apparatus and the slave robot apparatus as the dialog between the robot apparatus.
It should be noted that the action of the main robot apparatus, indicating its ‘happiness’ is rendered increasingly garish with an increasing number of times of reactions from the slayer robot apparatus. This may be grasped as what may be called the action modulation.
If the reaction from the slave robot apparatus cannot be recognized within a preset time duration in the reaction awaiting state ST3, the main robot apparatus performs an action indicating ‘sadness’. The robot apparatus also increments the variable NumTimUp, indicating the number of times of non-reaction, by 1. The case where there is no reaction from the slave robot apparatus may be such a case where the slave robot apparatus has gone away to some distant place by its own autonomous behavior or may also be such a case where the slave robot apparatus cannot detect the sound scale command due to, for example, ambient noise.
Meanwhile, even when the slave robot apparatus fails to make reactions, it is expressing the action indicating ‘sadness’. It may be said that, in such case, the main robot apparatus recognizes the failure of the slave robot apparatus by a reaction action.
Lacking the reactions from the slave robot apparatus, the main robot apparatus performs the aforementioned action and again selects a sound scale command at random at the command issuing state ST2 to output the so selected sound scale command to wait again for the reaction from the slave robot apparatus in the reaction awaiting state St3. If, in this reaction awaiting state St3, the main robot apparatus has been unable to recognize the reactions from the slave robot apparatus for a preset number of times on end, that is if the number of times of non-reaction has reached a preset value, it performs an action of increased sadness. In the present instance, when the reaction from the slave robot apparatus cannot be recognized four times on end, that is if the variable NumTimUp, indicating the number of times of non-reaction, exceeds 3, the main robot apparatus performs the action indicating ‘increased sadness’ to then revert to the idling state ST1.
Thus, lacking the reaction from the slave robot apparatus, the main robot apparatus performs variable actions, depending on the number of times of non-reactions, whereby the user is able to appreciate the actions exchanged between the main and slave robot apparatus as the interaction between the robot apparatus.
Meanwhile, in the present embodiment, the case in which the action of ‘increased sadness’ is to be performed from the aforementioned reaction awaiting state ST3 is such a case where, in the reaction awaiting state ST3 to which the main robot apparatus transferred a plural number of times, the reaction from the slave robot apparatus has not been detected on end. It is because the variable NumTimUp indicating the number of times of non-reaction is initialized to zero when the slave robot apparatus has made response in the reaction awaiting state ST3. If, lacking a reaction from the slave robot apparatus to an action from the main robot apparatus, the slave robot apparatus has reacted to subsequent actions from the main robot apparatus, the main robot apparatus performs the action ‘happiness’ and initializes the variable NumTimUp, indicating the number of times of non-reaction, to zero.
The above-described processing is, of course, not limitative. For example, the number of times of non-reactions can be continuously counted without initializing the variable NumTimUp to zero responsive to reactions from the slave robot apparatus, that is, without dependency on whether there has been or has not been made a reaction.
Thus, the main robot apparatus is able to perform various actions, depending on the reacting actions of the slave robot apparatus to the actions from the main robot apparatus, inclusive of the non-reacting actions, and on the number of times of non-reactions. This renders it possible for the user to appreciate the actions performed as a result of reactions between the robot apparatus, as the dialog being made between the robot apparatus.
The main robot apparatus achieves this dialog by controlling an original model or a robot apparatus produced by another manufacturer, the action of which is controlled by the external control command, such as a sound scale command, in synchronism with its own actions. As a result, the user is able to appreciate the reacting actions between different models of the robot apparatus of the same manufacturer or between robot apparatus of different manufacturers as the dialog between the robot apparatus.
(3-2) Robot Apparatus Having a Dialog with the Same Dialog Function.
The dialog which is made on the basis of the dialog function between two robot apparatus, according to the present invention, is hereinafter explained. Each apparatus, explained here, has the same dialog function, which enables the two robot apparatus to have the dialog with each other.
Each robot apparatus includes action detection means for detecting the information contained in the action output by the counterpart robot apparatus, and action outputting means for outputting the actions corresponding to the information detected by the action detection means. Specifically, the main robot apparatus realizes this by its CPU as an action detecting function of detecting the information contained in the action output by the counterpart robot apparatus, and as an action outputting function of outputting the action corresponding to the information as detected by the action detection function. That is, the CPU 10 achieves these functions by a variety of signal processing modules.
Both robot apparatus are configured for outputting a preset action when in a preset state. Thus, when in a preset state, one of the robot apparatus outputs a preset action by the action outputting function, with the other robot apparatus outputting an action corresponding to the action output by the first-stated robot apparatus.
The two robot apparatus are of similar structure to each other to achieve the dialog with the counterpart robot apparatus. The dialog between the two robot apparatus is hereinafter explained with reference to
Both of the robot apparatus are in the idling states ST11, ST21, as shown in
When in the preset state, each robot apparatus outputs a preset action. Thus, depending on the timing of being in the preset state, the ‘other robot apparatus’ may be a master, with the remaining robot apparatus being a recipient. That is, each of the robot apparatus may become a ‘master’ or a ‘recipient’ depending on circumstances.
The robot apparatus which has been the first to be in the preset state to start or fire the dialog function to become a master utters ‘Yahhoo!’ and the ‘scale sound (1)’, as an action ST12. It should be noted that the ‘Yahhoo!’ is the enunciation as the language, while the ‘scale sound (1)’ is the scale sound, such as ‘pi-po-paa’, that is no other than a sound sequence. In the following explanation, scale sounds (2), (3) and (4) . . . , are given as examples of the scale sounds. Each of these is of a different sound scale pattern. The scale sounds (2), (3) and (4) . . . , represent techniques established in the prior art as the sound scale commands.
Meanwhile, the scale sounds are output in association with the enunciation such as ‘Yahhoo!’ in such a case where the recipient robot apparatus as a counterpart of the dialog cannot recognize the language ‘Yahhoo!’ as the enunciation action as being meaningful. Thus, the scale sounds are not necessary if the robot apparatus is able to recognize the action as being meaningful. That is, it is when the action cannot be understood by the counterpart apparatus as being meaningful, that is when the enunciation as the action is a long text, that the scale sound is output simultaneously with the action. In such case, the scale sound operates as effective dialog means between the robot apparatus.
The recipient recognizes such enunciation of the master to transfer from an idling state ST21 to a reactive action ST22. That is, the recipient boots its dialog function, with the master's enunciation as a trigger.
The recipient enunciates ‘Yahhoo!’ and the scale sound (2)’ as the reactive action ST22. The recipient recognizes the enunciation made by the master by the ‘scale sound (1)’ of the ‘Yahhoo!’ and the ‘scale sound (1)’ as enunciated by the master.
The scale sound is the technique already established as the sound scale command in the control robot apparatus, as described above. By the application of the scale sound recognition system, thus established, the robot apparatus 1 accurately recognizes the command through a scale sound pattern of the scale sounds.
The master recognizes the enunciation of the ‘scale sound (2)’ of the recipient and enunciates ‘happy’ and ‘the sound (3)’ as its reactive action ST13. The reactive actions to the action of the counterpart side are repeated as the contents of the reactive actions are changed.
That is, the recipient recognizes the enunciation of the ‘scale sound (3)’ of the master and utters ‘calm’ and ‘the scale sound (4)’ as the reactive action ST23. Meanwhile, ‘calm’ is the enunciation indicating the contents of, for example, ‘be calm’. The master recognizes the enunciation of the ‘scale sound (4)’ and utters ‘happy!’ and ‘the scale sound (5)’ as its reactive action ST14. Meanwhile, ‘happy!’ is no more than the emphasized expression of ‘happy’. The recipient recognizes the enunciation of the master of ‘the scale sound (5)’ to utter ‘happy’ and ‘the scale sound (6)’ as its reactive action ST24. The master then recognizes the enunciation of the ‘scale sound (6)’ of the recipient to utter ‘satisfied’ as the last action of the dialog action. In keeping therewith, the recipient similarly utters ‘satisfied’ in synchronism with the master's enunciation.
In this manner, the two robot apparatus, operating as a master and as a recipient, are able to react to the counterpart side's action, whereby the user is able to appreciate the action performed as a result of reaction between the robot apparatus as the dialog between the robot apparatus.
By exploiting the recognition system of the scale sound, established as the technique, the counterpart side's action can be recognized, as described above. This, of course, is not limitative, such that one of the robot apparatus is also able to directly recognize the meaningful language uttered by the robot apparatus, such as ‘Yahhoo!’, to take a reactive action, whereby the user is able to grasp the dialog by the robot apparatus as being more realistic dialog.
Lacking the reaction from the counterpart robot apparatus within a preset time, this state may be deemed to be a timeout state to revert again to idling.
In
Specifically, when the master utters ‘Yahhoo!’, the recipient utters ‘Yahhoo!’ as a reactive action. Meanwhile, if the enunciation is felt to be possibly understood with relative ease, the scale sound need not be output. That is, if the enunciation such as ‘Yahhoo!’ can be recognized readily, the master utters only ‘Yahhoo!’, without correspondingly outputting the scale sound, with the counterpart side robot apparatus then recognizing this ‘Yahhoo!’.
The master is responsive to the recipient's enunciation ‘Yahhoo!’ to utter ‘Happy’ and the scale sound (sound scale command)’, while the recipient responsive to the enunciation ‘Happy’ and ‘the scale sound’ of the master utters ‘calm’ and ‘the scale sound’. The master responsive to the recipient's enunciation ‘Happy’ and ‘the scale sound’ again recognizes the enunciation ‘the scale sound’ utters ‘Happy’ and ‘the scale sound’. The recipient is responsive to the master's enunciation ‘Happy!’ and the ‘scale sound’ to recognize the ‘scale sound’ to utter ‘Happy’ and ‘the scale sound’. The master responsive to the enunciation ‘Happy’ and ‘the scale sound’ of the recipient utters ‘comfort’ as the last action of the dialog, whilst the recipient utters ‘comfort’, in synchronism therewith, as the last action of the dialog.
As for exchange in case the master issues a command and the recipient operates responsive thereto, shown in
Although the recipient is able to apprize the master of the end of the action by the scale sound, the master may also perform time management to detect the end of recipient side's action. For example, the master may hold the time of execution of the rightward rotation of the recipient's body, measure the time elapsed as from the time of raising the right hand or uttering the scale sound and compare the measured time with the time of execution of the rightward rotation it holds to grasp the end of the rightward trunk rotation of the recipient actually proceeding.
After detection of the end of the recipient's rightward rotation, the master performs the action of raising the left hand (left fore limb), while uttering the ‘scale sound (sound scale command). The recipient is responsive to the master's action of raising the left hand (left fore limb), and to the ‘scale sound (sound scale command) to recognize the enunciation of the ‘scale sound’ to perform the action of rotating its trunk leftwards. The master then measures the time elapsed since raising the left hand or since the enunciation of the scale sound and compares it to the time of execution of the leftward rotation it holds to be apprized of the end of the leftward rotation the recipient is actually performing.
After detecting the end of the recipient's leftward rotation, the master performs the action of raising both hands (both fore limbs), while uttering ‘scale sound (sound scale command)’. The recipient is responsive to the master's action of raising both hands (both fore limbs) and the enunciation (sound scale command) to recognize the enunciation ‘sound scale sound’ to perform an action of moving forwards.
In this manner, the exchange proceeds in the case where the master issues a command and the recipient operates responsive thereto.
As explained in the foregoing, with reference to
In the above-described embodiment, the communication between the robot apparatus takes place by the scale sound. This, however, is not limitative, such that, for example, the information can be transmitted or received between robot apparatus by infrared rays.
Likewise, in the above-described embodiment, the dialog by the actions of the robot apparatus is made mainly based on the sound outputting or speech uttering action. This again is not limitative such that the dialog may also be had by actuating movable parts, such as limbs. In such case, the robot apparatus recognizes the counterpart side's action by image recognition means, such as CCD camera 20. If it is retained to be rather difficult to recognize the movement of the robot apparatus, the scale sound, for example, may be output in keeping with the movement of the robot apparatus to permit the meaning of the movement action to be recognized by the scale sound to achieve the reliable dialog between robot apparatus.
If the robot apparatus autonomously generates the sentence to utter the speech, which may be complex and difficult to understand, the counterpart robot apparatus is able to act reliably to comply with the complex text by simultaneously outputting the content information as the scale sound.
According to the present invention, as described above, a robot apparatus is able to detect the information contained in an action output by another robot apparatus by action detection means to cause action outputting means to output an action corresponding to the information as detected by the action detection means, so that the robot apparatus is able to operate responsive to the action of other robot apparatus to improve the entertainment performance of the robot apparatus.
| Number | Date | Country | Kind |
|---|---|---|---|
| P2000-372092 | Dec 2000 | JP | national |
| PCT/JP01/10699 | Dec 2001 | WO | international |
This is a Continuation Application of U.S. Ser. No. 10/182,805, filed on Feb. 5, 2003, now U.S. Pat. No. 6,889,117 which is hereby incorporated by reference in its entirety herein.
| Number | Name | Date | Kind |
|---|---|---|---|
| 6381515 | Inoue et al. | Apr 2002 | B1 |
| 6415203 | Inoue et al. | Jul 2002 | B1 |
| 6421585 | Takamura | Jul 2002 | B1 |
| 6438457 | Yokoo et al. | Aug 2002 | B1 |
| 6442450 | Inoue et al. | Aug 2002 | B1 |
| 6463356 | Hattori et al. | Oct 2002 | B1 |
| 6505097 | Fujita et al. | Jan 2003 | B1 |
| 6668211 | Fujita et al. | Dec 2003 | B1 |
| Number | Date | Country |
|---|---|---|
| 2000-218065 | Aug 2000 | JP |
| 2000-233386 | Aug 2000 | JP |
| 2001-157983 | Jun 2001 | JP |
| WO 0032361 | Jun 2000 | WO |
| Number | Date | Country | |
|---|---|---|---|
| 20050090935 A1 | Apr 2005 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | 10182805 | Feb 2003 | US |
| Child | 10989713 | US |
