Conversational speech analysis method, and conversational speech analyzer

Abstract

The invention provides a conversational speech analyzer which analyzes whether utterances in a meeting are of interest or concern. Frames are calculated using sound signals obtained from a microphone and a sensor, sensor signals are cut out for each frame, and by calculating the correlation between sensor signals for each frame, an interest level which represents the concern of an audience regarding utterances is calculated, and the meeting is analyzed.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic view of a conversational speech analysis according to the invention;

FIG. 2 is an image of the conversational speech analysis according to the invention;

FIG. 3 is a flow chart of the conversational speech analysis used in the invention;

FIG. 4 is a speech/nonspeech activity detection processing and corresponding flow chart;

FIG. 5 is a frame-based sound processing and corresponding flow chart;

FIG. 6 is a sensor activity detection processing and corresponding flow chart;

FIG. 7 is a frame-based sensor analysis process and corresponding flow chart;

FIG. 8 is an interest level judgment process and corresponding flow chart;

FIG. 9 is a display process and corresponding flow chart;

FIG. 10 is a speech database for storing frame-based sound information;

FIG. 11 is a database for storing frame-based sensor information;

FIG. 12 is an interest level database (sensor) for storing sensor-based interest levels;

FIG. 13 is an interest-level database (microphone) for storing microphone-based interest levels;

FIG. 14 is a customized value database for storing personal characteristics;

FIG. 15 is a database used for speaker recognition;

FIG. 16 is a database used for emotion recognition;

FIG. 17 is a time-based visualization of utterances by persons in the meeting; and

FIG. 18 is a time-based visualization of useful utterances in the meeting.

Claims

1. A conversational speech analyzing system comprising: a microphone which captures conversational speech;a sensor which captures sensor information in an area where a meeting is being held; anda computer connected to the microphone and a sensor,wherein the computer stores the speech data and sensor information captured by the microphone and the sensor, respectively, analyzes the progress status of this meeting using characteristic sensor information captured from the sensor in the speech frames and the nonspeech frames of this sound data, and outputs the analysis result.

2. The conversational speech analyzing system according to claim 1, wherein the computer classifies the sound data captured from the microphone as speech frames when speech is detected, and nonspeech frames when speech is not detected, divides the sensor information at the interfaces of the speech frames and the nonspeech frames, and evaluates the interest level in this meeting of persons in the vicinity using a sensor information target level corresponding to speech frames and nonspeech frames.

3. The conversational speech analyzing system according to claim 2, wherein the computer matches a speaker of this speech to speech frames of the speech data captured from the microphone, stores it, and matches the interest level in this meeting of persons in the vicinity to the speaker, and stores it.

4. The conversational speech analyzing system according to claim 2, wherein the computer matches sensor information captured from the sensor with one of the persons in the vicinity, stores it, and matches the interest level in the meeting with the person matched to this sensor information, and stores it.

5. The conversational speech analyzing system according to claim 1, wherein the sensor is a sensor which performs sensing of a body state of the person in the vicinity of this meeting.

6. The conversational speech analyzing system according to claim 1, wherein the computer extracts one or plural detection occasions of a state amount by the sensor and its variation amount from the sensor information, and treats it as feature information.

7. The conversational speech analyzing system according to claim 2, wherein speech data in a speech frame with a high interest level is selected and outputted.

8. A conversational speech analysis method in a conversational speech analyzing system having a microphone, a sensor and a computer connected to the microphone and sensor, the method comprising: a first step that captures conversational speech using the microphone, and stores it in the memory of the computer;a second step that captures sensor information in the vicinity of the meeting using the sensor, and stores it in the memory of the computer; anda third step, wherein the progress status of the meeting is analyzed in the computer using the features of the stored sensor information corresponding to the speech frames and the nonspeech frames extracted from the stored speech data, and the analysis result is output.

9. The conversational speech analysis method according to claim 8, wherein, in the third step, the computer classifies speech data captured from the microphone into speech frames when speech is detected, and nonspeech frames when speech is not detected, divides the sensor information into the speech frames and the nonspeech frames, and evaluates the interest level in this meeting of persons in the vicinity of the meeting using a sensor information target level corresponding to the divided speech frames and nonspeech frames.

10. The conversational speech analysis method according to claim 9, wherein the progress status of the meeting is analyzed using the features of the stored sensor information corresponding to the speech frames and nonspeech frames extracted from the stored speech data, and the analysis result is output.

11. The conversational speech analysis method according to claim 9, wherein, in the third step, the computer matches the sensor information captured from the sensor with one of the persons in the vicinity, stores it, and matches the interest level in the meeting with the person matched to this sensor information, and stores it.

12. The conversational speech analysis method according to claim 8, wherein the sensor performs sensing of a body state of a person in the vicinity of the meeting.

13. The conversational speech analysis method according to claim 8, wherein, in the third step, the computer extracts one or plural detection occasions of a state amount by the sensor and its variation amount from the sensor information, and treats it as feature information.

14. The conversational speech analyzing system according to claim 9, wherein in the third step, the computer selects and outputs the speech data in a speech frame with a high interest level.