Caption Correction Device

Abstract

A device of the present invention obtains a character string of a speech recognition result and a confidence factor thereof. A time monitor monitors time and determines whether or not processing is delayed by checking the confidence factor and time status. When the processing is not delayed, a checker is asked to perform manual judgment. In this event, speech is processed and the manual judgment of the speech recognition result is performed on the basis of the processed speech. When the processing is delayed, automatic judgment is performed by use of the confidence factor. When the character string is judged to be correct as a result of the manual judgment or the automatic judgment, the character string is displayed as a confirmed character string. When the character string is judged to be incorrect, automatic correction is performed by matching on the basis of a next candidate obtained by the speech recognition, texts and attributes of the presentation, a script text, and the like. Character string after the automatic correction is displayed as an unconfirmed character string.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

For a more complete understanding of the present invention and the advantage thereof, reference is now made to the following description taken in conjunction with the accompanying drawing.

FIG. 1 is a view schematically showing a caption correction device 10 according to a preferred embodiment of the present invention.

FIG. 2 is a view showing a concept of judgment of allocation by a judgment allocation unit 2.

FIGS. 3A to 3D are graphs showing a concept of a threshold used for allocation of automatic judgment and manual judgment by the judgment allocation unit 2.

FIG. 4 is a screen example showing manual judgment of jobs as a speech recognition result.

FIG. 5 is an image diagram of margin-provided emphasis reproduction.

FIG. 6 is a table showing an experimental example of a recognition rate in simple matching.

FIG. 7 is a view schematically showing a caption correction system 100 in a case where there is one checker according to the embodiment of the present invention.

FIG. 8 is a view schematically showing a caption correction system 200 in a case where there are two checkers according to the embodiment of the present invention.

FIG. 9 is a view showing an example of a screen in execution of an application of the present invention.

FIG. 10 is a view showing an effect of improvement in a speech recognition rate in an example of the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT

With reference to the drawings, the present invention will be described in detail below according to an embodiment.

FIG. 1 schematically shows a caption correction device 10 as a preferred embodiment of the present invention. The caption correction device 10 includes a speech recognition unit 1, a judgment allocation unit 2, a manual judgment unit 3, an automatic judgment unit 4, an automatic correction unit 5 and a caption display unit 6. However, the device is not limited to the configuration described above but may have other configurations as long as the same functions are achieved or may include a sub-combination thereof.

The speech recognition unit 1 converts a speech of a speaker into a text. In this event, a confidence factor (also called a degree of reliability or a recognition score) for the converted text is outputted. The converted text is not limited to one, and a second candidate, a third candidate and candidates subsequent thereto are outputted together with confidence factors corresponding to the respective candidates. As to the speech recognition itself, a heretofore known technique may be used.

The judgment allocation unit 2 automatically determines, in real time, whether to use automatic judgment means or manual judgment means. The judgment is based on the confidence factor of speech recognition, a maximum allowable delay time, a current delay time (processing time), the number of unprocessed jobs, and average job processing time. The confidence factor of speech recognition is obtained from the speech recognition unit 1, and the maximum allowable delay time is defined in advance, within which jobs are transcribed as being processed in “real time.” Specifically, in a case where the number of unprocessed jobs is small and the difference between the delay time and the maximum allowable delay time is still greatly different from the maximum allowable delay time, automatic judgment is selected when the confidence factor of a character string to be judged is greatly different from an average. This is because the reliability of the speech recognition result is considered to be high when the confidence factor is greatly different from the average. Furthermore, for the judgment allocation unit 2, the average job processing time and the number of unprocessed jobs are taken into consideration. Thereby, a degree of allocation to the automatic judgment can be dynamically changed, as the delay time approaches the maximum allowable delay time.

FIG. 2 shows a concept of determination of the allocation. In the example of FIG. 2, the maximum allowable delay time is set at 8 seconds. Assuming that the current delay time 41 is 3 seconds and the average job processing time (42a to 42c) is 2 seconds, the number of remaining jobs that can be processed is 2. Here, the job may be considered to be a unit processed by the speech recognition unit 1 or a unit for which the confidence factor is determined. For example, in an example of a judgment screen for a speech recognition result as shown in FIG. 4 to be described later, each row corresponds to one job.

In the example of FIG. 2, assuming that the number of unprocessed jobs is 2 or less, there is still room until the delay time reaches the maximum allowable delay time, taken the average job processing time (42a and 42b) into consideration. In this case, the judgment allocation unit 2 determines that high reliability can be obtained by the automatic judgment when the difference between a confidence factor of a character string obtained as a result of speech recognition and the average is greatly large. In other words, the determination depends on whether the confidence factor is larger or smaller than a predetermined threshold. On the other hand, when the average job processing time is longer or the number of unprocessed jobs is larger, the judgment allocation unit 2 determines that it is better to allocate the check to the manual judgment, on the basis of the confidence factor.

Here, conditions for the manual judgment may be defined as the following expression 1. However, the conditional expression for determining whether to allocate a job to the manual judgment or to the automatic judgment is not limited to the following. In example 3, another conditional expression is described.

$\begin{array}{cc}\begin{array}{c}\mathrm{Probability}\mathrm{Density}g\left(x\right)\mathrm{of}\\ \mathrm{Confidence}\mathrm{Factor}x\mathrm{for}\mathrm{Given}\\ \mathrm{Job}\end{array}\le \frac{1}{\begin{array}{c}\mathrm{Number}\mathrm{of}\mathrm{Processable}\mathrm{Jobs}\\ \mathrm{for}\mathrm{Achieving}\mathrm{Real}\text{-}\mathrm{time}\\ \mathrm{Processing}(\mathrm{Finishing}\\ \mathrm{Processing}\mathrm{Within}\mathrm{Maximum}\\ \mathrm{Allowable}\mathrm{Delay}\mathrm{Time})\end{array}}& \mathrm{Expression}1\end{array}$

Each of FIGS. 3A to 3D conceptually shows a threshold a used in the determination described above. Here, as shown in each of FIGS. 3A to 3D, the distribution of confidence factor X complies with a certain probability density function (here, a normal distribution function). In each of FIGS. 3A to 3C, regions where the automatic judgment is chosen due to the confidence factor X different from the average μ by α or more are indicated by slant lines. However, as to the automatic judgment, there are cases of correct judgment and incorrect judgment. Assuming that a is set at 0, everything is processed by the automatic judgment as shown in FIG. 3D.

Here, the description will be given with reference to FIG. 1, again. Both of the manual judgment unit 3 and the automatic judgment unit 4 can also be regarded as a hybrid check device together, which includes both automatic and manual check functions as described above. In the case of manual check, for the purpose of supporting operations by a checker (a judge), the hybrid check device includes functions such as speech speed conversion, removal of silent portions, margin-provided emphasis reproduction, automatic reminder by repetitive reproduction and automatic stop. FIG. 4 shows an example of a screen with which the manual check is performed. Here, FIG. 4 shows processing of confirming the result obtained by the speech recognition. The checker confirms the result by pressing selection buttons “confirmed.” When the checker cannot confirm (when the result is judged to be incorrect) the result, the checker presses selection buttons “incorrect” and the result is passed, as unconfirmed characters, to the automatic correction unit 5 for processing.

Here, the speech speed conversion will be described. It is generally said that a speaking speed is faster than a reading speed. For this reason, it is generally required to adjust a speech speed in order to compare caption candidates with speech by reading the caption candidates. There are a method of directly changing a pitch according to a selected speed, a method of changing only a speed without changing a pitch, and the like. The removal of silent portions is a measure for reducing time for the check, when the speaker makes no sound, by automatically removing the silent portion.

Moreover, the margin-provided, emphasis reproduction means reproducing a preceding line “fade-in” reproduction of a preceding line (or a preceding word) and “fade-out” reproduction of a subsequent line (or a subsequent word). The margin-provided emphasis reproduction means that a preceding line (or a preceding word) is reproduced in a fade-in manner, and a subsequent line (or a subsequent word) is reproduced in a fade-out manner. FIG. 5 is an image diagram of the emphasis reproduction. Here, characters that “fade in” are indicated in a manner that the characters gradually get larger word by word, and characters that “fade out” are indicated in a manner the characters gradually get smaller word by word. In such a manner, it is possible to support the manual check of words.

Moreover, the automatic reminder by repetitive reproduction means reminding the checker to input by reproducing repetitive sounds. Furthermore, the automatic stop means stopping the reminder when the checker uses a mouse, a keyboard or the like to input.

Note that the caption correction device 10 can include the manual check function of the manual judgment unit 3 as many as the number of checkers. This will be described later.

The automatic correction unit 5 shown in FIG. 1 performs automatic correction by matching a keyword list dynamically generated from a backup material of a presentation with character strings judged to be incorrect. The backup material includes, for example, a presentation page, a speaker note and a supplementary text material such as a script.

In the automatic correction unit 5, when a result of matching does not exceed a threshold for any of keywords from the keyword list, a correction result is obtained from a second candidate obtained from the speech recognition. The candidates after the second candidate are character strings each having a confidence factor lower than that of the first candidate among the character strings transcribed by the speech recognition unit 1. For the matching, DP matching (Dynamic Programming Hatching) is used, which is a common method of comparing character strings. Furthermore, when there is no match at the character string level, matching is also performed at a phonemic level.

Here, in simple matching, it should be noted that, in many cases, a recognition rate is adversely lowered by the matching. Specifically, a rate at which recognition is improved by correcting incorrect portions is set to be Re, and a probability of adversely lowering the recognition rate by accidentally correcting correct portions is set to be Rc. Here, a result of comparison between Re and Rc is not necessarily Re>Rc. FIG. 6 shows one piece of actual experimental data of the simple matching for reference. In the matching method of the present invention, the matching is performed after the check is performed by the automatic judgment or the manual judgment, in advance. Accordingly, the risk of adversely lowering the recognition rate is small.

Here, the description will be given with reference to FIG. 1, again. In the automatic correction unit 5, when no proper correction result can be obtained by the provision of the second candidate obtained from the speech recognition with given conditions, a correction result is obtained by use of the third candidate and those subsequent thereto. When none of the candidates can foe a proper correction result, the original first candidate is outputted as it is without being corrected.

Generally, in a presentation using a presentation application, a plurality of presentation pages are used for a slide show. For this reason, it is possible to create a keyword list from character strings included in each of the presentation pages actually presented by a presenter of the presentation. What stores keywords included in each of the presentation pages is called a page unit presentation keyword DB, which is indicated as a page unit presentation keyword DB 7 in FIG. 1. For example, when a page switching event 8 occurs in response to an operation by the presenter using the mouse or the keyboard, the switching of pages can be transcribed. Here, presentation keywords are a list of keywords such as words and paragraphs, which are obtained from presentation pages, speaker notes and supplementary texts such as scripts. Basically, for the list of keywords, TF•IDF (Term Frequency•Inverse Document Frequency) is used. The TF•IDF is a method which uses a morphological analysis and weighting by use of element attributes in the presentation. Since the TF•IDF is a heretofore known method, description thereof will be omitted here. The provision of the page unit presentation keyword DB 7 allows the automatic correction unit 5 to use the DB for the post-editing of captions, and thus to display the captions in real time by automatic matching with the keywords.

Lastly, the caption display unit 6 automatically merges the character strings (confirmed character strings and unconfirmed character strings) obtained from the automatic judgment unit 4, the manual judgment unit 3 and the automatic: correction unit 5. In addition, the caption display unit 6 uses various display styles in terms of font attributes and the like, depending on whether the character strings are correct or incorrect. Here, the caption display unit 6 expresses the character strings judged to be incorrect by using other character types such as phonemes, hiragana characters, in addition to a normal character type.

In the embodiment shown in FIG. 1, the caption correction device 10 according to the present invention has been described above as a single device. However, the caption correction device 10 may have a system configuration in which each of the functions is regarded as one separate device. FIG. 7 shows this system configuration of a case where there is one checker. Here, FIG. 7 shows devices and internal processings thereof, which devices correspond to the respective functional units shown in FIG. 1.

The following is a flow of processing by a caption correction system 100 shown in FIG. 7. However, since configurations of the respective devices approximately correspond to those in the device shown in FIG. 1, redundant descriptions will be omitted.

1) A recognition result character string is obtained by speech recognition device 11. Here, not only the recognition result character string but also a confidence factor corresponding to the recognition result character string. In addition, character strings each having a confidence factor lower than that of the recognition result character string are obtained as next candidates (the second candidate, the third candidate or the like) for the recognition result character string. The character strings of the respective next candidates may be stored in a next candidate DB 16 as shown in FIG. 7.

2) The recognition result character string is allocated to either manual judgment or automatic judgment by a judgment allocation device 12 (Step S10). The manual judgment provides a confidence factor to be obtained as a result higher than that provided by the automatic judgment. For this reason, when there is sufficient spare time, the manual judgment is selected. Meanwhile, when the confidence factor obtained from the speech recognition is greatly different from an average (for example, 50%), there is a high possibility that the recognition result character string is automatically judged (for example, the character string is judged to be correct when the confidence factor is 95% and the character string is judged to be incorrect when the confidence factor is 5%) . Accordingly, in order to achieve real-time captions, the judgment is dynamically allocated to the automatic judgment in accordance with the degree of difference between the confidence factor obtained by the speech and the average of the confidence factors, in a situation where there is a delay in consideration of job situations, maximum delay time and actual elapsed time. An example of the judgment allocation device 12 will be described later as example 3.

3) In the case of the manual judgment, speech is automatically reproduced by speech speed conversion (Step 811), removal of unnecessary silent portions (Step S12), and a speech reproduction method (margin-provided emphasis reproduction, automatic reminder by repetitive reproduction and automatic stop). Then the checker is allowed to perform the manual judgment by using a method described in example 4 for comparing the reproduced speech with the speech recognition result (Step 13). A UI (User Interface) for the judgment by the checker will be described in example 2 to be described later. In a case where the automatic judgment is selected in the judgment step of S10, the automatic judgment (Step S14) is directly executed. Moreover, in a step S15 of judging confirmation, a job queue 18 managed by the judgment allocation device 12 is updated,

4) For the character string that is not confirmed (NO in the judgment step S15), matching the character string with the page unit presentation keyword DB 17 is performed by use of the matching method described above. In a step S17, a score Sp (Score Point) according to the matching is calculated (Step S16). When the Sp is equal to or more than a predetermined reference value, a character string obtained by correcting the character string is set to be an unconfirmed character string (Step S19). On the other hand, when the Sp is lower than the reference value, a character string obtained by correcting the second candidate is outputted as an unconfirmed character string (Step 518).

5) In a case where the second candidate is judged to be not suitable as a correction result according to previously given conditions, such as discriminatory expressions, the third candidate is sequentially obtained from the next candidate DB 16. Accordingly, correction is performed by use of the obtained candidate.

6) If none of the candidates obtained by the speech recognition device 11 can be a correction result, the first candidate is outputted as it is without being corrected.

7) As to the confirmed character string and the unconfirmed character string (subjected to matching), the font attributes thereof are changed so as to clearly differentiate the two strings. This will be described later as example 1.

Normally only one checker is sufficient as described above. However, the system of the present invention is also applicable to the case where there are two checkers or more. In such a case, by providing manual judgment devices 13 for the number of checkers, the recognition rate and caption display are further improved. FIG. 8 shows a caption correction system 200 in the case where there are two checkers.

Characteristics of the case where there are a plurality of checkers are as follows.

1) The same number of manual judgment devices 13a and 13b as that of checkers are provided.

2) A judgment allocation device 12 allocates a job to available one of the manual judgment devices 13a and 13b.

3) Character strings which are not confirmed by the manual judgment devices 13a and 13b are corrected once by an automatic correction device 15. Thereafter, the character strings are returned to the judgment allocation device 12. When any one of the manual judgment devices 13a and 13b are available, the judgment allocation device 12 allocates jobs to be manually judged thereto the manual judgment device 13a or 13b. On the other hand, when the manual judgment devices 13a and 13b are not available, unconfirmed display is directly performed.

4) There is no point in executing matching twice by the automatic correction device 15 for the same character string in such a manner in which the character string is passed to the automatic correction device 15 from the manual judgment device 13a or 13b, returned to the manual judgment device 13a or 13b, and again passed to the automatic correction device 15. Specifically, in such a case, the same keyword and the second candidate which are used in the first matching by the automatic correction device 15 are not used twice. Accordingly, matching is performed by use of keywords other than the keyword used in the first time or the third candidate other than the second candidate.

EXAMPLE 1

Method for Displaying Confirmed and Unconfirmed Character Strings

FIG. 9 shows an example of the present invention. In FIG. 9, an image of a speaker (presenter) is displayed on a window screen denoted by reference numeral 64. The screen 64 shows a result of recognition of speech made by the speaker. Additionally, a window screen 60 snows results each obtained by manual or automatic check performed by the manual judgment device 13 on recognition character strings which is obtained by the speech recognition. However, confidence factors of the speech recognition may be hidden. In order to display whether the check by the manual judgment device 13 is confirmed or not confirmed, a checker performs confirmation judgment as indicated by S61. The character string judged to be incorrect is set to be an unconfirmed character string and is subjected to matching in a step 62 (the automatic correction device 15). As a result of the matching, the corrected character strings are displayed on a window 63 while being classified into confirmed character strings and unconfirmed character strings. Here, FIG. 9 shows an example of a display method of a case where the confirmed character strings are displayed in black and the unconfirmed character strings are displayed in italic (it is actually much better to display in other colors). As can be seen from FIG. 9, even the unconfirmed display may be correctly displayed by use of, for example, a PPT file or presentation backup materials of other texts. Moreover, the confirmed character strings can be confirmed by the automatic judgment as well as by the manual judgment. Accordingly, character strings confirmed respectively by both judgments can also be displayed while being differentiated from each other by use of different font attributes.

EXAMPLE 2

Manual Operation in Manual Judgment Device 13

An example of a UI (User Interface) for a manual operation by a user in the manual judgment device 13 will be described here.

1) A key operation (or a mouse click and the like) is performed in any event. For example, Enter key is pressed when a target character string is correct and Space key is pressed when the target character string is incorrect.

2) Selection is carried out as in the case of 1), but a time-out period is provided. When none of the keys is pressed within the period, default is forcibly selected. Since the recognition result is usually correct in many cases, improvement in performance can be expected by setting the default to be correct.

EXAMPLE 3

Judgment Allocation Device 12

Conditions of the automatic judgment are set as follows. The more the confidence factor is different from the average, the more likely the automatic judgment is to be carried out in consideration of allowable delay time, average job time and elapsed time. In addition, as the allowable delay time is running out, the confidence factor close to the average is also more likely to be allocated to the automatic judgment.

Description will be given in detail of a calculation method in the case where a normal distribution is assumed for a confidence factor Ci. A probability density function f(x) of the normal distribution where an average and a variance are represented respectively by μ and σ, is given by the following expression,

$\begin{array}{cc}f\left(x\right)=\frac{1}{\sigma \sqrt{2\pi }}{}^{-\frac{{\left(x-\mu \right)}^2}{2{\sigma }^2}}& \mathrm{Expression}2\end{array}$

Based on the expression described above, a probability that a random variable X becomes equal to or less than α is given by the following expression,

$\begin{array}{cc}P\left(X\le a\right)={\int }_{-\infty }^af\left(x\right)x& \mathrm{Expression}3\end{array}$

Generally, this integral calculation is not easy. However, by using a linear relationship of the probability density function and given values, it is possible to determine which is to be required the automatic judgment or the manual judgment. The automatic judgment is required when the following conditional expression is satisfied assuming that the allowable delay time is Twmax, the average job processing time is Tave and the process time is Ti. On the other hand, the manual judgment is required when the conditional expression is not satisfied. In the case of the automatic judgment, it is possible to judge that the confidence factor not more than the average μ leads to unconfirmed display and that the confidence factor larger than the average μ leads to confirmed display.

|μ−Ci|≧|normsin v(1/Nj)|  Expression 4

Note that “normsin v” represents an inverse function of a standard normal cumulative distribution. Here, since Nj is the number of remaining jobs, Nj is set at a value rounded to the nearest natural number without exceeding Ni given by the following expression.

Ni=(Tw max−Ti)/Tave   Expression 5

Note that the above values can be obtained in the same manner even when there are multiple manual judgment devices.

EXAMPLE 4

Method of Comparing Speech Recognition Result with Speech

As a common method of a checker to compare a speech recognition result with speech, the checker usually compares character strings with the speech by reading the character strings while listening to the speech. However, for example, it is also possible for visually impaired people or the like to compare character strings with input speech while simultaneously reproducing the character strings by speech synthesis.

EXAMPLE 5

Checker Using No Sound

The method of comparing the speech recognition result with the input speech has been described as the means of the present invention. However, in many cases, the speech recognition result includes significant errors. For this reason, even though there is no input speech, simple check can be executed. Specifically, people with hearing difficulties or the like can also surely perform the check.

EXAMPLE 6

Operational Effects

As operational effects of the present invention, reduction in costs, improvement in a recognition rate, improvement in understanding and a real-time characteristic will be summarized as below.

Cost Reduction Effects

First, by use of the method of the present invention, significant cost reduction can be expected compared with manual caption correction. Note that, as to delay time from the actual time, no significant difference is found between the respeak, the stenography and the embodiment of the present invention.

Recognition Rate Improvement Effects

FIG. 10 shows a result of a matching experiment based on the result of the demonstration experiment at a certain university introduced in BACKGROUND OF THE INVENTION. Although already described above, it is generally said that the real-time captions require the recognition rate of 85% or more, preferably, 90% or more. A probability that the recognition rate exceeds 85% is improved from 27% to 82%, and a probability that the recognition rate exceeds 90% is improved from 0% to 27%

Other Effects

Although quantification is difficult, particularly significant improvement in the recognition rate is found for important keywords. For this reason, the improvement in the recognition rate significantly contributes to improvement in understanding compared with the simple speech recognition itself. Moreover, as to discriminatory expressions not intended by the speaker and formatting, significant improvement is achieved by matching with the page unit presentation keywords.

The present invention has been described above as the caption device that is one embodiment or the system. However, the device or the system of the present invention can be realised as hardware, software or a combination of hardware and software. As implementation by the combination of hardware and software, implementation in a computer system having a predetermined program is cited as a typical example. In such a case, when the predetermined program is loaded into the computer system and executed, the program allows the computer system to execute the processing according to the present invention. This program includes a group of commands that can be expressed by any language, code or notation. Such a group of commands enables the system to directly execute specific functions or to execute the functions after any one of or both of (1) conversion into another language, code or notation and (2) copying onto another medium are performed. Needless to say, the scope of the present invention includes not only such a program itself but also a program product including a medium in which the program is stored. The program for executing the functions of the present invention can be stored in arbitrary computer-readable media such as a flexible disk, a MO, a CD-ROM, a DVD, a hard disk device, a ROM, an MRAM and a RAM. The program described above can be downloaded from another computer system connected through a communications line or can be copied from another medium in order to be stored in the computer-readable medium. Moreover, the program can also be stored in a single or a plurality of recording media by compression or division into several pieces.

Note that the computer or the computer system used in the present specification refers to not only a general personal computer or a genera 1-purpose computer but also various devices equipped with a microprocessor (a CPU or a MPU).

According to the: present invention, in a caption correction system using speech recognition, it is possible to improve a recognition rate, to improve a real-time characteristic, and to achieve significant cost reduction compared with manual caption production.

Although the present invention has been described above based on the embodiment and the examples, the technical scope of the present invention is not limited to those described in the embodiment and the like Various changes or modifications can be added to the foregoing embodiment. Moreover, it is apparent from description of the scope of claims that embodiments having such changes or modifications added can also be included in the technical scope of the present invention.

Claims

1. A caption correction device for correcting, in real time, a caption for speech in a presentation, comprising: a speech recognition unit, for recognizing the speech in the presentation, and for outputting, as a result of the speech recognition, one or a plurality of texts of character string candidates and confidence factors of the speech recognition which correspond to the respective character string candidates;a judgment allocation unit for selecting one of automatic judgment and manual judgment by use of the confidence factors, job processing information and a predetermined maximum allowable delay time, in order to handle a first character string candidate having the highest value of the confidence factor among the character string candidates;an automatic judgment unit for automatically judging whether to set the first character string candidate as a confirmed character string in response to the selection of the automatic judgment by the judgment allocation unit; anda manual judgment unit for manually judging whether to set the first character string candidate as a confirmed character string in response to the selection of the manual judgment by the judgment allocation unit.

2. The caption correction device according to claim 1, further comprising: an automatic correction unit for outputting, as a correction result, keywords obtained on the basis of a matching score obtained by comparing a keyword list of the presentation with the character string which is not confirmed by the manual judgment unit; anda caption display unit for displaying the confirmed character strings and the character strings as the correction result while differentiating the both character strings.

3. The caption correction device according to claim 1, wherein the job processing information includes a current delay time, the number of unprocessed jobs in the speech recognition unit, and an average job processing time thereof.

4. The caption correction device according to claim 1, wherein the judgment allocation unit selects the manual judgment in a case where a probability density, which is obtained from the confidence factor corresponding to a given job, is not more than the inverse of the: number of jobs which can be processed within the maximum allowable delay time.

5. The caption correction device according to claim 1, wherein the judgment allocation unit selects the automatic judgment in accordance with the degree of difference between the confidence factor and the average of the confidence factors.

6. The caption correction device according to claim 1, wherein the manual judgment unit includes at least one of speech speed conversion means, silent portion removal means, margin-provided emphasis reproduction means, automatic reminder means by repetitive reproduction, and automatic stop means.

7. The caption correction device according to claim 1, wherein the manual judgment units are provided as many as the number of judges who perform the manual judgment.

8. The caption correction device according to claim 2, wherein the keyword list is dynamically generated from a page unit presentation keyword DB.

9. The caption correction device according to claim 2, wherein the automatic correction unit outputs the keywords as the correction result when the matching score is not less than a predetermined reference value, andthe automatic correction unit outputs a second character string candidate as the correction result when the matching score is less than the predetermined reference value.

10. The caption correction device according to claim 9, wherein in a case where the second candidate is judged to be not suitable as the correction result on the basis of predetermined conditions, the automatic correction unit subsequently judges the remaining candidates one by one as to whether or not the candidate is suitable as the correction result.

11. The caption correction device according to claim 10, wherein in the judgment based on the conditions, when any one of the second candidate and the remaining candidates is any one of discriminatory and provocative expressions which are predetermined, the candidate is judged to be not suitable as the correction result.

12. The caption correction device according to claim 2, wherein the caption display unit displays the confirmed character string and the corrected character string respectively by different font attributes or different letter types.

13. A method of correcting, in real time, a caption for speech in a presentation, comprising the steps of: recognizing the speech in the presentation, and then outputting, as a result of the speech recognition, one or a plurality of texts of character string candidates and confidence factors of the speech recognition which correspond to the respective character string candidates;selecting one of automatic judgment and manual judgment, by use of the confidence factors, a predetermined maximum allowable delay time, a current delay time, and the number of unprocessed jobs in the speech recognizing step and an average job processing time thereof, in order to confirm a first character string candidate having the highest value of the confidence factor among the character string candidates;automatically judging whether to set the first character string candidate as a confirmed character string in response to the selection of the automatic judgment in the selecting step; andmanually judging whether to set the first character string candidate as a confirmed character string in response to the selection of the manual judgment in the selecting step.

14. The method according to claim 13, further comprising the steps of: outputting keywords as a correction result on the basis of a matching score obtained by comparing a keyword list of the presentation with the character string which is not confirmed by the manually judging step; anddisplaying the outputted confirmed character strings and the character strings as the correction result while differentiating the both character strings.

15. The method according to claim 13, wherein, in the selecting step, the automatic judgment is selected in accordance with the degree of difference between the confidence factor and the average of the confidence factors.

16. A computer program for causing a computer to execute the steps in the method according to any one of claims 13 to 15.

17. A program product for correcting, in real time, a caption for speech in a presentation, the program product comprising a computer-readable medium on which a computer program is recorded, wherein the computer program includes:speech recognition code means which transcribes the speech in the presentation, and which outputs, as a result of the speech recognition, one or a plurality of texts of character string candidates and confidence factors of the speech recognition which correspond to the respective character string candidates;judgment allocation code means which selects one of automatic judgment and manual judgment, by use of the confidence factors, a predetermined maximum allowable delay time, a current delay time, and the number of unprocessed jobs outputted by the speech recognition code means and an average job processing time thereof, in order to handle a first character string candidate having the highest value of the confidence factor among the character string candidates;automatic judgment code means which automatically judges whether to set the first character string candidate as a confirmed character string in response to the selection of the automatic judgment by the judgment allocation code means; andmanual judgment code means which manually judges whether to set the first character string candidate as a confirmed character string in response to the selection of the manual judgment by the judgment allocation code means.