Computer-aided transcription system using pronounceable substitute text with a common cross-reference library

Abstract

A transcription network having linked computer terminals for a court reporter and for examining, defending, and associate attorneys is disclosed. Cross-reference libraries, phoneme libraries, phoneme dictionaries, and pronunciation indices are used by a reporter terminal to generate readable text transcripts which are communicated to the other terminals for real-time review. Cross-reference libraries provide for the bulk of the transcription while stenographic keystrokes that cannot be directly transcribed are converted to phoneme text or other pronounceable form for down-line readability. A common cross-reference library is provided to supplement local cross-reference libraries to aid transcription and to provide for standardization. The reporter terminal provides for update and preparation of the local cross-reference libraries using case law and evidence prior to a proceeding, as well as provides key-stroke training for the court reporter. Foreign language tables are also provided to service proceedings involving two or more languages.

Description

INCORPORATION BY REFERENCE

The descriptive matter of application Ser. No. 08/065,163, now U.S. Pat. No. 5,815,639, and application Ser. No. 08/036,488, now U.S. Pat. No. 5,369,704, is incorporated herein by reference in its entirety, and is made part of this application.

BACKGROUND OF THE INVENTION

This invention relates to a computer-aided transcription system used by court reporters which provides readable testimony to attorneys, judges, court reporters, witnesses and clients in real-time.

As is well known, legal proceedings such as a deposition or trial involve the participation of, among others, an examining attorney who asks questions and a witness who must answer ("testify") while under oath. These answers ("testimony") are recorded by a court reporter, along with the associated questions and related conversation, using a stenographic recorder. A stenographic recorder is a machine which provides a set of keys which are stroked by the court reporter in various combinations and sequences to represent spoken words.

Older versions of the stenographic recorder only record the court reporter's key-strokes much the way a computer printer does, creating a paper tape containing the printed key-strokes as a record of the legal proceeding. After the legal proceeding ends, the court reporter reads the paper tape and manually transcribes each printed key-stroke back into the words that were spoken, creating a readable transcript of the testimony. This manual transcription process takes a great deal of time.

Newer versions of the stenographic recorder produce electronic signals representative of each key-stroke recorded. The electronic signals drive computers that help automate the transcription process. These computers are referred to as Computer Aided Transcription ("CAT") systems which translate stenographic key-strokes into text using a court reporter's personalized cross-reference library. These CAT systems only automatically transcribe those key-strokes that have textual counterparts in that court reporter's cross-reference library. Keystrokes having no counterparts must be manually transcribed by the court reporter. The ratio between the number of words automatically transcribed and those requiring manual transcription is referred to herein as the "percentage of automated transcription".

Using the newer versions of recorders and CAT systems, only seasoned, consistent court reporters having well-developed cross-reference libraries can achieve a relatively high percentage of automated transcription, while beginners and inconsistent reporters tend to produce relatively low percentages. Low percentages result in costly inefficiency resulting from the longer period of manual transcription required. Additionally, if real-time down-line transcription is desired, a high percentage of automated transcription becomes critical. In a down-line transcription environment, both counterpart text for transcribed key-strokes and all untranscribed key-strokes are sequentially transmitted down-line to the various attorneys, clients, judges and witnesses for their real-time review. However, in most cases, only court reporters can read untranscribed key-strokes. The down-line reviewers generally cannot comprehend them. As a result, the lower the percentage of automated transcription, the less likely a court reporter will ever be rehired by the down-line attorneys.

The percentage of automated transcription depends on several factors. For example, if a court reporter fails to consistently depress the same stenographic recorder keys for the same word or word sequence, the percentage of automated transcription tends to drop. Currently, to overcome this problem, the court reporter must increase their skill through practice. Beginner court reporters face a more substantial problem. Not only must they learn to consistently stroke the stenographic keys, but they must also enhance and refine their "beginner" cross-reference library so that the desired high percentage of automated transcription can be reached. The enhancement and refining process takes a great deal of time with the end result being a personalized cross-reference library that no other reporter can use.

Additionally, even the most seasoned court reporters cannot anticipate all of the unique or new words and names that will be encountered in an upcoming deposition. As a result, current CAT systems fail to transcribe such words and names. To increase the percentage of automated transcription, court reporters often ask the attorneys for a list of potential unique words and names well before the testimony is recorded so that appropriate refinement of their cross-reference library might be made. However, attorneys cannot remember or anticipate all such words, and, more often than not, forget to provide the requested list. Furthermore, because of the anticipation problem and nature of such words, one hundred percent automated transcription cannot be guaranteed. Thus, incomprehensible, untranscribed key-strokes always reach the down-line reviewers.

Because of personalization, each court reporter must regularly back-up and carry copies of their cross-reference library (usually on disk) in case their library is accidentally lost or damaged. Failure to do so may result in the court reporter not being able to conduct the deposition, and, possibly more devastating, the court reporter may have to start from scratch as would a beginner to develop another cross-reference library. Also resulting from personalization, a court reporter's library cannot be used to transcribe the key-strokes of another reporter. Doing so not only yields a low percentage of automated transcription, but might also result in virtually undetectable, wrong transcription.

To handle depositions or trials involving two or more languages, translators are required. However, using translators creates one significant problem--the validity of the translation. Translators, like any other human, often unknowingly misunderstand what is being said. Even more often, a translator will get sloppy and tend to summarize what they hear. In either case, the transcript becomes flawed. With an increased number of international legal matters, the use of translators with their inherent translation errors continues to grow.

Currently, there are over thirty thousand court reporters and hundreds of thousands of attorneys and judges in the United States alone. Hence, it would be highly desirable to solve the foregoing variety of problems enumerated above facing court reporters and down-line attorneys, judges and witnesses in conducting legal proceedings such as a deposition or trial.

Therefore, it is an object of the present invention to provide a method and apparatus which will aid court reporters in transcription of testimony generated in real-time by an automatic transcription system.

It is another object of the present invention to provide a method and apparatus which will aid court reporters in eliminating the need for personalized transcription libraries for use in transcription of testimony generated in real-time by an automatic transcription system.

A further object of the present invention is to provide a method and apparatus which will aid beginning and inconsistent court reporters in the transcription of testimony generated in real-time by an automatic transcription system.

A further object of the present invention is to provide a method and apparatus which will aid court reporters in the transcription of testimony generated in real-time by an automatic transcription system through use of a fully functional, standard cross reference library.

It is another object of the present invention to provide a method and apparatus which will aid court reporters in developing and maintaining a common cross reference library for use in the transcription of testimony generated by a CAT system.

It is yet another object of the present invention to provide a method and apparatus which will aid court reporters in training themselves to use a standard cross reference library substantially common to all court reporters.

Another object of the present invention is to provide a method and apparatus which will aid court reporters in training themselves to use a common cross reference library.

A further object of the present invention is to provide a method and apparatus which will act as a backup to the cross-reference library in providing automated transcription for testimony generated by an automatic transcription system.

It is another object of the present invention to provide down-line reviewers with comprehensible text representing untranscribed key-strokes representing words or names having no textual counterpart in the cross-reference library.

SUMMARY OF THE INVENTION

These and other objects of the present invention are achieved in a reporting system having a transcriber which operates on coded representations of spoken words and having cross-reference and phoneme libraries. The reporting system comprises a means which uses a cross-reference library for identifying associations between coded representations of spoken words and actual alphabetic and numeric text. A phoneme means provides for associating spoken words with pronounceable text using a phoneme library, and operates on spoken words for which the cross-reference means cannot identify associations.

The reporting system accomplishes other objects by additionally providing a pronunciation index means which operates on representations of spoken words for which the phoneme and cross-reference means cannot identify associations. A phoneme library is also provided which is responsive to the phoneme means by identifying associations between code representations of spoken words and corresponding alphabetical and numeric text. Additionally, foreign language tables are provided by the reporting system to provide for foreign language translation.

Objects are also achieved in a transcription system which operates on coded representations of spoken words. This system comprises a local cross-reference library which is stored in a local database and a common cross-reference library which is stored in a remote database. A means is provided for updating the common and local cross-reference libraries. Additionally, means might also be provided for court reporter training and interactive updating of the cross-reference libraries.

Objects are also achieved in an alternate transcription system which comprises a cross-reference library and a case database which contains words relating to a specific lawsuit at issue. A means is provided for automatically comparing the cross-reference library with the case database, and for updating the cross-reference library.

Other objects and further aspects of the present invention will become apparent in view of the following detailed description and claims with reference to the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a perspective view the transcription system of the present invention illustrating the interaction of a CAT system configured to proved real-time, down-line transcription.

FIG. 2 is a diagram representing the association of data fields in records used as the fundamental building blocks in the data structure of the cross-reference library according to the present invention.

FIG. 3 is a detailed diagram representing the overall data structure used by the CAT system of the present invention.

FIG. 4 is a diagram representing the phoneme library used in the present invention wherein substitutions of readable phoneme text can be found for untranscribed key-strokes to aid down-line viewing.

FIG. 5 is a diagram representing the production key index used in the present invention wherein substitutions of a series of pronunciation-strings can be found when phoneme text is unavailable.

FIG. 6 is an illustration of the phoneme record used as a basic building block in the overall data structure of the phoneme library which is used to locate exact text substitution for identified phoneme text according to the present invention.

FIG. 7 is a perspective view of a down-line terminal of the present invention which illustrates the use of phoneme text replacement of untranscribed key-strokes.

FIG. 8 is an exemplary diagram of a foreign language table which is used to cross-reference text of a first language that corresponds to translated words of a first language.

FIG. 9 is a detailed perspective view of the CAT system of the present invention which utilizes case evidence text, case law words and court reporter training sessions to prepare for an upcoming trial or deposition.

FIG. 10 is a detailed perspective view of the CAT systems of the present invention which utilizes a common cross-reference library to help to create a standard cross reference library used by each court reporter which will result in higher transcription percentages.

DESCRIPTION OF THE PREFERRED EMBODIMENT

FIG. 1 provides an overview of the interaction of a CAT system 13 according to the present invention for providing real-time, down-line transcription. In one embodiment of the present invention, at a trial or deposition a stenographic recorder 11 records digital codes representing the key-stroke entered by a court reporter via a keyboard 31. The recorded key-strokes are intended to correspond to the words spoken at a deposition or trial. The stenographic recorder 11 communicates these key-strokes codes to a CAT system 13 via a link 9. Upon receipt, the CAT system 13 attempts to transcribe the key-stroke codes into the exact text of the words which were spoken to provide for a real-time textual display of the transcript.

To accomplish transcription, the CAT system 13 primarily uses a cross-reference library 15 to locate the exact text counterpart of the spoken words. However, whenever the cross-reference library 15 fails to provide an exact text counterpart, the CAT system 13 resorts to using a phoneme library 17 to provide readable substitute text (referred to hereinafter as "phoneme text") as the next best option to the exact text replacement. Although this phoneme text does not provide exact transcription, it does provide pronounceable text which can be easily understood by the reader.

The CAT system 13 transmits both the exact and, where necessary, phoneme text down-line to attorney terminals 19 and 21 via a communication link 23 for real time review. Further detail regarding the down-line terminals such as terminals 19 and 21 can be found in the parent U.S. application Ser. No. 08/036,488, filed Mar. 24, 1993, by Bennett et al. now U.S. Pat. No. 5,369,704, which is incorporated herein by reference.

FIG. 1 also illustrates the use of a phoneme dictionary 23 which is used in an alternate embodiment of the present invention. When exact text cannot be located in the cross-reference library 15, the CAT system 13 first identifies the phoneme text in the phoneme library 17 as described above. However, instead of immediately sending the identified phoneme text down-line for viewing, the CAT system 13 makes a second attempt at accessing the exact transcription text via the phoneme dictionary 23. The phoneme dictionary 23 has the same structure and searching strategy as the cross-reference library 15. The only difference being instead of indexing using a series of key-stroke codes, the phoneme dictionary indexes using the phoneme strings. Only after a failed second attempt of locating an exact replacement will the CAT system 13 communicate the phoneme text down-line.

If when forced to resort to the phoneme library 17, no phoneme text counterpart to a key-stroke can be found, the CAT system 13 accesses a pronunciation index 25 which will substitute a dictionary-like pronunciation for the key-stroke codes prior to communication down-line. This provides a readable pronunciation to the viewer which is second only to the phoneme text.

Additionally, where the transcription involves participants of more than one language, the CAT system 13 utilizes foreign language tables 27 to provide readable real-time transcription for all of the down-line participants such as those located at the terminals 19 and 21.

More specifically, in response to the court reporter stroking a keyboard 31, the recorder 11 creates a key-stroke code and transmits this to the CAT system 13 (the reporter's terminal) along the communication link 9. The CAT system 13 displays the key-stroke codes received in two different formats on a screen 100 of the CAT system 13. The CAT system 13 provides a tape window 101 which is shaped and sized much like a paper tape, such as a paper tape 35 of the recorder 11. The tape window 101 displays alphabetic representations for received key-stroke codes in a line by line fashion wherein each key-stroke code is represented on a single line. The tape window 101 thus provides an electronic version of a paper tape which is generally in the format familiar to court reporters.

The screen 100 also includes a transcription window 103 which displays the alphabetic and numeric text representative of the spoken words, i.e., displays the transcribed version of key-stroke codes received from the recorder 11. Through user selection, the windows 101 and 103 can be configured to update the screen 100 upon complete transcription of each: 1) word; 2) sentence; 3) Q & A; or 4) Q or A. The display can also be chosen to operate in either scrolling or paging fashion. For example, with a preferred display selection, the transcription window 103 will only display the current Q & A as it is developed on a sentence by sentence basis. After completing the transcription of the first sentence of a new question, the CAT system 13 clears the currently displayed Q & A (the old Q & A) from the window 103, and displays the transcribed alphabetic and numeric text of the first sentence of the new question. As each additional sentence of the new Q & A is received and transcribed, it too is added to the window 103. After fully transcribing the new Q & A, window 103 holds the display of that new Q & A until the end of the first sentence of the next Q & A; upon which, this cycle repeats.

To set up the mode of updating the windows or to access any of a variety of other CAT system commands, the screen 100 also includes a command line 105. In one embodiment, the command line 105 lists several one-word commands displayed horizontally in a single line of text. The commands listed by the command line 105 can be selected directly using a mouse, pen, fingertip or other input device (none of which are shown) or indirectly by simultaneous pressing the "alt" key and the first letter of the desired command on the keyboard 107. Any other mode or method of command selection involving, for example, button bars, pull-down menuing, or icons might also be used.

As previously stated, a court reporter strokes the stenographic recorder 11 in response to the spoken word. Generally, a given word may demand either a single stroke or multiple strokes based on the complexity of its phonetic composition. Upon completing the strokes necessary for a given word, the court reporter immediately begins stroking the next word. No space bar or other word separator is stroked between words. As a result, all of the strokes run together. However, the court reporter does stroke to indicate an end of a sentence. Therefore, the CAT system 13 operates on key-strokes in sentence units (the basic unit of transcription) so that the beginning and end of each word can be most easily located. In addition, some court reporters use key-strokes to indicate punctuation. When used, the CAT system 13 can decrease the size of the basic unit of transcription by grouping only those words between punctuation. This may result in an easier identification of words and possibly provides a higher percentage of transcription.

Providing the foundation of the transcription process, the CAT system uses the cross-reference library 15 which lists the exact text of each word that the court reporter has transcribed to date using that library. Each of these words are associated with the corresponding stenographic key-stroke or sequence of key-strokes that represents that word. Although the contents of the cross-reference library 15 can be displayed in a visual manner, it is stored in a binary tree structure for efficient searching.

Upon receiving the first key-stroke code of the sentence, the CAT system 13 begins transcription. This first code is compared to all of the possible first stroke codes within the court reporter's cross-reference library. If a complete cross-referenced word is found, the CAT system 13 preliminarily treats the first key-stroke code as a complete word. Upon receiving the second key-stroke code in the sentence, the CAT system 13 again treats this second key-stroke code as the beginning of another word and looks again to all of the possible first stroke codes within the library. If a complete cross-referenced word is identified, the CAT system 13 similarly treats the second key-stroke code as a second complete word, and the process repeats. If at any point a cross-referenced word cannot be found for a single key-stroke code, subsequent and previous codes are utilized to branch to words requiring multiple key-strokes.

If no words can be identified from the single or multiple key-stroke codes, the CAT system 13 must back-track. Specifically, the CAT system 13 must back-track to the previous key-stroke code which was preliminarily classified as a complete transcribed word and treat it as if it is only a first part of a word. The CAT system 13 uses the subsequent key-stroke codes thereto to potentially verify or disprove this possibility by locating a multiple key-stroke word. If a multiple key-stroke word cannot be located, the CAT system 13 marks the code as being incapable of exact transcription and continues the rest of the sentence. If a multiple key-stroke word is located, the CAT system 13 continues this forward searching with backtracking strategy until the most complete transcription of the sentence unit can be found. Other strategies such as a backward searching approach are contemplated which might also be used.

FIG. 2 is a diagram representing the association of data fields in a record which is used as a basis for the overall data structure of the cross-reference library 15 according to the present invention. In particular, the CAT system 13 utilizes a linked-list arrangement of two types of data records: a key-stroke code listing (KCL) record 151 and a corresponding text (CT) record 153. Although other types of records are contemplated, these two types of records provide the preferred storage structure for the court reporter's cross-referencing library.

Basically, the CAT system 13 uses records 151 and 153 to associate each individual key-stroke code with as many subsequent key-stroke codes as proves necessary to reconstruct the spoken word. Particularly, the KCL record 151 associates: 1) a listed key-stroke code (LKC) field 155 for storing a specific key-stroke code; 2) a reporter listing counter field 156 for storing a value indicative of the number of times that the CAT system 13 uses the record; 3) a current listing counter field 157 for storing a value indicative of the number of times that the CAT system 13 uses the record in the current case; 4) a common listing counter 158 for storing a value indicative of the number of times that any CAT system, including the CAT system 13, used the record; 5) a first KCL.sub.-- record pointer field 159 for storing a pointer to the next KCL record on this level; 6) a CT record pointer field 161 for storing a pointer to an associated CT record; and 7) a second KCL.sub.-- record pointer field 163 for storing a pointer to a corresponding KCL.sub.-- record at the next listing level down.

Similarly, the CT record 153 associates: 1) a CT string field 165 for storing a string of text; 2) a reporter listing counter field 166 for storing a value indicative of the number of times that the CAT system 13 uses the current string; 3) a current listing counter field 167 for storing a value indicative of the number of times that the CAT system 13 uses the string in the current case; 4) a common listing counter 168 for storing a value indicative of the number of times that any CAT system, including the CAT system 13, used the current string; 5) a CT record homonym pointer field 169 for storing a pointer to another CT record containing a homonym to the contents of the CT string field 165; and 6) a grammatical word type field 170 for storing an indicator of the type(s) of word(s) the contents of the CT string field 165 happens to be (i.e., noun, verb, adverb, noun and verb, etc.).

FIG. 3 provides a detailed illustration representing the overall data structure of the cross-reference library used by the CAT system 13, KCL records 200, 201 and all KCL records (not shown) directly to the right and left of records 200, 201 constitute a first listing level. This first listing level is a linked-list of each beginning key-stroke code of the words held in the cross-reference library 15. The KCL records are "linked" using the first KCL record pointer field 159, i.e.,each pointer field contains-the address in memory where the next KCL record resides.

All words which can be represented by a single key-stroke can be located using a single KCL record at this first level. Words requiring multiple key-strokes must identify the first key-stroke of the word in one of the KCL records at the first listing level, and that identified KCL record should then point via field 163 to a second listing level. For example the KCL records 200 points to a second listing level comprised of KCL records 202, 203, etc. Similarly, a third listing level exists below the KCL record 203 beginning with a KCL record 204, and so on as necessary to reach multiple key-stroke words.

To directly identify exact text replacement using the cross-reference library, the CAT system 13 would first need to know the number of key-strokes required to represent every given word. Because this does not occur, the CAT system 13 must use a searching strategy to identify these numbers.

Because most words can be represented by a single key-stroke, the CAT system 13 initially treats all words as a single key-stroke word. Only after detecting transcription problems with subsequent key-strokes will the CAT system 13 back-track and consider whether the key-stroke might be the first of a multiple key-stroked word. In particular, using the identified KCL records constituting a second listing level, the CAT system 13 must locate a single KCL record containing the second key-stroke in the multiple key-stroke series. The identified KCL record at this second level will point to a subsequent level for providing a subsequent key-stroke in the multiple key-stroke series. This process continues until the last key-stroke is identified.

In addition, each of the KCL records at any listing level may or may not point via the field 161 to associated text. If a single word corresponds to a single key-stroke, the identified KCL record in the first listing level will point to a CT record which contains the text of that word. Similarly, a KCL record in the first listing level will point to a CT record which contains the text of that word. Similarly, a KCL record at the second level identified for a word represented by two key-strokes will point to a CT record containing the actual text of that word. In this way, any key-stroke or series of key-strokes which represent a word can be transcribed if the cross-reference library contains the path to the word formed by the key-stroke(s) of that word, i.e., if the cross-reference library contains the text counterpart.

More particularly, upon receiving the first key-stroke code from a sentence, the code is compared with each key-stroke code stored in each KCL record on the first listing level. For example, if the received code does not match the stored code in the listed key-stroke code field 155 of the KCL record 200, the CAT system 13 uses the contents of the field 159 of KCL record 200 to access the next KCL record, the record 201, for a similar comparison to the code stored therein. In this manner, by stepping through the first listing level, a matching KCL record can be found.

Assuming that the code stored in KCL record 200 does match the first key-stroke code received, the CAT system 13 accesses the associated CT record 205 if present to retrieve readable cross-referenced text. Additionally in this example, the CT record 205 provides the CAT system 13 with a pointer to a homonym stored in a CT record 207. The text located in CT records 205 and 207 possibly provide the desired transcription, but only by transcribing the entire sentence can the CAT system 13 be sure. Often times, the CAT system 13 discards such text in favor of multiple key-stroke text. Particularly, the CAT system 13 uses the KCL record 200 as a back-tracking point. If in transcribing the sentence, the KCL record 200 only proves to be the first of two key-strokes, the CAT system 13 uses the next level pointer 163 in the KCL record 200 to access a second listing level. This second listing level is specifically associated with the KCL record 200 and begins with KCL records 202 and 203 followed by all KCL type records (not shown) to the right of record 203. Any second code received which follows a first code which matches that stored in the KCL record 200 is compared to the codes stored in the KCL records on the second listing level. The KCL record 204 represents yet a third listing level under the key-stroke sequence stored in the record 200 and 203. CT records may or may not be associated with a given KCL record, depending on whether a corresponding word exists for the represented key-stroke code sequence. The KCL record 204 having no CT record associations exemplifies such a situation.

Only a single CT record is generally associated with a single KCL record, such as is shown with KCL record 203 and a CT record 209. Only when homonyms exist will there be multiple CT record association, as illustrated with the KCL record 200 and the CT records 205 and 207. Multiple CT record associations, however, are indirect in that each KCL record can only identify, i.e., point to, a single CT record. Additional CT record "homonyms" are pointed to by the identified CT record.

If homonyms are found, signifying multiple CT record association, two selectable modes are provided to display the homonyms. In an expanded mode, all homonyms are displayed within parenthesis so that selection of the appropriate homonym can be made upon review of the grammatical context. The ordering of the display of homonyms is based on the most to the least probable replacement. For example, the homonyms "their", "there" and "they're would be displayed as (there.vertline.their.vertline.they're). Alternatively, in the selection mode, only the most probable homonym is displayed with an asterisk (*there) indicating the homonym status. By selecting the word, the other homonyms can be selected as a replacement. In addition, to correct such common homonym transcription problems, another level of transcription is contemplated using so called "artificial intelligence" techniques for determining the correct word grammatically in the context of the given sentence.

Upon receiving a first key-stroke code of a sentence from stenographic recorder 11, the CAT system 13 begins a transcription expedition by parsing through a first listing level of the cross-reference library in an attempt to find a matching KCL record. If a matching KCL record is found which has an associated CT record transcription, the CAT system 13 records the match and treats the second (next) code received as the beginning of a new word by parsing the first listing level.

If a matching KCL record found for first code received has no associated CT record, the CAT system 13 treats the second key-stroke code received as the second part of the word by branching to the second listing level pointed to by the matching KCL record (on the first listing level). Note that if properly constructed, there should never be any KCL record which has neither a pointer in field 161 to an associated CT record or a pointer in field 163 to a subsequent level of KCL records. If a match is found at the second listing level with an associated CT record transcription, the CAT system 13 treats the third key-stroke code received as the beginning of a new word by parsing the first listing level, and repeating the cycle.

If after transcribing a series of key-strokes in a sentence, the CAT system 13 encounters a dead end, i.e., an associated CT record can not be identified, back-tracking must occur. The CAT system 13 returns to the last matching KCL record of the previously transcribed word, and continues the transcription process through subsequent listing levels to see if what had been considered an entire word is really only a portion thereof. If a match is found-with an associated CT record transcription, the CT record at that subsequent (deeper) listing level is stored, and the following key-stroke code received is treated as the beginning of a new word, repeating the cycle.

With each successive, unsuccessful parsing round, the previously described transcription process becomes more and more complex with potentially many parallel and nested transcription pathways being considered. If available, the first completely transcribed sentence found is communicated to attorney terminals 15 and 16. Otherwise, the sentence formulation with the greatest number of key-strokes transcribed will be prepared for communication.

If a completely transcribed sentence is unavailable, a second phase of transcription, the phoneme transcription phase, is invoked. In this phase, as illustrated by the phoneme library representation in FIG. 4, each untranscribed key-stroke is cross referenced into a phoneme library 17 to obtain readable phoneme text. By pronouncing-the phoneme text, the lay-person (down-line viewer) will identify the actual word or words which the cross-reference library did not contain.

Specifically, the phoneme library 17 is a double entry list containing a list 223 of key-stroke codes and a corresponding list 229 of phonemes. The listed key-stroke codes 223 provides storage for key-stroke codes which have been directly associated with phoneme text stored in the corresponding phoneme list 229. The CAT system 13 parses through the series of listed key-stroke codes held in the listed.sub.-- keystroke.sub.-- code 223 to locate an untranscribed key-stroke code. If located, the CAT system 13 extracts the associated phoneme text stored in a field of the corresponding phoneme list 229. For example, if the CAT system 13 receives a key-stroke code representing the key-stroke "P H AO EU " for which no counterpart exists in the cross-reference library 15, the CAT system 13 parses the listed.sub.-- key-stroke.sub.-- code entries 223 and locates a specific entry in a field 225. The corresponding readable phoneme text entry "my" in a field 227 of the corresponding.sub.-- phoneme list 229 is then extracted. Instead of transmitting the key-stroke "P H AO EU " down-line, the CAT system 13 substitutes the phoneme text "my" in the current sentence so that down-line pronunciation is simplified.

In a second example, if the court reporter responds to the word "Columbia" by pressing the key-stroke "K H R U PL" followed by the key-stroke "P W A E ", the CAT system 13 first attempts to find exact transcriptions for the two keystrokes using the cross-reference library. If exact transcription text cannot be found, the CAT system 13 parses the phoneme library 17 and finds the phoneme text "KLUMM" and "BEEYA" for the first and second key-strokes, respectively. Because the CAT system 13 cannot be sure if the two key-strokes form a single word or two independent words, a tilde (" ") is placed between the two phoneme text strings for substitution and down-line display. Although only single phoneme text strings are shown, any of the corresponding phonemes in the list 229 may also contain multiple strings of phoneme text separated by a tilde (" ") where needed to provide better pronounceability.

Additionally, upon the court reporter substituting exact text of the word or words spoken for phoneme text that was used during a proceeding (an interactive process carried out by the court reporter after the deposition), the CAT system 13 automatically updates the cross-reference library by adding the key-stroke or key-stroke sequences and associated exact text.

If a key-stroke cannot be found by parsing the listed key-stroke code fields 223 of the phoneme library 220, the key-stroke undergoes an alternate form of pronounceable text substitution using the pronunciation index 25 as illustrated in FIG. 5. In particular, using the key-stroke "P H AO EU" as an example, if the cross-reference and phoneme libraries 15 and 17 yield no positive results, the characters of the key-stroke are cross-referenced into the pronunciation index 25 to identify pronunciation strings for each of the characters singly or in groups. The pronunciation index 25 is modelled from pronunciation keys using long or short vowel and consonant combinations found in any ordinary dictionary. The grouping of characters to be cross-referenced into the index 25 is governed by standard stenographic key-stroke characters and character combinations known to all court reporters. For example, every court reporter knows that a "PH" is equivalent to an "M"; the pronunciation index 25 indicates this. Once pronunciation strings are identified for all of the series of characters in the key-stroke, the strings are combined and substituted for the untranslated key-stroke code and communicated down-line for viewing.

More specifically, in the example illustrated, the CAT system 13 locates the "PH" sub-string in a character series field 253 and extracts the associated pronunciation string "m" from a field 255. Similarly, the CAT system 13 locates "AO" and "EU" in fields 257 and 259, and extracts the associated pronunciation strings "i" and "e" in fields 261 and 263, respectively. Together, these character substitutions form the overall string mie" which is substituted for the untranscribed key-stroke code for down-line viewing.

The overall pronounceable string generated by accessing the pronunciation index 25 can be used to generate a more complete phoneme library 17. For example, if the pronounceable string is "mie", after the deposition the court reporter can choose a more easily pronounceable string such as "my" and place it in the phoneme library 17. Thereafter, if the same key-stroke is ever encountered again, the updated phoneme library 17 will provide text without having to resort to the pronunciation index 25. The CAT system 13 interactively controls this entire process, and also updates the cross-reference library accordingly.

In another embodiment of the present invention illustrated in FIGS. 1 and 6, instead of directly substituting the phoneme text for down-line viewing, the CAT system 13 makes a second attempt at locating exact text using a phoneme dictionary 23. Using the same linked list structure and searching strategy as is detailed in relation to the cross-reference library 15, the CAT system 13 attempts to locate exact text. The only differences between the cross-reference library 15 and the phoneme dictionary 23 can be found in the replacement of the KCL type record 151 (FIG. 2) with a phoneme type record 301 (FIG. 6), and, instead of working with an entire sentence unit, the unit of transcription using the phoneme dictionary 23 would only be the particular sequence of key-strokes having no counterpart text in the cross-reference library 15.

Because the overall data structure, association with CT records, and searching strategy as illustrated by FIGS. 2 and 3 for the cross-reference library are identical to that found in the phoneme dictionary 23, those details regarding the phoneme dictionary 23 need not be revisited. To repeat the description of such structure and operation for the phoneme dictionary 23, the reader is directed to the previous discussions relating to FIGS. 2 and 3 above, keeping in mind the substitution of the phoneme record 301 of the phoneme dictionary 23 with the KCL record 151 of the cross-reference library 15.

Using the phoneme dictionary 23, the CAT system 13 can identify exact text replacements even in circumstances where there are two or more ways to correctly "stroke" a word even though some ways are not registered in the cross-reference library.

In many instances, a key-stroke code or code sequence which cannot be located in the cross-reference library 15 proves to be a result of a typing-error. To help correct for such errors, the pronunciation string produced using the pronunciation index 25 for a word having no exact text replacement in the cross-reference library 15 is compared by the CAT system 13 to all of the words stored in the cross-reference library 15 to find those that offer the most probable match with possible typing errors corrected. The CAT system 13 takes into account both extra key-strokes and misplaced key-strokes while accounting for the layout of the keyboard 31 of the stenographic recorder 11. By visually presenting all of the probable matches to the court reporter, the CAT system 13 provides for easy identification and selection of the mistyped word, and performs automatic word substitution.

This process usually takes place interactively after the deposition.

In the preferred configuration, the CAT system 13 does not directly compare the words stored in the cross-reference library with the pronunciation strings produced using the pronunciation index 25. Instead, the CAT system 13 converts each stored word into a pronunciation string using the pronunciation index 25 so that a like-kind comparison can be made. All such conversion of all of the stored words in the cross-reference library 15 takes place prior to beginning the correction process.

As illustrated in FIGS. 2 and 6, counters are placed in each KCL, phoneme and CT record for several purposes. To provide for optimization of the overall structure involved, each time a specific record is used in a final transcription, the associated counters are incremented. At the end of a deposition, the court reporter can run optimizing routines which sort the records in each linked list based on the count contained in the counters. In this way, for example, optimization could ensure that the most common matching KCL records are encountered first, speeding up the parsing response time. Similarly, the most common homonyms and phoneme text will be used first. Additional benefits attributable to the counters will become more apparent in the descriptions that follow.

FIG. 7 is a perspective view of the down-line terminal 19 of the present invention which more clearly illustrates the use of phoneme text replacement of untranscribed key-strokes. The down-line terminal 19 includes a screen 351 and a keyboard 353. The screen 351 is split into a transcription window 355 and a communication window 357, having a common command line 359. The window 355 displays the transcribed questions and answers received from the CAT System 13. As in the illustrative example, the communication window 357 contains a series of communications which relate to the questions and answers (Q & A's) displayed. More detail regarding the functionality of the communication window 357 and the terminal 350 can be found in the pending parent U.S. application Ser. No. 08/036,488, filed Mar. 24, 1993, by Bennett et al. which is incorporated herein by reference in its entirety.

The transcription window 355 illustrates the display of a fifty-seventh question (Q57) and fifty-eighth question (Q58) which provide a visual indication of the operation of the phoneme library 17. Both of these questions provide an exemplary transcription of a situation where the cross-reference library 15 did not contain exact text for the name "Columbia". Q57 exemplifies the down-line display without the use of the phoneme library 17 wherein the actual key-strokes "KHRUPL" and "PWAE" are placed into the sentence. It should become apparent to those who are not court reporters that these key-strokes cannot be easily understood. In contradistinction, Q58 provides the substitution "KLUMM" and "BEEYA" which when placed together provides most any reader with a readable and understandable pronunciation for the word "Columbia".

FIG. 8 is an exemplary diagram of a foreign language table which is used to cross-reference text of a first language that corresponds to translated words of a first language. Depositions involving participants in two or more languages are also contemplated by the present invention. In a first embodiment, the transcription process operates as previously described by providing a word for word transcription of the spoken question in a first language. Each of these words are then cross-referenced into a second language table to generate a sentence in a second language. Although the use of the second language table can be assigned to each attorney terminal down-line, in a preferred mode, the CAT system 13 performs the task. Once translated, the CAT system 13 transmits the second language sentence down-line to viewing participants who understand that second language. By using additional language tables, the CAT system 13 can provide translations for as many languages as is needed.

If the CAT system 13 performs the translation task, the CAT system 13 also manages the specific down-line routing of first and second language sentences, automatically. To accommodate automatic routing, all participants are required to identify their viewing language and their role in the deposition upon logging in to the system 13. The CAT system 13 stores the identified information in a log-in list. By referencing the log-in list, the CAT system 13 automatically configures itself so as to provide each participant with a translation in their appropriate language.

An exemplary second or foreign language table 27 is illustrated in FIG. 8. Using the language table 27, the CAT system 13 parses the text entries of a first language in a series of identified.sub.-- CT.sub.-- String fields 403 to locate the transcribed text of a word in a first language. Once located, the text of a second language is extracted from a corresponding.sub.-- translation field 405 and placed into a second language sentence. Each transcribed word in the first language undergoes this process so that a second language sentence can be fully constructed. Thereafter, both the first and second language sentences are transmitted down-line to the appropriate participant. For example, the word "beach" located in a field 407 in a first language, English, would be directly substituted by the word "playa" of a second language, Spanish, contained in field 409. Although translations would not always be exact because of word context, the use of a human translator could be minimized. More importantly, the translation provides a check to see if the translator has summarized the translation.

As a supplement to the language table 401, word context analysis might also be implemented to increase the accuracy of the translations. Such analysis might run in parallel with the direct replacement offered by the language table 401 so that a quick and probably less complete translation would be initially displayed, while a slower and probably more complete translation using the grammatical contextual analysis would continue running in the background. Whenever the background translation finishes, the current displayed translation could be directly updated to display the more complete translation. It is contemplated that the background translation using the grammatical context sensitivity might run on an off-site computer, and not within the CAT system 13 if so desired. In such a configuration, the CAT system 13 would provide the quick translation without contextual analysis for display for a fast response time.

If the down-line participants that speak during the deposition all share a common language, the court reporter need only understand the common language and provide a cross-reference library and a phoneme library for that language. Alternate language translations for non-speaking, down-line participants, are provided using as many different foreign language tables as needed, as described above. Routing and language information is retrieved from the log-in list.

If, however, the down-line participants that speak during the deposition share two or more languages, in a preferred embodiment, a multi-lingual court reporter provides a cross-reference library and phoneme library for each of the languages being spoken. Moreover, multilingual court reporters may not be necessary so long as the court reporter has a cross-reference library for all languages being spoken. This is possible because the court reporter is merely creating key-strokes for sounds that they hear. Specifically, all court reporters might be able to select standard foreign language cross-reference libraries from a common remote database as described below in relation to FIG. 10.

Additionally, for each spoken language, language tables providing translation into each of the multiple languages are also necessary. During the translation process, when the multi-lingual court reporter indicates the beginning of a question or answer, or otherwise identifies the speaker, the CAT system 13 automatically selects the appropriate cross-reference and phoneme libraries to transcribe the spoken language, then accesses the appropriate foreign language tables to construct sentence translations. The resulting translations are automatically routed down-line. The CAT system 13 performs the automatic selection of libraries and tables and routing with reference to the entries in the log-in list.

Although the use of multiple phoneme libraries is preferred in a multiple language deposition, they may be completely ignored. Otherwise, when attempting to translate phoneme text into a second language, the phoneme text should be replaced by equivalent phoneme text in the second language. Therefore, pronunciation of the spoken word would sound the same in either language. Because with phoneme text the actual transcription of a word is unknown, a second language translation cannot be found.

Therefore, as a substitute, a pronunciation for each of the participants in the phonemes of their language which sounds like the spoken word in the first language is provided. The pronunciation key index 25 could be similarly adapted.

Unlike the previous features relating to increased transcription percentages and readability in real time, FIG. 9 illustrates features of the present invention for better preparing both the court reporter and the CAT system 13 for depositions or trials. At some time prior to an upcoming deposition or trial, the court reporter using the CAT system 13 updates the cross-reference library 15, and improves their stroking skills using the updated information.

Initially, the CAT system 13 compares the words in the cross-reference library 15 with the words stored in both a case evidence database 455 and a case law database 457. The CAT system 13 places the words which do not exist in the cross-reference library 15 into two new word lists which correspond to new case evidence words and new case law words. The CAT system 13 then presents these words to the court reporter for stroking practice and addition to the cross-reference library 15.

In particular, access to the case evidence database 455 would be provided by an attorney involved in the upcoming deposition or trial. Such access could either occur remotely or locally, depending on the location and accessibility of the attorney's case evidence database 455. For security purposes, the attorney may choose to provide the court reporter with access only to the list of words contained in the case evidence database 455. A simple parsing routine could easily collect such a list.

Instead of generating an entire list of all of the words contained in the case law database 457, the manager of the case law database 457 might instead generate lists of legal terms grouped by category of law and make them available to the CAT system 13. The court reporter then asks the attorneys for the areas of law that are involved, and interactively uses the CAT system 13 to select the appropriate lists of legal words. In the alternative, the CAT system 13 might parse the complaint and answer stored in the case evidence database 457, and, by comparing the names of the categories of law, could automatically select the appropriate word lists.

Once selected, the CAT system 13 automatically compares the case law and case evidence word lists to the words stored in the cross-reference library 15. The CAT system 13 discards the words from the lists which can be found in the cross-reference library, and provides an interactive training and updating session for the court reporter to add the remaining words to the cross-reference library 15. In particular, the CAT system 13 provides an interactive display consisting of a control line 459, an interaction window 461 for managing the list of new words, and a key-stroke practice window 463. The CAT system sequentially displays each remaining word from the new word lists in an interactive fashion in the window 461, prompting the court reporter to provide key-strokes via the stenographic recorder 11. The CAT system 13 displays each key-stroke entered in the window 463. The court reporter can practice stroking the word, and add the preferred key-strokes and the new word automatically to the cross-reference library 15. Upon completing the new lists, the CAT system 13 has prepared the court reporter and the cross-reference library 15 for the upcoming deposition or trial.

Referring to FIG. 10, an alternate embodiment of the training and updating features of the present invention provides training and updating functionality. During a deposition or trial, the CAT system 13 utilizes a locally stored cross-reference library 554 as described in detail with regards to FIG. 1 above. Prior to the deposition or trial however, the court reporter using the CAT system 13 logs-in to a remote database 551 to undergo training and to update the cross-reference library 554. A list of new words from the case evidence database 555 and case law database 557 are generated in a similar manner as described in relation to the embodiment illustrated in FIG. 9, by comparison to the contents of the local cross-reference library 554. The generated lists are then compared to the contents of common cross-reference library 553 and those of the new words that can be located in the common cross-reference library 553 are marked.

As described previously in reference to FIG. 9, the CAT system 13 sequentially displays the new words an interaction window 561. Similarly, the court reporter may practice key-stroking in a key-stroke practice window 563, and control the CAT system 13 via a command line 559. Additionally, however, a new window, a suggested key-stroke window 565 has been added to display the key-strokes stored in the common cross-reference library 553 for any new word which is contained in the common library 553 but not in the local library 554. The window 561 prompts the court reporter to adopt the key-strokes found in the common library 553. The court reporter may practice the suggested key-stroke in the window 563 and either adopt the suggest key-strokes or choose their own, and the local cross-reference library 554 is updated. If the court reporter chooses their own key-strokes, ignoring these suggested, the common cross-reference library 553 is also updated to add an alternate key-stroke pathway to the new word. Therefore, it can be appreciated that multiple key-strokes or sequences of key-strokes may define a path to the same word in the common-reference library 553. If multiple key-stroke pathways exist prior to the court reporters updating and training session for a new word found in the local cross-reference library 554, the CAT system 13 will present all of the key-stroke possibilities in the window 565 so that the court reporter might practice all before choosing one of the possibilities. In the alternative, the court reporter might add all of the key-strokes possibilities for a new word to the local library if training time is available.

If the new words do not exist in either the common or local cross-reference libraries 553 and 554, the court reporter is free to create, practice, and adopt any key-strokes that they may come up with. Adoption of such key-strokes automatically updates the common and local libraries 553 and 554.

Other court reporters using the CAT system 13 of the present invention would similarly log-in to the remote database 551 to both update their own local cross-reference libraries 554 and train. In this way, the cross-reference library 553 could be updated and maintained.

Also to maintain the common cross-reference library 553, each time a court reporter logs-in, the local cross-reference library 554 is compared to the common cross-reference library 553. Based on this comparison, the common cross-reference library 553 is updated to include new words and associated new key-stroke(s) or new key-stroke(s) used for old words from the local cross-reference library 554.

The counters illustrated in KCL and TC records of the local and common cross-reference libraries are also updated upon each log-in session. Referring back to FIG. 2, the current listing counters of the local cross-reference library 554 (FIG. 10) 157 and 167 are added to the common listing counters 158 and 168 in the counterpart records of the common cross-reference library 553 (FIG. 10). The local current listing counters 157 and 167 are then reset. In this way, the common cross-reference library 553 can track both the word usage and the most often used key-strokes representing a given word. During training sessions, the most common key-stroke(s) to a given word can be suggested first, followed by less popular key-stroke(s). Such suggestions urge the court reporters toward a common stroking standard.

In addition, a court reporter using the CAT system 13 may request a standardization session in which the common and local libraries 553 and 554 are compared so that the court reporter can do the common key-strokes to words as indicated by the counters in the common library 553. This process is carried out interactively with training using the windows 561, 563 and 565 as previously described.

Additionally, new court reporters might adopt the entire common cross-reference library 553 or a commonly used portion thereof as their local cross-reference library 554. Similarly, seasoned court reporters after the log-in process would include their own local cross-reference library 554 or portion thereof as their local cross-reference library 553. Afterwards, the CAT system 13 provides interactive training for the new words which are identified by looking for a zero count in any of the reporter listing counters 151 and 153. In other words, if a court reporter has stroked a word at least once before, the KCL and TC records used would all have at least a count of one in the reporter listing counters having identified the new words in the newly adopted local cross-reference library 554, the CAT system 13 provides for training based on the new words constructed from the common listing counters 158 and 168, wherein the most common words are presented first. Over multiple training sessions, the court reporter can implement a full local cross-reference library 554 with associated training to provide a higher percentage of automatic transcription.

During depositions or trials, some exact transcription will not be possible due to stroking errors or new word encounters. The CAT system 13 also provides the three window display described above to correct the transcript. More important than correcting the current transcript, the court reporter must train and update the local cross-reference library 554 (if necessary) to prevent reoccurrences. Specifically, the window 561 is used to display the current transcript which includes each untranscribed word. The window 565 displays the key-stroke or key stroke sequence used for the untranscribed word. The court reporter identifies the correct word viewing these two displays and corrects the transcript using the CAT system 13. Upon correcting the transcript, the corrected text entered is immediately searched against the common and/or local cross-reference 553 and 554. If the word turns out to be new, the court reporter is given an opportunity to practice the key-stroking and adopt a key-stroke or set of key-strokes which they feel best represents the new word. This training and selection occurs in the window 565 using the stenographic recorder 11. Upon selection of the key-strokes, the common and local databases are updated with the new word. If instead the word which fails to be automatically transmitted turns out to be an old word which was mis-stroked, the window 565 merely provides the court reporter with an opportunity to practice the correct key-stroking which is automatically displayed in the window 563.

Although not shown, the remote database 551 may also store a common phoneme library, common phoneme dictionary, common foreign language tables and common pronunciation index. Each of these database local counterparts could be updated and maintained in a similar manner as with the common and local cross-reference libraries 553 and 554.

Furthermore, it is contemplated the use of the phoneme library 17 and phoneme dictionary 23 might not be used so that the pronunciation index 25 would be the only pronounceable text substitution used by the CAT system 13. Similarly, the pronunciation index 25 might not be used.

Although a stenographic recorder 11 is illustrated for use with the present invention, other recording techniques are also contemplated. For example, voice transcription might be implemented. Similarly, a pen-based note-pad computer having Optical Character Recognition (OCR) software which recognized shorthand might also be used. In either of these examples, a counterpart cross-reference library would exist as well as the phoneme library for phoneme text replacement, etc.

Additionally, the data structures and flow described above is merely one embodiment of the many possible ways of carrying out the specific objects of the present invention. Moreover, it is evident that the embodiments of the present invention described hereinabove are merely illustrative and that other modifications and adaptations may be made without departing from the scope of the appended claims.

Claims

1. A reporting system used by at least a lay person, the reporting system comprising:

a screen;

a processing system that attempts to identify exact textual representations for words spoken in real time;

said processing system identifying substitute text for those of the words spoken that the processing system fails to identify exact textual representations; and

said processing system directing display of the exact textual representations and the substitute text on the screen.

2. The reporting system of claim 1 wherein said processing system utilizes a cross-reference library in attempting to identify exact textual representations.

3. The reporting system of claim 2 wherein said processing system utilizes a phoneme library to identify the substitute text.

4. The reporting system of claim 1 wherein said processing system utilizes a phoneme library to identify the substitute text.

5. The reporting system of claim 1 wherein said processing system utilizes a pronunciation index to identify the substitute text.

6. The reporting system of claim 1 that utilizes foreign language translation.

7. A reporting system used by a court reporter and a lay person, the reporting system comprising:

a processing system that generates first textual representations for spoken words in real-time;

said processing system attempting to identify second textual representations for spoken words in real-time;

said processing system identifying third textual representations for spoken words in real-time for those of the words spoken that the processing system fails to identify the second textual representations, said third textual representations being different than the first textual representations corresponding thereto;

a first screen that displays at least the first textual representations; and

a second screen that displays at least the second and the third textual representations.

8. The reporting system of claim 7 wherein the first textual representations comprise stenographic keystrokes.

9. The reporting system of claim 8 wherein the third textual representations comprise pronounceable substitute text.

10. The reporting system of claim 7 wherein the third textual representations comprise pronounceable substitute text.

11. The reporting system of claim 7 wherein said processing system utilizes a cross-reference library in attempting to identify the second textual representations.

12. The reporting system of claim 11 wherein said processing system utilizes a phoneme library to identify the third textual representations.

13. The reporting system of claim 7 wherein said processing system utilizes a phoneme library to identify the third textual representations.

14. The reporting system of claim 7 wherein said processing system utilizes a pronunciation index to identify the third textual representations.