This application is based on and claims priority under 35 USC 119 from Japanese Patent Application No. 2021-022762 filed Feb. 16, 2021.
The present disclosure relates to an information processing apparatus and non-transitory computer readable medium.
Japanese Unexamined Patent Application Publication No. 09-073515 discloses a character recognition apparatus that is free from involving the frequent movement of the gaze of a user in character recognition and operation complication accompanied by the gaze movement.
When information written or entered at a predetermined item is read through optical character recognition (OCR), the accuracy of the OCR is not necessarily 100%. A user typically performs at least one of verification or correction operation on the resulting data. As an amount of workload in the operations by the user increases, labor costs increase. The operations may be completed in a shorter time.
Aspects of non-limiting embodiments of the present disclosure relate to providing a function of assisting a user to verify and/or correct results of reading information written or entered at a predetermined item.
Aspects of certain non-limiting embodiments of the present disclosure address the above advantages and/or other advantages not described above. However, aspects of the non-limiting embodiments are not required to address the advantages described above, and aspects of the non-limiting embodiments of the present disclosure may not address advantages described above.
According to an aspect of the present disclosure, there is provided an information processing apparatus including a processor configured to: display, on one screen, information written or entered at multiple predetermined items and a read result of reading the information from the items such that which piece of the information and the read result is displayed is changeable on a per item basis; and perform a function of assisting a user to at least verify or correct the read result on the one screen.
Exemplary embodiments of the present disclosure will be described in detail based on the following figures, wherein:
An information processing apparatus 20 of a first exemplary embodiment is described below.
Referring to
The CPU 21 executes a variety of programs and controls the elements. The CPU 21 reads a program from the ROM 22 or the memory 24 and executes the read program using the RAM 23 as a working area. In accordance with the program stored on the ROM 22 or the memory 24, the CPU 21 controls the elements and performs a variety of arithmetic operations. According to the first exemplary embodiment, the ROM 22 or the memory 24 stores an information processing program that performs at least one of a first execution process, a second execution process, a third execution process, and a fourth execution process. The information processing program may be pre-installed on the information processing apparatus 20, or appropriately installed on the information processing apparatus 20 by storing the information processing program on a non-volatile recording medium or by delivering the information processing program via a network. The examples of the non-volatile recording medium may include a compact disk read-only memory (CD-ROM), magneto-optical disk, hard-disk drive (HDD), digital versatile disc ROM (DVD-ROM), flash memory, and memory card.
The ROM 22 stores a variety of programs and a variety of data. The RAM 23 works as a working area and temporarily stores a program or data.
The memory 24 includes the HDD, solid-state drive (SSD), or flash memory, and stores the variety of programs including an operating system and the variety of data.
The input unit 25 includes a pointing device, such as a mouse, a variety of buttons, keyboard, microphone, camera or the like and receives a variety of inputs.
The display 26 is, for example, a liquid-crystal display and displays a variety of information. The display 26 may be a touch panel and serve as the input unit 25.
The communication unit 27 is an interface used to communicate with another device. The communication may comply with wired communication standards, such as Ethernet (registered trademark) or fiber distributed data interface (FDDI) or wireless communication standards, such as 4G, 5G, Wi-Fi (registered trademark).
When the information processing program is executed, the information processing apparatus 20 performs a process in accordance with the information processing program using the hardware resources described above.
An optical character recognition (OCR) process performed by the information processing apparatus 20 is described below. The following discussion assumes that a fax function or a scan function of an image forming apparatus (not illustrated) is used, thus multiple documents including text, such as characters and symbols, written or entered at multiple predetermined items are digitized into image data, and the image data is usable through the OCR operation.
A read range within which image data of a document serving as a read target is read is set in the OCR process. In the OCR process, the image data included in the set read range is read and read results are displayed in text.
If a job serving as an execution unit in reading the image data included within the read range is performed, the image data within the read range in each of multiple pages (for example, 10 pages) of a document is read. According to the first exemplary embodiment, multiple read ranges used to read image data in each of multiple predetermined items are set on each document serving as the read target.
Examples of documents used in the first exemplary embodiment are illustrated.
In the documents P1 through P3 as illustrated in
Referring to
The information processing apparatus 20 reads, in the OCR process, image data within each frame set as a read range regardless of whether layouts of documents included in a job are identical, and displays read results as text.
Examples of read results of the image data included in the read range in the OCR process are illustrated herein.
The match level at which the read results in the OCR process match the text written or entered beforehand on the document serving as the read target corresponding to the read results is not necessarily 100%. For this reason, a user may typically verify and/or correct the displayed read results (hereinafter this is referred to as “verification and correction job”). As an amount of the verification and correction job by the user increases, labor costs increase. The verification and correction job may be completed in a shorter period of time.
The information processing apparatus 20 of the first exemplary embodiment performs a first function as one of functions intended to assist a user to perform the verification and correction job on the read results in the OCR process.
In step S10 in
In step S11, the CPU 21 causes document information described below on each item of the documents and the read results of the image data on the document information to be displayed side by side on a verification screen 30 described below of the display 26. The CPU 21 then proceeds to step S12.
In step S12, the CPU 21 receives a switching operation from the user to switch a display status of the read results on the verification screen 30 between a display state with the read results displayed and a hidden state with the read results not displayed. The process thus ends. According to the first exemplary embodiment, multiple methods are available for the user to switch between the display state and the hidden state. The methods are described below.
Display examples on the display 26 of the first exemplary embodiment are described.
The display 26 in
The verification screen 30 in
The information display partition 31 on the verification screen 30 displays the document information that is written or entered at multiple predetermined items of a document serving as a read target.
The result display partition 32 on the verification screen 30 displays the read results of the image data of the document information included in the read range of the document serving as the read target.
According to the first exemplary embodiment, as illustrated in
The display 26 in
With the result display partition 32 deleted from the verification screen 30 on the display 26 in
In the document serving as the read target in
In the typical OCR process, the image data included in the frame set as the read range is read and converted into text data. A portion written or entered outside the frame may be typically not converted into text data. As a result, in a method in which the document information and read result are displayed side by side with the read results remaining displayed (hereinafter referred to as a first comparative arrangement), a portion added outside the frame in the verification and correction job of the user may be overlooked. In the first comparative arrangement, the user may find out the portion added outside the frame after completing the verification and correction job and thus may be obliged to perform the verification and correction job again.
According to the first exemplary embodiment, the CPU 21 causes the document information written or entered at multiple predetermined items of a document serving as a read target and the read results of the image data on the document information included in the read range to be displayed in a manner such that which piece of the document information and read result is displayed on the verification screen 30 is changeable on a per item basis of the document. The CPU 21 performs a first function that causes the document information and read result to be displayed side by side and receives the switching operation from the user to switch between a display state with the read result displayed and a hidden state with the read result not displayed. The operation “written or entered at the items” refers to not only an operation to write or enter data within the frame of the item but also an operation to write or enter data outside the frame corresponding to the item (see
As described above, the first exemplary embodiment provides a function of assisting the user to perform the verification and correction job on the read result. According to the first exemplary embodiment, when the first function is performed, whether to display the read result adjacent to the document information on the verification screen 30 is switched. Also, according to the first exemplary embodiment, the user may verify a portion outside the frame corresponding to the item of the document by switching the read result to the hidden state. In comparison with the first comparative arrangement, overlooking of the document information on the document may be controlled. Furthermore, according to the first exemplary embodiment, since the document information and read result are displayed side by side, the document information and read result are easily compared with each other. In comparison with the first comparative arrangement, the number of operation steps in the verification and correction job of the user may be reduced.
The correction partition 34 displays, in a manner ready to be corrected on the verification screen 30, a text that is the same as the read result displayed in the result display partition 32.
A function enabled to input a correction to the read result on the verification screen 30 is to be provided for the user to perform a correction operation to the read result in the verification and correction job. According to the first exemplary embodiment, the CPU 21 receives a user correction to the read result via the correction partition 34.
The display 26 in
The correction partition 34 in
When the first function is performed as described above, the CPU 21 displays the correction partition 34 in addition to the document information and read result on the verification screen 30. According to the first exemplary embodiment, the text displayed as the document information and read result is not correctable but the text displayed in the correction partition 34 is correctable. The read result may thus be corrected using the correction partition 34. According to the first exemplary embodiment, the user may perform the verification and correction job on the read result while viewing the document information and read result displayed on the same screen.
The correction partition 34 may display beforehand the same text as the read result displayed in the result display partition 32 as illustrated in
When a correction to the read result using the correction partition 34 is received with the verification screen 30 in
The display 26 in
The correction partition 34 in
The correction to the read result using the correction partition 34 may be accepted by left-clicking the mouse on the text displayed in the correction partition 34. Alternatively, the correction to the read result using the correction partition 34 may also be accepted by left-clicking the mouse at any location within or on the frame of the correction partition 34.
The document information displayed in the information display partition 31 and the read result displayed in the result display partitions 32 and 36 in
Referring to
If a correction to one read result among multiple the read results displayed on the verification screen 30 in
The display 26 in
The result display partition 36 in
As described above, if the first function is performed, the CPU 21 causes the multiple read results with the same contents and the document information corresponding to the read results to be displayed side by side on the verification screen 30. If a correction to the one of the read results is received on the displayed verification screen 30, the CPU 21 switches the other read results than the one read result to the hidden state. According to the first exemplary embodiment, the switching operation whether to display a portion of the read results displayed side by side on the verification screen 30 is performed. According to the first exemplary embodiment, since the user may verify an portion outside the frame corresponding to an item of the document serving as the read target by causing the read result other than the correction target to be invisible to the user, overlooking of the document information of the document may be controlled in comparison with the first comparative arrangement.
The display method of the document information displayed in the information display partition 31 of the first exemplary embodiment is described below.
According to the first exemplary embodiment, the document information and read result are displayed side by side with one above the other on the verification screen 30. The manner of displaying the document information and read result is not limited to this vertically stacked arrangement. The document information and read result may be displayed laterally side by side.
According to the first exemplary embodiment, two read results are displayed on the verification screen 30. Alternatively, the number of read results to be displayed on the verification screen 30 may be 3 or more.
A second exemplary embodiment of the information processing apparatus 20 is described. A portion of the second exemplary embodiment common to the first exemplary embodiment is briefly described or not described at all.
The information processing apparatus 20 of the second exemplary embodiment performs a second function as described below as a function of assisting the user to perform the verification and correction job on the read result in the OCR process.
In step S20 in
In step S21, the CPU 21 causes the document information at each item of the documents and the read result of the image data of the document information to be displayed on the verification screen 30 of the display 26. The CPU 21 then proceeds to step S22.
In step S22, the CPU 21 determines whether a change operation to change which piece of the document information and read result is displayed on the verification screen 30 has been performed via the input unit 25. If the CPU 21 determines that the change operation has been performed (yes path in step S22), the CPU 21 proceeds to step S23. On the other hand, if the CPU 21 determines that the change operation has not been performed (no path in step S22), the CPU 21 waits on standby until the change operation is performed. According to the second exemplary embodiment, the change operation is an operation of an enter key or a tab key on the keyboard as the input unit 25.
In step S23, the CPU 21 changes which piece of the document information and read result is displayed on the verification screen 30 and causes the document information and read result to be displayed in the same display region as the region prior to the change operation. The process thus ends.
The display example of the display 26 of the second exemplary embodiment is described below.
A ninth display example of the verification screen 30 is displayed on the display 26 in
According to the second exemplary embodiment, which piece of the document information and read result at each of the items of the document P1 in
When a predetermined icon (not illustrated) used to display the verification screen 30 on the display 26 is selected through left-clicking, the CPU 21 causes the verification screen 30 in
In order to reduce the number of operation steps in the verification and correction job on the read result by the user, the movement of the gaze of the user may be reduced in verifying the read result.
The information display partition 310 in
In the second comparative arrangement, the text “ABC” as the document information and the text “ABC” as the read result are displayed side by side with one text above the other text as illustrated in
In the second comparative arrangement, the gaze of the user vertically moves to verify multiple read results.
In contrast, according to the second exemplary embodiment, the CPU 21 performs the second function that causes the document information and read result to be displayed at the information display partition 31 and the result display partition 32 as the same display regions as display regions prior to the change of the document information and read result on the verification screen 30. According to the second exemplary embodiment, the movement of the gaze of the user in verifying the read result may be reduced in comparison with the arrangement in which the document information and read result are displayed in the display regions that are different from the display regions prior to the change.
A second comparative example to the second exemplary embodiment (hereinafter referred to as a third comparative arrangement) is described below.
Referring to
In the third comparative arrangement, the user has difficulty in focusing concurrently on the text displayed as the document information and the text displayed as the read result. In the third comparative arrangement, for example, the user may focus first on the text displayed as the read result and then moves the gaze to the text displayed as the document information for focusing. In the third comparative arrangement, the gaze movement of the user in comparing the document information with the read result may be expected to increase.
According to the second exemplary embodiment, in contrast, if the second function is performed, the CPU 21 displays the document information and read result in substantially equal size.
According to the second exemplary embodiment, display sizes of the height and width of a single text of the document information and the height of the frame of the information display partition 31 and result display partition 32 are set beforehand and stored on the memory 24. According to the second exemplary embodiment, three types of methods are available as described below to set the document information and read result to substantially equal display sizes.
A first method utilizes a presence range W and the number of texts N of the document information displayed in the information display partition 31. Referring to
A second method utilizes the height of one text of the document information displayed in the information display partition 31. In the second method, as illustrated in
A third method utilizes the height of the frame of the information display partition 31 and result display partition 32. In the third method, for example, as illustrated in
As described above, according to the second exemplary embodiment, the user may focus concurrently on the text displayed as the document information and the text displayed as the read result. According to the second exemplary embodiment, the movement of the gaze of the user in verifying the read result may be smaller than when the document information and read result are different in display size.
The substantially equal display size does not necessarily signify the same display size and a slight degree of difference in display size is acceptable. For example, if the display size of the document information does not match the display size of the predetermined read result, the CPU 21 causes the read result to be displayed at a display size closest to the display size of the document information. In this case, as well, the document information and read result are considered to be displayed in the substantially equal display size.
If the display size of the document information and read result is smaller than a first predetermined size, the CPU 21 causes to be displayed the document information, the read result, and the correction partition 34 that is used to accept a correction to the read result from the user. The CPU 21 causes a correction display having the same text as the read result displayed in the correction partition 34 to be displayed at a second size larger than the first size on the verification screen 30.
As illustrated in
The correction partition 34 in
Since the display size of the document information and read result in
In the arrangement described above, according to the second exemplary embodiment, visibility to the user who verifies the read result may be increased to be higher than when only the text at a size smaller than the first predetermined size is displayed on the verification screen 30.
According to the second exemplary embodiment, the change operation is performed by an operation of the enter key or tab key on the keyboard as the input unit 25. The disclosure is not limited to the operation performed by the enter key or tab key. The change operation may be performed by an operation of a predetermined button displayed on the display 26, a mouse operation, or an operation of a key different from the enter key and tab key on the keyboard.
A third exemplary embodiment of the information processing apparatus 20 is described below. A portion of the third exemplary embodiment common to the other exemplary embodiments is briefly described or not described at all.
The information processing apparatus 20 of the third exemplary embodiment performs a third function as described below to assist the user to perform the verification and correction job on the read result in the OCR process.
In step S30 in
In step S31, the CPU 21 causes the document information at each item of the document and the read result of the image data of the document information to be displayed on the verification screen 30 of the display 26. The CPU 21 then proceeds to step S32.
In step S32, the CPU 21 determines whether a change operation to change which piece of the document information and read result is displayed on the verification screen 30 is performed once via the input unit 25. If the CPU 21 determines that the change operation is performed once (yes path in step S32), the CPU 21 proceeds to step S33. On the other hand, if the CPU 21 determines that the change operation is not performed (no path in step S32), the CPU 21 waits on standby until the change operation is performed once. For example, as in the second exemplary embodiment, the change operation is performed by an operation on the enter key or tab key on the keyboard as the input unit 25.
In step S33, the CPU 21 changes which piece of the document information and read result is displayed and causes the next piece of document information and read result to be displayed on the verification screen 30. The process thus ends.
The display example of the display 26 of the third exemplary embodiment is described below.
The display 26 in
The display 26 in
The display 26 in
The changing of the read result may be sped up in order to reduce the number of operation steps in the verification and correction job of the user on the read result.
In the fourth comparative arrangement, if the user left-clicks the mouse after the user moves a mouse pointer onto the next button 330 with the verification screen 300 in
The mouse operation is to be performed in the fourth comparative arrangement in which the document information and read result displayed are changed in response to the operation of the button displayed on a display 260. It is typically known that a mouse operation takes more time than a keyboard operation.
The accuracy of the OCR process has recently increased and the possibility that a read result matches a text written or entered beforehand on a document serving as a read target, namely, the possibility that the read result is correct has increased. If the read result is correct, a minimum amount of operation may be used to change to the next piece of document information and read result.
According to the third exemplary embodiment, if the CPU 21 receives the operation of the enter key or tab key once as the change operation, the third function to change which piece of the document information and read result is displayed on the verification screen 30 is performed. If the read result is correct, a single keyboard operation completes two job steps including “verifying that the read result is correct” and “changing to the next piece of document information and read result.” According to the third exemplary embodiment, the number of operation steps in the user verification and correction job on the read result may be reduced in comparison with the arrangement in which multiple steps of user operation is to be performed to change which piece of the document information and read result is displayed on the verification screen 30.
As the amount of image data, such as a text, included in the read range is smaller in the OCR process, the possibility of the correct read result is higher. Conversely, as the amount of image data, such as a text, included in the read range is larger in the OCR process, the possibility of the correct read result is lower. For this reason, if the text included in the read result is longer, an erroneous recognition of part of the text may occur and a single keyboard operation alone may have difficulty completing the two steps of job “verifying that the read result is correct” and “changing to the next piece of document information and read result.”
According to the third exemplary embodiment, if the change operation is received once when a portion of the piece of document information and read result at each item of the document serving as the read target is displayed on the verification screen 30, the CPU 21 performs the third function that displays another portion of the piece of document information and read result at each item of the document serving as the read target by changing which portion of the piece of document information and read result is displayed.
The display 26 in
According to the third exemplary embodiment, the text written or entered beforehand in the same frame is segmented into multiple texts and the segmented texts are displayed as the document information and read result in the information display partition 31 and result display partition 32. If the text in the read result is longer in the third exemplary embodiment, a single keyboard operation thus completes the two steps of job “verifying that the read result is correct” and “changing to the next piece of document information and read result.” The number of operation steps in the user verification and correction job on the read result may be reduced in comparison with the arrangement in which the change operation to the document information and read result displayed on the verification screen 30 is performed on a per text basis within the read range.
According to the third exemplary embodiment, the text is segmented by row. The method of segmenting the text is not limited to this method. For example, the text may be segmented by N texts (for example, by three texts), or morpheme, word, phrase, or sentence. The analysis of the morpheme, word, phrase, and sentence may be performed using related art techniques.
According to the third exemplary embodiment, the change operation is performed by the operation of the enter key or tab key on the keyboard as the input unit 25. The disclosure is not limited to this key operation. The change operation may be a mouse operation or an operation of a key other than the enter key and tab key on the keyboard.
A fourth exemplary embodiment of the information processing apparatus 20 is described below. A portion of the fourth exemplary embodiment common to the other exemplary embodiments is briefly described or not described at all.
The information processing apparatus 20 of the fourth exemplary embodiment performs a fourth function to assist the user to perform the verification and correction job on the read result in the OCR process.
In step S40 in
In step S41, the CPU 21 causes the document information at each item of the document and the read result of the image data of the document information to be displayed on the verification screen 30 of the display 26. The CPU 21 then proceeds to step S42.
In step S42, the CPU 21 determines whether an input operation to correct the read result via the result display partition 32 is performed using the input unit 25. Upon determining that the input operation is performed (yes path in step S42), the CPU 21 proceeds to step S43. On the other hand, upon determining that no input operation is performed (no path in step S42), the CPU 21 waits on standby until an input operation is performed. For example, according to the fourth exemplary embodiment, the input operation is an operation on a character key or numerical key on the keyboard as the input unit 25.
In step S43, the CPU 21 switches the display status of the read result on the display 26 to the hidden state and deletes from the verification screen 30 the read result displayed in the result display partition 32. The CPU 21 then proceeds to step S44.
In step S44, the CPU 21 causes input contents to be displayed in the result display partition 32 in response to the input operation performed in step S42. The process thus ends.
The display example on the display 26 according to the fourth exemplary embodiment is described below.
The display 26 in
The text displayed as the document information does not match the text displayed as the read result on the verification screen 30 in
The display 26 in
The display 26 in
The result display partition 32 in
The number of user operations to delete the read result displayed may be reduced in order to reduce the number of operation steps in the user the verification and correction job on the read result.
In an arrangement in which a delete key or backspace key on the keyboard is used to delete the displayed read result, a correction is to be input by operating a character key, numerical key, and the like after deleting the read result. If the read result fails to match the text written or entered beforehand onto the document serving as the read target corresponding to the read result, namely, the read result is in error, the number of operation steps in the verification and correction job increases in response to the deletion operation in comparison with the case in which the read result is correct.
According to the fourth exemplary embodiment, the CPU 21 performs the fourth function. In the fourth function, the CPU 21 sets the read result to the hidden state with no read result displayed if the input operation is received on the verification screen 30 and causes input contents responsive to the received input operation to be displayed in the result display partition 32 serving as a display region to display the read result. If the read result is in error, a single keyboard operation alone completes the two operation steps including “deleting all the read result” and “inputting contents responsive to the input operation.” According to the fourth exemplary embodiment, the number of operation steps in the user verification and correction job on the read result may be reduced in comparison with the arrangement in which the instruction to set the displayed read result to the hidden state is different from the instruction to display the input contents responsive to the input operation.
If the read result is in error and simply deleted, the number of operation steps in the verification and correction job may actually increase. For example, if an erroneous portion of the read result is one character, a correction may be more efficiently performed by using the read result.
According to the fourth exemplary embodiment, if the CPU 21 receives a particular operation different from the input operation, the fourth function is performed to set the read result to the display state on the verification screen 30 and receive from the user who uses the currently displayed read result. According to the fourth exemplary embodiment, the particular operation is performed by left-clicking the mouse as the input unit 25 on the read result displayed in the result display partition 32.
The text displayed as the document information does not match the text displayed as the read result on the verification screen 30 in
If the particular operation is received with the verification screen 30 in
The display 26 in
The display 26 in
In the result display partition 32 in
As described above, according to the fourth exemplary embodiment, the number of operation steps in the user verification and correction job on the read result may be reduced in comparison with the arrangement in which the displayed read result is simply set to the hidden state in response to the operation of the user.
In the arrangement described above, the fourth exemplary embodiment may provide in combination a function of receiving the input with all the read result deleted and a function of receiving the correction using the displayed read result.
According to the fourth exemplary embodiment, the particular operation is the operation of left-clicking the mouse as the input unit 25 on the read result displayed in the result display partition 32. The particular operation is not limited to this operation. For example, the particular operation may be left-clicking the mouse at any location within or outside the result display partition 32 or an operation of any key not used to enter text, such as an arrow key on the keyboard.
According to the embodiments, the verification screen 30 is displayed on the display 26 of the information processing apparatus 20. The terminal on which the verification screen 30 is displayed is not limited to the information processing apparatus 20. For example, the information processing apparatus 20 may perform at least one process selected from the group including the first through fourth execution processes and the verification screen 30 for the process result may be displayed on the display of one of apparatuses other than the information processing apparatus 20. Such apparatuses include a general-purpose computer, such as a server computer or a personal computer, and a mobile terminal, such as a smart phone or a tablet terminal.
In the embodiments above, the term “processor” refers to hardware in a broad sense. Examples of the processor include general processors (e.g., CPU: Central Processing Unit) and dedicated processors (e.g., GPU: Graphics Processing Unit, ASIC: Application Specific Integrated Circuit, FPGA: Field Programmable Gate Array, and programmable logic device).
In the embodiments above, the term “processor” is broad enough to encompass one processor or plural processors in collaboration which are located physically apart from each other but may work cooperatively. The order of operations of the processor is not limited to one described in the embodiments above, and may be changed.
The foregoing description of the exemplary embodiments of the present disclosure has been provided for the purposes of illustration and description. It is not intended to be exhaustive or to limit the disclosure to the precise forms disclosed. Obviously, many modifications and variations will be apparent to practitioners skilled in the art. The embodiments were chosen and described in order to best explain the principles of the disclosure and its practical applications, thereby enabling others skilled in the art to understand the disclosure for various embodiments and with the various modifications as are suited to the particular use contemplated. It is intended that the scope of the disclosure be defined by the following claims and their equivalents.
Number | Date | Country | Kind |
---|---|---|---|
2021-022762 | Feb 2021 | JP | national |