Patent Application
20150310269
The hardware can comprise at least one communications/output unit 105, at least one display unit 110, at least one centralized processing unit (CPU) 115, at least one hard disk unit 120, at least one memory unit 125, and at least one input unit 130. The communications/output unit 105 can send results of extraction processing to, for example, a screen, printer, disk, computer and/or application. The display unit 110 can display information. The CPU 115 can interpret and execute instructions from the hardware and/or software components. The hard disk unit 120 can receive information (e.g., documents, data) from CPU 115, memory unit 125, and/or input 130. The memory unit 125 can store information. The input unit 130 can receive information (e.g., a document image or other data) for processing from, for example, a screen, scanner, disk, computer, application, keyboard, mouse, or other human or non-human input device, or any combination thereof.
The software can comprise one or more databases 145, at least one localization module 150, at least one image processing module 155, at least one OCR module 160, at least one document input module 165, at least one document conversion module 170, at least one text processing statistical analysis module 175, at least one document/output post processing module 180, and at least one systems administration module 185. The database 145 can store information. The image processing module 155 can include software which can process images. The OCR module 160 can include software which can generate a textual representation of the image scanned in by the input unit 130 (e.g., scanner). It should be noted that multiple OCR modules 160 can be utilized, in one embodiment. The document input module 165 can include software which can work with preprocessed documents (e.g., preprocessed in system 100 or elsewhere) to obtain information (e.g., used for training). Document representation (e.g., images and/or OCR text) can be sent to the localization module 150. The document conversion module 170 can include software which can transform a document from one form to another (e.g., from Word to PDF). A text processing statistical analysis module 175 can include software which can provide statistical analysis of the generated text to pre-process the textual information. For example, information such as the frequency of words, etc. can be provided. A document/output post processing module 180 can include software which can prepare a result document in a particular form (e.g., a format requested by a user). It can also send result information to an external or internal application for additional formatting and processing. The system administration module 185 can include software which allows an administrator to manage the software and hardware. In one embodiment, individual modules can be implemented as software modules that can be connected (via their specific interface) and their output can be routed to modules desired for further processing. All described modules can run on one or many CPUs, virtual machines; mainframes, or shells within the described information processing infrastructure, such as CPU 115. Database 145 can be stored on hard disk drive unit 120.
The localization module 150 can utilize at least one document classifier, at least one dynamic variance network (DVN), at least one dynamic sensory map (DSM), or at least one fuzzy format engine, or any combination thereof. A document classifier can be used to classify a document using, for example, a class identifier (e.g., invoice, remittance statement, bill of lading, letter, e-mail; or by sender; vendor, or receiver identification). The document classifier can help narrow down the documents that need to be reviewed or to be taken into account for creating the learn sets. The document classifier can also help identify which scoring applications (e.g., DVNs, DSMs, and/or fuzzy format engines) should be used when reviewing new documents. For example, if the document classifier identifies a new document as an invoice from company ABC, this information can be used to pull information learned by the DVN, DSM, and fuzzy format engine from other invoices from company ABC. This learned information can then be applied to the new document in an efficient manner, as the learned information may be much more relevant than, for example, information learned from invoices from company BCD. The document classifier is described in more detail with respect to
As mentioned above, the localization module 150 can include numerous scoring applications, such as, but not limited to, DVNs, DSMs, or fuzzy format engines, or any combination thereof. DVNs can be used for determining possible target values by using references on a document or piece of a document to determine possible locations for any targets. A score can be given for each possible target value identified by the DVN. DVNs are discussed further below with respect to
Information generated by the localization module 150 can be sent to the databases(s) 145 or to external inputs (e.g., input unit 130, communication network 101, hard disk unit 120, and administration module 185). The output or part of the output of the localization module 150 can be stored, presented or used as input parameters in various components (e.g., communications/output unit 105, display unit 110, hard disk unit 120, memory unit 125, communication network 101, conversion module 170, database(s) 145, OCR module 160, statistical analysis module 175) either using or not using the post-processing module 180. Such a feedback system can allow for iterative refinement.
As indicated above, the document classifier can be used to classify a document using, for example, a class identifier (e.g., invoice, remittance statement, bill of lading, letter, e-mail; or by sender, vendor, or receiver identification). The document classifier can operate based on text in a document. The document classifier can also be based on positional information about text in a document. Details relating to how a document classifier can classify a document using any combination of textual and/or positional information about text from the document is explained in more detail in the following patents, which are herein incorporated by reference: U.S. Pat. No. 8,276,067; U.S. Pat. No. 6,976,207; and U.S. Pat. No. 7,509,578 (all entitled “Classification Method and Apparatus”).
Once the text information and text positional information is obtained for at least one training document, this information can be used to return an appropriate class identifier for a new document. (It should also be noted that a human or other application can provide this information.) For example, if invoices issued by company ABC are to be reviewed, certain text (e.g., “ABC”) or text positional information (e.g., where “ABC” was found to be located on training documents using, for example, DVNs or DSMs) found on the training set of documents can be searched on new documents to help determine if the new document is an invoice issued by company ABC. Documents identified as invoices issued by company ABC can be reviewed with company ABC-specific DVNs, DSMs and/or fuzzy searching machines.
It should be noted that the document classification search can be performed in a fuzzy manner. For example, punctuation or separation characters, as well as leading or lagging alphabetical characters and leading or lagging zeroes can be ignored. Thus, for example, “123-45”, “1/2345”, “0012345”, “INR1234/5” can be found if a fuzzy search is done for the string “12345”. Those of ordinary skill in the art will see that many types of known fuzzy searching applications can be used to perform the document classification search. Other examples of fuzzy representations and their respective classification are described in further detail in the following patents, which are herein incorporated by reference: U.S. Pat. No. 7,908,430; U.S. Pat. No. 6,983,345; and U.S. Pat. No. 7,433,997 (all entitled “Associative Memory”).
As explained above, the document classifier can help narrow down the documents that need to be reviewed. The document classifier can also help identify which scoring applications (e.g., DVNs, DSMs, and/or fuzzy format engines) should be used when reviewing new documents. This learned information from the DVNs, DSMs, and/or fuzzy format engines can then be applied to the new document in an efficient manner, as the learned information may be much more relevant than, for example, information learned from invoices from company BCD.
It should be noted that the above method 200 can provide increased redundancy and accuracy. Because every reference is a potential basis for target localization, there can be hundreds of reference anchors per page for each target. Thus, even for torn pages, where all classical keywords are missing, a target localization can be found.
In addition, it should be noted that a reference with a typo or misrecognized by an OCR engine at a particular position can automatically be used as an anchor based on where the reference is found. Thus, in some embodiments, there is no need to specify traditional keywords or apply any limitation to anchor references. In this way, strict and/or fuzzy matching can be utilized to match any similar reference to at least one reference in a new document.
Furthermore, the following characteristics of the reference can be taken into account when matching: font; font size; style; or any combination thereof. Additionally, the reference can be: merged with at least one other reference; and/or split into at least two references.
In 315, variance filtering can be performed by selecting similar reference vectors. The variance filtering can compare the references and the reference vectors for all documents in the learn set, compare the type of references, and keep similar reference vectors. Similar reference vectors can be similar in terms of position, content similar, and/or type similar for the reference. A reference can be positionally similar when the reference is usually found in one or more particular places on a page. Content similarity relates to references having the same type of content (e.g., when the references are all the same word or similar words). Type similarity relates to the reference usually being a particular type (e.g., a numerical value, a word, a keyword, a font type, etc.). Similarity types can be tied to other similarity types (e.g., when the references are all content similar, the same word or similar words, but only when the references are type similar as well (e.g., all of the type “date”)).
It should be noted that the reference consistency tests can be fuzzy. An example of fuzzy testing with a positionally similar reference is when everything within a defined x and y coordinate space is utilized, and the space parameters are able to be adjusted. An example of content consistency is determined by comparing words. Thus, “Swine-Flu”, “swineflu”, “Schweinegrippe” and “H1N1” can be assumed to be identical for a special kind of fuzzy comparison. “Invoice Number”, “Inv0!ce No.” and “invoiceNr” can be assumed to be identical for another kind of fuzzy comparison. An example of type similar fuzzy testing is when more than one type can be used (e.g., both “number” type and “number/letter” type for a date).
In 320, the similar reference filters are used to create the DVN. For example,
It should be noted that the content, position and type of reference can be used to filter reference vectors and construct the DVN, especially when only totally similar reference vectors are used.
Note that the image of a reference can be blurry in some situations because identical content with small positional changes can render the words readable but blurry. When the content is not the same (e.g., numbers for the invoice date, invoice number, order date and order number), the content may be unreadable in the overlay. As shown in
For example, using the example of 710, 805 and 810 of
In 410, all of the reference vectors that relate to the “keyword” references can be used to point towards the target. In 415, the integrating of the pointer information from all of the reference vectors and the reference keywords can then used to localize (determine) the target.
For example, in
Once possible positions for the locality of any targets are found using the DVNs, possible values for the targets can be found (e.g., Jan. 10, 2009 as the value for the target “invoice date”). Each possible value for the target can be given a score. The score can be determined by the ratio of the reference vectors hitting the target against the reference vectors not pointing to the target. Additionally, the fuzzy edit distance between the learned reference(s) (e.g., text) and the reference(s) used for localization can be integrated as a weight. For example, if all possible reference words on a document could be found exactly at the same relative position from the target as the ones stored in the learn set, the highest score can be returned. Additional references not contained in the learn set, or references with no vectors pointing towards the respective target can reduce the score.
It should be noted that DVNs can be used for many additional tasks, including, but not limited to: the addition of reference vectors, reference correction, document classification, page separation, recognition of document modification, document summarization, or document compression, or any combination thereof. These tasks are explained in more detail below.
Addition and/or removal of Reference Vectors. DVNs can be dynamically adapted after target localization. When at least one reference vector is learned and used to localize a target, all other possible reference vectors can be created and dynamically added to the DVN learned in 210 of
Reference Correction. Reference vectors can be used for reference correction. An example is illustrated in
Another example of reference vectors being used for reference correction is when the reference vectors are used to locate a target of, for example, a specific type. Additional information present can then be used to correct a potentially corrupted target. For example, if the reference vectors point towards the reference “29 Sep. 1009”, and this reference is known to be a date field target from a currently retrieved document, then a correction of that target to “29 Sep. 2009” is possible. To do this correction, the high similarity between “September” and “September” is used in a fuzzy content comparison and additional information about the entry being a date can be used to correct the year to a (configurable) time period that seems to be valid. It should also be noted that, if a date field target is clearly located, then the reference vectors can be followed back to the potential anchor references. It for example, the position & information for such an anchor reference perfectly fits, then the actual reference present there, but not fitting to the anchor reference present in the learned DVN could be replaced by the one from the learned DVN. For example, if the invoice number field target was located, the surrounding classical keyword which is corrupted and shows “Inv0!ce Number” could be replaced by the one stored for this position from the learned DVN. Thus, after that correction, “Invoice Number”, could be read at that position.
Document Classification. As explained earlier with respect to
Page Separation. Positional information regarding anchor references can also be used for page separation. In a stack of different documents (e.g., single documents, multi-page documents), the changes in the DVNs positional information (also referred to as “quality of fit” can provide information about the starting page of a new document. This method can be used to, for example, repackage piles of documents into single documents.
Recognition of Document Modification. DVNs can also be used in a reverse manner (e.g., after having located a target, looking up how well the anchor words on the present document fit to the learned anchor words of the DVN), to recognize a document modification. For example, in
DVN for Multiple Targets. DVNs can also be used to localize multiple targets on a document in some example embodiments. In this regard, a multi-target area that may, for example, contain multiple words is trained. Examples of multiple word targets include, but are not limited to, addresses, terms and conditions, phrases, full names, and content of free-form fields in forms. The relative position of a bounding box or bounding area (formed for example by polygon, splines or other shapes forming an circumference around at least one target, typically comprising all desired targets) is then anchored to the document with the method described above for localization of single targets. In some example embodiments, the bounding box or bounding area may be extend over multiple rows. The two dimensional statistics of the spread, size, probability density of target localizations within an area covered of the bounding area are trained based upon the statistics of the learn-set documents presented to the system. There, positional information, variance across positions, frequency of targets, their abstracted content (e.g., via a character to format transformation) play a role in how to determine the area which is learned. For target localization purposes, the statistically enriched bounding area is positioned on the document by the anchoring method described above for the DVN target localization. Then potential targets desired for extraction within this area are selected based on the learned statistics for the bounding area and returned as a multi-target localization. Such a return value typically comprises more than one word as a result.
Document Summarization. DVNs can also be used to automatically summarize document content. This process is illustrated in
Document Compression. DVNs can also be used for compression of a document or set of documents. In
Fuzzy format engines can collect a list of fuzzy formats for at least one target from training documents. During the extraction phase, the fuzzy format engine can calculate a score for the matching of the learned formats to the potential target. For example, given the target value “102.65$” for an amount type target, the fuzzy format engine could learn from the training documents that, in the representation “ddd.ddR”, d represents digit and R represents a currency signal. If the fuzzy format engine then finds a string “876.27$”, then this string can be determined to be a potential target value with a very high score (e.g., 10). However, if the string “1872,12$” is found, the score could be reduced by one for the additional digit, and reduced by another one for the comma instead of the period, for a score of 8. As another example, a fuzzy format engine could learn that “INVNR-10234” could be represented as “CCCC-ddddd”, where C represents capital characters and d represents digits. Those of ordinary skill will see that many type of fuzzy format engines can be used, and there can also be many types of scoring utilized. Examples of other possible scoring systems are, for example: the different handling of missing or additional characters and digits (e.g., having a 0.125 score penalty per missing or additional character vs. a 0.25 penalty for a missing or additional digit); character string similarity measures that can be obtained as described in the following patent patents, which are herein incorporated by reference: U.S. Pat. No. 7,908,430; U.S. Pat. No. 6,983,345; and U.S. Pat. No. 7,433,997 (all entitled “Associative Memory”).
While various embodiments of the present invention have been described above, it should be understood that they have been presented by way of example, and not limitation. It will be apparent to persons skilled in the relevant art(s) that various changes in form and detail can be made therein without departing from the spirit and scope of the present invention. Thus, the present invention should not be limited by any of the above-described exemplary embodiments.
In addition, it should be understood that the figures described above, which highlight the functionality and advantages of the present invention, are presented for example purposes only. The architecture of the present invention is sufficiently flexible and configurable, such that it may be utilized in ways other than that shown in the figures.
Further, the purpose of the Abstract of the Disclosure is to enable the U.S. Patent and Trademark Office and the public generally, and especially the scientists, engineers and practitioners in the art who are not familiar with patent or legal terms or phraseology, to determine quickly from a cursory inspection the nature and essence of the technical disclosure of the application. The Abstract of the Disclosure is not intended to be limiting as to the scope of the present invention in any way.
Finally, it is the applicant's intent that only claims that include the express language “means for” or “step for” be interpreted under 35 U.S.C. 112, paragraph 6. Claims that do not expressly include the phrase “means for” or “step for” are not to be interpreted under 35 U.S.C. 112, paragraph 6.
This patent application claims priority and benefit as a continuation-in-part of U.S. patent application Ser. No. 12/610,915, filed Nov. 2, 2009, entitled “System and Method of Using Dynamic Variance Networks.” The entirety of the application is incorporated herein by reference.
| Number | Date | Country | |
|---|---|---|---|
| Parent | 12610915 | Nov 2009 | US |
| Child | 14635077 | US |
