Embodiments are directed to processing documents such as bills or invoices utilizing an extraneous highlight or other color-based indicator applied to the document. Embodiments can be applied to documents as generated or received by recipients, copies or reproductions thereof, and electronic versions thereof (e.g., a bill or invoice that is a .pdf document or included in an electronic mail message).
Embodiments are also directed to processing documents such as bills or invoices utilizing a marking instrument such as a highlighter and a mobile communication device. The type of processing may depend on the type of document processed, and processing may involve storing resulting data to database or taking other action such as initiating an electronic bill payment.
Embodiments are also directed to processing documents such as bills or invoices without having to manually enter bill or invoice data, e.g., into an electronic document such a pdf document, document generated with a word processing program, or into a web page form as part of an electronic payment to a merchant or financial institution.
One embodiment is directed to a computer-implemented bill processing method in which a computing apparatus receives an image of a printed bill or hardcopy thereof. The bill includes at least one manually highlighted section and at least one unhighlighted section. Bill data within the manually highlighted section is determined, and the printed bill is processed for electronic payment based at least in part upon the bill data within the highlighted section.
A further embodiment is directed to a computer-implemented bill processing method in which a computing apparatus receives an image of a printed bill that includes extraneous color data. The computing apparatus determines bill data within a section of the bill comprising the extraneous color data and processes the printed bill for electronic payment based at least in part upon the determined bill data.
Yet another embodiment is directed to method for processing a printed bill and involves a payee, recipient or user manually highlighting at least one section of the printed bill and acquiring an image of the manually highlighted paper bill (e.g., using a camera or other image capture device of a mobile communication or computing device), and electronically paying the printed bill based at least in part upon the at least one manually highlighted section.
Further computer-implemented methods involve extraneous data in the form of a computer generated highlight, e.g., generated in response to a user command or the user manipulating an input device of a computing or mobile communication device such as a mouse or touchscreen tap or drag action to manually select and highlight bill data displayed on a screen. In these embodiments, a computing apparatus receives an image of a printed bill that includes at least one highlighted section generated by the computer, and at least one unhighlighted section, determines bill data within the computer highlighted section, and processes the printed bill for electronic payment based at least in part upon the determined bill data.
Additional embodiments involve both manual and computer generated highlights of printed and electronic versions of a bill. For example, in cases in which one or more fields or sections of a printed or hardcopy bill or invoice were manually highlighted, but after the image was acquired, it was determined that additional highlighting was necessary. The partially highlighted section can be resolved by additional highlighting generated or executed by, or utilizing, a computing apparatus. For example, resolution may also be in the form of detecting a partially highlighted section and requesting the user to repeat or complete highlighting of the partially highlighted section. Detection of a partially highlighted section may be based at least in part upon detection of a field or section that normally is a certain shape (e.g., rectangular) and/or dimension, but detection by the bill processing program that highlighting was only applied to part of the rectangle (resulting in an open-ended rectangle).
Further embodiments are directed to articles of manufacture or computer program products comprising non-transitory computer readable storage media having stored thereupon a sequence of instructions which, when executed by a computer, causes the computer to perform process for processing a bill or invoice and involves manually colored or highlighted and/or computer highlighted sections or fields of a bill or invoice. Embodiments may be embodied within media of a computer or a mobile communication device such as a smartphone or tablet computing device, which may also be operable to generate and/or process colored or highlighted bills or invoice.
Further embodiments are directed to systems configured, programmed or operable to implement methods and execute computer program products for processing images of bills or invoices including manual or computer generated colored highlighted sections, other extraneous color data or other markings. Systems may involve or comprise a computing apparatus such as a computer or mobile communication device of a user, such as a recipient of a bill or person or entity adding highlighting to a bill or invoice.
Systems may involve one or more or all of computing apparatus of a recipient of a bill, source of a bill, intermediary that performs processing of a bill on behalf of a source, and a financial institution utilized for electronic payment of the bill in embodiments in which electronic payment is made through a financial institution instead of directly through a merchant or source of a bill). For example, a system may comprise only a mobile communication device or a computing apparatus configured or operable to execute certain steps or processes involving one or more or all of highlighting, acquiring an image and processing the image according to embodiments, a mobile communication device and a recognition system such as an optical character recognition system, a mobile communication device and a payer computer, or only a payer computer configured or operable to execute certain steps or processes involving one or more of highlighting, acquiring an image and processing the image according to embodiments.
System embodiments may involve one or more components operable for highlighting, imaging and processing of the bill. Systems may involve a computer of a source of a bill such as when a merchant or financial institution highlights a bill for its customer so that the customer can employ embodiments with a mobile communication device or computing having an image capture component without the need to manually highlight the bill.
In a single or multiple embodiments, extraneous data such as color indicators or data, such as an added highlight or modification, is transparent such that the underlying alphanumeric or other data is visible or readable through the highlight mark or modification using a process such as optical character recognition. The color may, for example, be fluorescent pink, blue, green (including lime green), cyan, yellow, orange or purple, and shades or variants thereof. One manner in which such color marks or highlights may be manually applied to the bill is by use of a marking instrument such as a felt-tip highlighter pen or marker.
Embodiments may involve manual highlighting by a payer or bill recipient, by a merchant or source of the bill or by an intermediary that processes bills in behalf of a merchant or source and that is in communication with the payer or bill recipient. Thus, the bill recipient may manually mark, color or highlight a bill with a marking instrument and/or computing apparatus, or the bill recipient may receive a bill already marked by the payee for the convenience of the user.
In a single or multiple embodiments, document processing involves processing of a bill or invoice. Bill or invoice processing involves an image including one or multiple manually colored or highlighted sections or fields, and one or multiple unhighlighted sections or fields.
Colored or highlighted fields may include one or more fields or combinations thereof such as fields required to complete an electronic bill payment, which may involve information such as one or more or all of a payee identification, payee address, account number, payment amount, and payment due date, or other data or combinations thereof needed for bill processing.
In a single or multiple embodiments, the user such as a bill recipient may not know some or all of the sections or fields that are to be highlighted. In these instances, a computing apparatus implementing embodiments may receive a request for sections of the paper bill to be manually highlighted, look up or determine which sections are to be manually highlighted for bill processing (e.g., for a particular type of bill or for a particular merchant), and send a reply back to the user with the requested information.
One manner in which a bill or invoice is processed involves an electronic bill payment or bill pay, which involves communicating the determined bill data together with authorization or instructions to complete the electronic bill payment from a payer's account through a network to a computer of a financial institution hosting the account and that executes the instructions. Electronic bill payment may be made directly to the merchant or indirectly through a financial institution.
In a single or multiple embodiments, determining the bill data involves masking of the image while leaving colored or highlighted sections or fields unmasked. For example, one embodiment involves masking unhighlighted sections of the image, identifying the at least one unmasked and highlighted section within the masked image and extracting bill data from the at least one identified unmasked and highlighted section. One manner in which masking is performed is based on the different colors of a bill's background and alphanumeric data. One method involves determining a first color of a background of an uncolored or unhighlighted section of the paper bill, determining a second color of alpha-numeric data within the paper bill, and generating an electronic mask having a color being based at least in part upon the first color and the second color (e.g., the darker of the two colors, or a third color that is darker than the two colors). The generated electronic mask covers the unhighlighted sections while the highlighted sections remain unmasked. Thus, for example, if a bill background is white while printed text and numbers are black, the electronic mask may be black to black out everything except for the highlighted sections. The colors of the alphanumeric data can be based on one or more types of alphanumeric data, and the background color can be determined based at least in part upon color data of pre-determined sections or groups of pixels within the image (e.g., corners of the image).
In a single or multiple embodiments, determining the bill data involves counting or identification of a pre-determined minimum number of contiguous or adjacent pixels in the image having a color that differs from a background color of the image. The background color can be determined by analyzing colors of pixels or groups of pixels in the corners or other locations of the image. The pixels are read, color data is determined, compared with the background, and a highlighted section or field is identified based at least in part the comparison and determining that a pre-determined minimum number of pixels has a color or color data different than the background color. Bill data is then extracted from the identified section or field.
In a single or multiple embodiments, bill processing involves electronic bill payment utilizing any suitable payment method and payment of any suitable account such as a credit card, merchant or utility bill by transfer of funds between accounts of a financial institution, transfer of funds between financial institutions, transfer of funds from a financial institution to a merchant or other payee.
The foregoing and other aspects of embodiments are described in further detail with reference to the accompanying drawings, wherein:
Embodiments relate to processing documents such as a bill or invoice utilizing an extraneous color data or modification such as an added color or highlight that is not essential or pertinent to the bill or invoice or data thereof. An image of the colored or highlighted bill is acquired, and the colored or highlighted section subsequently analyzed for processing. One embodiment involves processing a highlighted bill or invoice for electronic or on-line bill payment.
For example, referring to
Referring to
The device 220 may be a device of or utilized by the user 225 such as a bill recipient, payer, or accountant 235c that manages the user's finances. Embodiments may involve various computers, computing apparatus, computing components, mobile communication devices 220 including a smartphone, tablet computing device or other computing or mobile device capable of wireless or cellular communications or components thereof. For ease of explanation, reference is made to device 220.
The device 220 may communicate with a computer 230a of a financial institution 235a (“FI” in
In the illustrated embodiments, the device 220 includes an image acquisition element 222 such as a camera or video recording component that typically include one or more optical components and one or more processing components. The image acquisition element 222 may be integrated within the device 220 or a separate device from which an image can be transferred or transmitted. The image acquisition element 222 is operable to acquire images and/or video 223 of the bill 210 embodying bill data 212 and a modification 214 to the bill 210, as described in further detail below.
Embodiments may involve an image 223 acquired with an image acquisition element 222 in the form of a camera, an image or a frame of an image 223 acquired with an image acquisition element 222 in the form of a camera or video component, or a screen shot or crop of an image 223 displayed on a computer. For ease of explanation, reference is made generally to an image 223 acquired with an image acquisition element 222 in the form of a camera of the device 220, but it will be understood that an image 223 (or video) may be acquired with an image acquisition element 222 of or integrated within the device 210 or with an image acquisition element 222 that is different and separate from the device 220.
Referring to
Referring to
Referring to
Referring again to
At 306, and with further reference to
Referring again to
Referring again to
In certain embodiments, the highlight 600 identifies the field(s) 216h while underlying highlighted bill data 212 remains visible or readable by the user 225 or by a system or program such as the recognition program 226. Application of the highlight 600 may be performed by the payee or user 225, by the source of the bill 210 such as a financial institution 235a or merchant 235b, which marks the bill 210 with the highlight 600 for the user's reference, or by another bill processor on behalf of or to facilitate bill processing for the user 225.
Other marking instruments for applying a color or highlight may also be utilized, and a highlighter pen 218 is provided as one example of how embodiments may be implemented. For example, in one embodiment, the marking instrument may also apply a clear or substantially clear layer of ink, or an ink layer that does not have a visible color, but which is otherwise detectable with the image acquisition element 222 or other detection device because of optical properties of the layer and/or capabilities of the image acquisition element 222. Thus, while the layer may not be visible to the human eye, optical properties of the layer may be such that it is visible or detectable by the image acquisition element 222.
In another embodiment, the marking instrument may be a wax-type layer, which may apply a thin colored or clear layer or wax to the bill 210, but through which the underlying bill data 212 can be read or detected as a result of the optical properties of the layer and/or capabilities of the image acquisition element 222. For example, the highlighter pen 218 may be a crayon or crayon-type marking instrument.
In another embodiment, the marking instrument is a label or label generator, and labels can be manually affixed to a section of the bill 210 over the bill data 212 while the underlying bill data 212 can still be read or detected as a result of the optical properties of the label and/or capabilities of the image acquisition element 222. For ease of explanation, reference is made to the highlighter pen 218, and field(s) 216h including respective bill data 212 being marked with respective highlights, as indicated by “highlighted text of interest” 704 as shown in
With continuing reference to
Continuing with reference to
Referring to
Continuing with reference to
Continuing with 806, the bill processing program 221 identifies bill data 212, which may be alphanumeric data within an unhighlighted field, and at 808, determines the color 914 of the alphanumeric bill data 212 within the unhighlighted field(s). If necessary, if bill data 212 includes multiple colors, the bill processing program 221 may also determine or select a darker color or the darkest color of multiple colors 914 of alphanumeric bill data 212.
Continuing with reference to
According to another embodiment, identification of highlighted field(s) 216h involves a pixel color comparison and counting method and identifying highlighted field 216h based at least in part upon identification of a certain number of pixels having different color than a reference or background color.
More particularly,
With continuing reference to
At 1108, the bill processing program 221 compares the color or color data 1212 of the scanned pixels 1200 to the background color 912 and determines whether the scanned pixel color data 1212 is the same or different than the background color 912. Further processing can be implemented as needed for multi-colored bills, and may involve comparisons to a saved copy of a reference bill. If the pixel color data 1212 is the same as the background color 912 or only deviates from the background color 912 by a certain or small amount, then the method 1100 proceeds to
Referring to
With continuing reference to
With continuing reference to
Continuing with 1120, if the count number if at least the pre-determined number, then the method 1100 proceeds to 1122 to identify a pre-determined minimum number of adjacent or contiguous pixels of the highlight color 602 or a color that differs substantially from the background color 912 thereby identifying the location of the highlighted field 216h within the image corresponding to that pre-determined minimum number of pixels 1200 of a different color.
While embodiments involving masking and pixel counting in order to identify a highlighted field have been described, it will be understood that other methods may be utilized, and that both the masking and pixel count methods may be used together sequentially or simultaneously according to other embodiments. Accordingly, masking and pixel counting are provided as examples of how step 312 in
Referring again to
For example, referring again to
In the illustrated example, four highlighted fields 216h are for account number (98765), a bill amount (280.00), address or name of the payee (Company X Home Owner's Association, 123 North Avenue, Los Angeles, Calif. 90017) and due date (Jul. 1, 2012). The extracted bill data 212 is then associated with a record or field type 1400 for the bill processing or electronic bill payment (which may be specified by a financial institution 235a or merchant 235b) to be performed such that the bill processing or electronic bill payment can then be completed using the extracted bill data 212 associated with respective bill processing fields.
While one example of how embodiments may be implemented have been described with reference to four highlighted fields 216h (account number, amount, payee identification/address and due date) are manually highlighted, identified and bill data 212 therein extracted for bill processing, it will be understood that embodiments may involve other numbers of highlighted fields 216h, e.g., one, two, three, five, six, seven and other numbers of highlighted fields 216h. The types and numbers of fields 216h to be highlighted may depend on the types of information required for different types of bill processing, one example of which is on-line or electronic bill payment, which may be made directly to the payee or merchant 235b, to a financial institution 235a, or indirectly to the payee or merchant 235b through a financial institution 235a.
Embodiments may be incorporated into a mobile or web application or other program that is native or downloaded to, executes on or by, or utilized in conjunction with the mobile communication device 220 or other computing apparatus. Further, embodiments may be incorporated into financial management systems or personal finance programs that collect bill data, used for bill processing, or used to interface with a financial institution for bill processing.
Further, bill processing or electronic payment may involve various types of payment transactions and payments made from various accounts including, for example, electronic check, credit card, debit card, electronic payment systems, automated clearing house (ACH), and other payment methods involving accounts such as credit card, savings, checking, money market, loan and other accounts, for payment of bills such as credit card bill, utility bill, phone bills, mortgage and other loan bills, home owners association bills, insurance bills, etc. Thus, it will be understood that embodiments may be utilized for various accounts, forms of payments and bills, and may be utilized in virtually any type of paper bill than can be manually highlighted to allow users to perform bill processing or payments from various locations at various types, and such capabilities are provided by embodiments through the use of a mobile communication device and a highlighter.
Method embodiments may also be embodied in, or readable from, a non-transitory, computer-readable medium or carrier, e.g., one or more of the fixed and/or removable data storage data devices and/or data communications devices connected to a computer. Carriers may be, for example, magnetic storage medium, optical storage medium and magneto-optical storage medium. Examples of carriers include, but are not limited to, a floppy diskette, a memory stick or a flash drive, CD-R, CD-RW, CD-ROM, DVD-R, DVD-RW, or other carrier now known or later developed capable of storing data. The processor 1520 performs steps or executes program instructions 1512 within memory 1510 and/or embodied on the carrier to implement method embodiments.
Although particular embodiments have been shown and described, it should be understood that the above discussion is not intended to limit the scope of these embodiments. While embodiments and variations of the many aspects of the invention have been disclosed and described herein, such disclosure is provided for purposes of explanation and illustration only. Thus, various changes and modifications may be made without departing from the scope of the claims.
For example, while embodiments are described with reference to an individual consumer, embodiments may also be applied to commercial or business consumers such that bills of individuals and enterprises can be paid through the use of a highlighter and a mobile communication device or computer with the ability to receive or acquirer an image of a highlighted bill.
Moreover, while certain embodiments are described with reference to a paper bill that is manually highlighted, embodiments may also involve displaying an electronic version of the bill on a screen and providing for computerized highlighting of the bill by manipulation of an input device of the computer. For example, embodiments may involve displaying the bill utilizing word processing software or browsers that allow for highlight simulation of fields or section of the displayed bill, e.g., using a highlighting program that allows users to attach highlighting to a web page viewed with certain web browsers. Further, while embodiments are described with reference to a document in the form of a financial document (such as a bill, invoice, negotiable instrument), embodiments may be applied to processing of other documents besides financial documents.
Moreover, while manual (or computer) highlighting is described as being performed by the end user (payer or customer), embodiments may also be executed or employed by other parties involved in the bill transaction or processing thereof. For example, a merchant such as utility or wireless communications company may employ manual or computerized highlighting of their bills and send the bills to the payers or customers. In this manner, the end user is not required to manually highlight the bill or scan the bill for computerized highlighting since the bill was already highlighted by the merchant for the customer's reference.
Additionally, embodiments may involve both manual and computer generated highlights of paper and electronic versions of a bill. For example, in cases in which one or more fields or sections of a paper or hardcopy bill or invoice were manually highlighted, but after the image was acquired, it was determined that additional highlighting was necessary. The partially highlighted section can be resolved by additional highlighting generated or executed by a computing apparatus or requested of the user to supplement manual highlighting and for the user take another image with the correct highlighting. Further, resolution may be in the form of requesting the user to repeat or complete highlighting of the partially highlighted section and may be based at least in part upon detection of a field or section that normally is a certain shape (e.g., rectangular) but detection by the bill processing program that highlighting was only applied to part of the rectangle (resulting in an open-ended rectangle).
Further, it will be understood that given the capabilities and user interaction of embodiments, electronic bill payments may be processed or completed utilizing only a highlighter and a mobile communication device, or only a mobile communication device in the event that the bill has already been highlighted, and that embodiments provide for processing of bills without having to manually enter bill or invoice data, e.g., into an electronic form or form in a web page, as part of an electronic payment to a merchant or financial institution.
Moreover, while various embodiments are described with reference to a mobile communication device embodying a bill processing program as well as a recognition program, it will be understood that a recognition program may also be utilized. Additionally, it will be understood that images may be acquired with a mobile communication device, stored to a data store of the mobile communication device or stored to an external data store or transmitted and stored to another computer or server, e.g., in the form of an electronic mail message for subsequent retrieval and processing.
While embodiments are described with reference to certain highlighters and resulting markings or indicators, it will be understood that other marking instruments may also be utilized. Further, while embodiments are described with reference to application of a highlight over bill data to highlight the actual bill data, in other embodiments, the bill processing program can be configured to detect a marking that is not applied over the bill data and then detect an adjacent field, and then extract the bill data from that field. For example, embodiments may be configured such that the user can use a marking instrument (which may or may not apply a transparent or clear marking) to apply a highlight dot to the left of a field of interest, and then the bill processing program is configured to detect that highlight dot, search for the field of interest to the right of the highlight dot, and then select or crop that detected field of interest, and provide the field to the OCR program to extract the bill data within that field. Thus, embodiments may involve highlighting actual bill data or pre-determined locations of the bill for use in locating fields of interest which are then processed according to embodiments described above.
Where methods and steps described above indicate certain events occurring in certain order, those of ordinary skill in the art having the benefit of this disclosure would recognize that the ordering of certain steps may be modified and that such modifications are in accordance with the variations of the invention. Additionally, certain of the steps may be performed concurrently in a parallel process when possible, as well as performed sequentially. For example, it will be understood that highlighting may be performed before an image is acquired (e.g., in embodiments involving manual highlighting), or performed after an image is acquired (e.g., in embodiments involving computer highlighting).
Accordingly, embodiments are intended to exemplify alternatives, modifications, and equivalents that may fall within the scope of the claims.
Number | Name | Date | Kind |
---|---|---|---|
5283829 | Anderson | Feb 1994 | A |
5465206 | Hilt et al. | Nov 1995 | A |
6918083 | Smith | Jul 2005 | B2 |
8391583 | Mennie et al. | Mar 2013 | B1 |
20020001393 | Jones et al. | Jan 2002 | A1 |
20020023055 | Antognini et al. | Feb 2002 | A1 |
20020156827 | Lazar | Oct 2002 | A1 |
20040122766 | Brooks et al. | Jun 2004 | A1 |
20050065893 | Josephson | Mar 2005 | A1 |
20120078732 | Heller | Mar 2012 | A1 |
Entry |
---|
http://en.wikipedia.org/wiki/Fluorescence, last motified Oct. 10, 2014, printed on Oct. 10, 2014 (15 pages). |
http://en.wikipedia.org/wiki/Security—thread, last modified Jul. 4, 2014, printed on Oct. 10, 2014 (1 page). |