Various embodiments described herein relate generally to the field of identity verification through image processing. More particularly, various embodiments are directed in one exemplary aspect to support automatic enrollment and identity verification upon processing a captured image of a document.
Mobile phone adoption continues to escalate, including ever-growing smart phone adoption and tablet usage. Mobile imaging is a discipline where a consumer takes a picture of a document, and that document is processed, extracting and extending the data contained within it for selected purposes. The convenience of this technique is powerful and is currently driving a desire for this technology throughout Financial Services and other industries.
At the same time, consumers are looking for quicker and easier ways to enroll in new products and services. During a typical enrollment process, consumers must identify themselves, providing common personal and demographic data. However, since mobile devices are increasingly becoming the desired device for such purposes, typing such data into a mobile device can be cumbersome. Additionally, the institution providing the desired service must verify and validate the identity of the customer in order to manage their business risk. Presently, there is no means or mechanism for providing automatic enrollment and identity verification through mobile imaging.
Disclosed herein are systems and methods for automatic enrollment and identity verification based on processing of a captured image of a document. Various embodiments enable, for example, a user to enroll in a particular service by taking a photograph of a particular document (e.g., a driver license) with a mobile device. One or more processes can then extract relevant identifying data from the captured image. The extracted identifying data (e.g., the person's name, gender, date of birth, height, weight, etc.) can then be used to automatically populate various fields of an enrollment application, thereby reducing the amount of information that the user has to manually input into the mobile device in order to complete an enrollment process. In some embodiments, a set of internal and/or external checks can be run against the data to ensure that the data is valid, has been read correctly, and is consistent with other data.
In a first exemplary aspect, a computer readable medium is disclosed. In one embodiment, the computer readable medium contains instructions which, when executed by a computer, perform a process comprising: receiving an image of a document; preprocessing the image of the document in preparation for data extraction; extracting a set of identifying data from the image of the document; and automatically populating fields of an enrollment form based at least in part upon the extracted set of identifying data.
Other features and advantages should become apparent from the following description of the preferred embodiments, taken in conjunction with the accompanying drawings.
Various embodiments disclosed herein are described in detail with reference to the following figures. The drawings are provided for purposes of illustration only and merely depict typical or exemplary embodiments. These drawings are provided to facilitate the reader's understanding and shall not be considered limiting of the breadth, scope, or applicability of the embodiments. It should be noted that for clarity and ease of illustration these drawings are not necessarily made to scale.
The various embodiments mentioned above are described in further detail with reference to the aforementioned figured and the following detailed description of exemplary embodiments.
In one embodiment, the results from the validating and verifying steps are organized into a Mobile Identity Risk Scorecard. This scorecard is a structured information model for presenting the risks associated with a proposed identity to Financial Services or other organizations. The exact contents of the scorecard can vary according to the intended use, but will generally include numeric indicators (0 to 1000), graphical indicators (red-yellow-green) or other patterned indicators which denote aspects of identity risk.
Document Capture
At block 302, an image of a document is captured. According to some embodiments, an application or browser session initiates the capture sequence on a mobile device or tablet. This can be implemented in the form of a library, embedded in a downloaded mobile application, a hybrid application invoked from within a mobile browser, or an automatic capture utility embedded in a mobile application. The capture sequence can guide the user through obtaining a mobile imaging-ready picture of the document. In some embodiments, one or more characteristics can be optimized before image capture, including, without limitation—focus, corner detection, lighting conditions, reflective properties, and closeness. Also, in some embodiments, feedback can provided to the user through an interactive set of visual cues, informing the user, for example, of how “well they are doing.”
In one form of the above, the consumer takes a picture of the front of their Driver's License. In another form, the MRZ line on a passport is read. In a third form, a full identity document is read, such as a government-issued ID or military ID.
Optionally, the user can also provide one or more “hints”—information which can be used to more accurately determine information on the document. For example, the user might provide their last name, which could be used to more accurately determine the location of the name and address on the document.
In some embodiments, the capture process can also read a barcode present on the identity document and extract key information relating to identity. This information can be used to cross-validate the information obtained during the Data Extraction process.
Pre-Processing
At block 304, the mobile image, once captured on the mobile device, can be preprocessed. Preprocessing can include a number of operations, including cropping, deskewing, and/or dewarping the image. Additionally, shadows can be eliminated, lighting issues can be enhanced, and the overall readability of the document image can be improved through one or more mathematical algorithms. The image can also be converted to a bitonal image in preparation for data extraction. Depending on the specific needs of the document type, multiple versions of the binarized image may be needed to handle document-specific readability issues, such as reverse text. In these cases, the preprocessing engine can create multiple bitonal images which can be used in combination during the data extraction process. In addition, a series of image quality and assurance (IQA) test scores can be calculated, indicating the quality of the original image.
Data Extraction
At block 306, relevant data can be extracted from the image of the document. A set of fields known to be available can be determined based on the document type. For example, in an Illinois Driver License, the fields known to be available can include a person's name, address, date of birth, height, weight, document expiration date, and other data.
In some embodiments, individual field confidence scores can also be calculated. For example, in one embodiment, confidence scores can be defined in a range from 0 to 1000, with 1000 representing high technical confidence in the readability of that field, and 0 representing low technical confidence. The confidence scores are calculated using a mathematical formula based on the ability to identify the characters included in each field, including such factors as sharpness. These statistical measures can be used when presenting the data to the user (for example, a low-confidence field can be highlighted, requesting that the user to confirm the data that has been extracted).
The confidence scores would be used by the application leveraging a Mobile Photo Account Opening and Identity Management solution, including applying thresholds to the confidence scores, highlighting those fields with a confidence score below a fixed value (example: highlight fields below 500). If a PDF417 barcode was scanned, the deconstructed string is parsed, identifying each of the relevant fields. A rules engine is applied, to handle a variety of exceptions in the content of the string, including missing fields, concatenated fields, abbreviated fields, and other state-level and local-level deviations. To date, more than 200 variations have been identified, so the use of a rules engine to organize the parsing of the string is a key component of the overall solution
Data Validation
At block 308, the extracted data can be validated using a variety of data validation techniques. As used herein, the term “validation” refers to the evaluation of data using rules and internally-consistent controls available within the mobile imaging process. These techniques can include, without limitation: validation that the information scanned from the PDF417 barcode matches the data obtained during data extraction, if available; validation that the information scanned using the barcode matches the data obtained during data extraction, if available; comparison of date fields to verify date format (This may be used to improve the data (for example, it is not possible to have a 13th month) or to validate the document (for example, exceptions would be flagged, such as expiration dates in the past, birthdates less than 16 years ago, birthdates over 100 years ago, etc.); validation that the expiration date is greater than today; validation that the date of birth is some date earlier than today; validation of data fields to known masks (example: zip code—(either XXXXX or XXXXX-XXXX) in the United States. Exceptions may be able to be corrected, by using a USPS database, or flagged as low-confidence); and validation of data fields to known minimum and maximum field lengths (ex. Validation of state field to defined set of 2-character abbreviations. Exceptions may be able to be corrected, by using a USPS database, or flagged as low-confidence). A myriad of other techniques for validation are possible in accordance with the scope of various embodiments.
Data Verification
At block 310, the extracted data can then be verified using a variety of data verification techniques. As used herein, the term “verification” refers to the evaluation of data using external data sources (110 in
Applications
In one embodiment, a Mobile Photo Account Opening and Identity Management solution may allow a consumer to fund the account once the information from the identity document is used to create a new account. To do this, the consumer would do one of the following: take a picture of a completed check, depositing it in the new account; take a picture of a blank check, to collect the routing and account number from the MICR line, to facilitate an ACH transfer; automatically scan a credit card, using an automatic capture utility, by holding the card in front of the camera of the mobile device, automatically detecting the 15-digit or 16-digit account number on the face of the card. This information is used by the calling application to pre-fill the information needed to complete a credit card funding transaction.
Multiple embodiments of potential applications are now provided herein.
In one embodiment, a system of automatically scanning a credit card, using an automatic capture utility, by holding the card in front of the camera of the mobile device, automatically detecting the 15-digit or 16-digit account number on the face of the card.
A system of Mobile Photo Account Opening and Identity Management, including the following: US Driver's License Capture (front of document), US Driver's License PDF417 scan (on back of document), Preprocessing of image, Data extraction from image, Deconstruction of PDF417 contents using a rules engine, Validation, including comparison of PDF417 contents to extracted data and Funding
A system of Mobile Photo Account Opening and Identity Management, including the following: US Driver's License Capture (front of document), Preprocessing of image, Data extraction from image, Validation, Funding,
A system of Mobile Photo Account Opening and Identity Management, including the following: US Driver's License Capture (front of document), US Driver's License PDF417 scan (on back of document), Preprocessing of image, Data extraction from image, Deconstruction of PDF417 contents using a rules engine, Validation, including comparison of PDF417 contents to extracted data.
A system of Mobile Photo Account Opening and Identity Management, including the following: US Driver's License Capture (front of document), Preprocessing of image, Data extraction from image, Validation.
A system of Mobile Photo Account Opening and Identity Management, including the following: Passport Capture (MRZ contents), Preprocessing of image, Data extraction from MRZ, Validation, Funding.
A system of Mobile Photo Account Opening and Identity Management, including the following: Passport Capture (MRZ contents), Preprocessing of image, Data extraction from MRZ, Validation.
A system of Mobile Photo Account Opening and Identity Management, including the following: Government or other identity document capture, Preprocessing of image, Data extraction, Validation, Funding.
A system of Mobile Photo Account Opening and Identity Management, including the following: Government or other identity document capture, Preprocessing of image, Data extraction, Validation.
Computer-Enabled Embodiment
For the purposes of the embodiments described herein, the term “computer” as used throughout this disclosure may be implemented as any computing device, including a mobile phone or a tablet.
While various embodiments have been described above, it should be understood that they have been presented by way of example only, and not of limitation. The breadth and scope should not be limited by any of the above-described exemplary embodiments. Where this document refers to technologies that would be apparent or known to one of ordinary skill in the art, such technologies encompass those apparent or known to the skilled artisan now or at any time in the future. In addition, the described embodiments are not restricted to the illustrated example architectures or configurations, but the desired features can be implemented using a variety of alternative architectures and configurations. As will become apparent to one of ordinary skill in the art after reading this document, the illustrated embodiments and their various alternatives can be implemented without confinement to the illustrated example. One of ordinary skill in the art would also understand how alternative functional, logical or physical partitioning and configurations could be utilized to implement the desired features of the described embodiments.
Furthermore, although items, elements or components may be described or claimed in the singular, the plural is contemplated to be within the scope thereof unless limitation to the singular is explicitly stated. The presence of broadening words and phrases such as “one or more,” “at least,” “but not limited to” or other like phrases in some instances shall not be read to mean that the narrower case is intended or required in instances where such broadening phrases may be absent.
This application claims priority under 35 U.S.C. § 120 as a continuation in part to U.S. patent application Ser. No. 15/077,801, filed Mar. 22, 2016, and entitled “SYSTEMS AND METHODS FOR CLASSIFYING PAYMENT DOCUMENTS DURING MOBILE IMAGE PROCESSING,” which in turn claims priority as a continuation to U.S. patent application Ser. No. 13/844,748, filed Mar. 15, 2013, and entitled “SYSTEMS AND METHODS FOR CLASSIFYING PAYMENT DOCUMENTS DURING MOBILE IMAGE PROCESSING,” now U.S. Pat. No. 9,292,737, issued on Mar. 22, 2016, which in turn claims priority as a continuation in part to U.S. patent application Ser. No. 12/778,943, filed May 12, 2010, and entitled “MOBILE IMAGE QUALITY ASSURANCE IN MOBILE DOCUMENT IMAGE PROCESSING APPLICATIONS,” now U.S. Pat. No. 8,582,862, issued on Nov. 12, 2013; and as a continuation in part to U.S. patent application Ser. No. 14/217,192, filed Mar. 17, 2014, and entitled “SYSTEMS AND METHODS FOR ENROLLMENT AND IDENTITY MANAGEMENT USING MOBILE IMAGING,” which in turn claims priority under 35 U.S.C. § 119(e) to U.S. Provisional Patent Application No. 61/802,098, filed Mar. 15, 2013, and entitled “SYSTEMS AND METHODS FOR ENROLLMENT AND IDENTITY MANAGEMENT USING MOBILE IMAGING,” each of which is incorporated herein by reference in their entirety.
Number | Name | Date | Kind |
---|---|---|---|
5600732 | Ott et al. | Feb 1997 | A |
5751841 | Leong et al. | May 1998 | A |
5761686 | Bloomberg | Jun 1998 | A |
5966473 | Takahashi et al. | Oct 1999 | A |
6038351 | Rigakos | Mar 2000 | A |
6038553 | Hyde, Jr. | Mar 2000 | A |
6345130 | Dahl | Feb 2002 | B1 |
6621919 | Mennie et al. | Sep 2003 | B2 |
6735341 | Horie et al. | May 2004 | B1 |
6807294 | Yamazaki | Oct 2004 | B2 |
6947610 | Sun | Sep 2005 | B2 |
6985631 | Zhang | Jan 2006 | B2 |
6993205 | Lorie | Jan 2006 | B1 |
7020320 | Filatov | Mar 2006 | B2 |
7245765 | Myers et al. | Jul 2007 | B2 |
7283656 | Blake et al. | Oct 2007 | B2 |
7331523 | Meier et al. | Feb 2008 | B2 |
7376258 | Klein et al. | May 2008 | B2 |
7377425 | Ma et al. | May 2008 | B1 |
7426316 | Vehvilinen | Sep 2008 | B2 |
7433098 | Klein et al. | Oct 2008 | B2 |
7548641 | Gilson et al. | Jun 2009 | B2 |
7558418 | Verma et al. | Jul 2009 | B2 |
7593595 | Heaney, Jr. et al. | Sep 2009 | B2 |
7606741 | King et al. | Oct 2009 | B2 |
7735721 | Ma et al. | Jun 2010 | B1 |
7793835 | Coggeshall | Sep 2010 | B1 |
7869098 | Corso et al. | Jan 2011 | B2 |
7873200 | Oakes, III et al. | Jan 2011 | B1 |
7876949 | Oakes, III et al. | Jan 2011 | B1 |
7949176 | Nepomniachtchi | May 2011 | B2 |
7950698 | Popadic et al. | May 2011 | B2 |
7953268 | Nepomniachtchi | May 2011 | B2 |
7982770 | Kahn et al. | May 2011 | B1 |
7978900 | Nepomniachtchi et al. | Jul 2011 | B2 |
7983468 | Ibikunle et al. | Jul 2011 | B2 |
7986346 | Kaneda et al. | Jul 2011 | B2 |
7996317 | Gurz | Aug 2011 | B1 |
8000514 | Nepomniachtchi et al. | Aug 2011 | B2 |
8023155 | Jiang | Sep 2011 | B2 |
8118216 | Hoch et al. | Feb 2012 | B2 |
8126252 | Abernethy et al. | Feb 2012 | B2 |
8180137 | Faulkner et al. | May 2012 | B2 |
8290237 | Burks et al. | Oct 2012 | B1 |
8300917 | Borgia et al. | Oct 2012 | B2 |
8320657 | Burks et al. | Nov 2012 | B1 |
8339642 | Ono | Dec 2012 | B2 |
8358826 | Medina, III et al. | Jan 2013 | B1 |
8379914 | Nepomniachtchi et al. | Feb 2013 | B2 |
8532419 | Coleman | Sep 2013 | B2 |
8538124 | Harpel et al. | Sep 2013 | B1 |
8542921 | Medina | Sep 2013 | B1 |
8559766 | Tilt et al. | Oct 2013 | B2 |
8959033 | Oakes, III et al. | Feb 2015 | B1 |
9058512 | Medina, III | Jun 2015 | B1 |
9842331 | Nepomniachtchi et al. | Dec 2017 | B2 |
20010014183 | Sansom-Wai et al. | Aug 2001 | A1 |
20010016084 | Pollard et al. | Aug 2001 | A1 |
20010019664 | Pilu | Sep 2001 | A1 |
20020003896 | Yamazaki | Jan 2002 | A1 |
20020012462 | Fujiwara | Jan 2002 | A1 |
20020041717 | Murata et al. | Apr 2002 | A1 |
20020067846 | Foley | Jun 2002 | A1 |
20030072568 | Lin et al. | Apr 2003 | A1 |
20030086615 | Dance et al. | May 2003 | A1 |
20030161523 | Moon et al. | Aug 2003 | A1 |
20030177100 | Filatov | Sep 2003 | A1 |
20040012679 | Fan | Jan 2004 | A1 |
20040037448 | Brundage | Feb 2004 | A1 |
20040109597 | Lugg | Jun 2004 | A1 |
20040213434 | Emerson et al. | Oct 2004 | A1 |
20040218799 | Mastie et al. | Nov 2004 | A1 |
20050011957 | Attia et al. | Jan 2005 | A1 |
20050065893 | Josephson | Mar 2005 | A1 |
20050097046 | Singfield | May 2005 | A1 |
20050100216 | Myers et al. | May 2005 | A1 |
20050143136 | Lev et al. | Jun 2005 | A1 |
20050196071 | Prakash et al. | Sep 2005 | A1 |
20050213805 | Blake et al. | Sep 2005 | A1 |
20050219367 | Kanda et al. | Oct 2005 | A1 |
20050220324 | Klein et al. | Oct 2005 | A1 |
20050242186 | Ohbuchi | Nov 2005 | A1 |
20050261990 | Gocht et al. | Nov 2005 | A1 |
20060008167 | Yu et al. | Jan 2006 | A1 |
20060008267 | Kim | Jan 2006 | A1 |
20060012699 | Miki | Jan 2006 | A1 |
20060045379 | Heaney et al. | Mar 2006 | A1 |
20060071950 | Kurzweil et al. | Apr 2006 | A1 |
20060072822 | Hatzav et al. | Apr 2006 | A1 |
20060140504 | Fujimoto et al. | Jun 2006 | A1 |
20060164682 | Lev | Jul 2006 | A1 |
20060177118 | Ibikunle et al. | Aug 2006 | A1 |
20060182331 | Gilson | Aug 2006 | A1 |
20060210192 | Orhun | Sep 2006 | A1 |
20060221415 | Kawamoto | Oct 2006 | A1 |
20060280354 | Murray | Dec 2006 | A1 |
20060291727 | Bargeron | Dec 2006 | A1 |
20070053574 | Verma et al. | Mar 2007 | A1 |
20070058851 | Quine et al. | Mar 2007 | A1 |
20070076940 | Goodall et al. | Apr 2007 | A1 |
20070081796 | Fredlund et al. | Apr 2007 | A1 |
20070084911 | Crowell | Apr 2007 | A1 |
20070086643 | Spier et al. | Apr 2007 | A1 |
20070140678 | Yost et al. | Jun 2007 | A1 |
20070154071 | Lin et al. | Jul 2007 | A1 |
20070156438 | Popadic et al. | Jul 2007 | A1 |
20070168382 | Tillberg et al. | Jul 2007 | A1 |
20070171288 | Inoue et al. | Jul 2007 | A1 |
20070195174 | Oren | Aug 2007 | A1 |
20070206877 | Wu | Sep 2007 | A1 |
20070211964 | Agam et al. | Sep 2007 | A1 |
20070288382 | Narayanan et al. | Dec 2007 | A1 |
20080031543 | Nakajima et al. | Feb 2008 | A1 |
20080040259 | Snow | Feb 2008 | A1 |
20080062437 | Rizzo | Mar 2008 | A1 |
20080089573 | Mod et al. | Apr 2008 | A1 |
20080174815 | Komaki | Jul 2008 | A1 |
20080183576 | Kim et al. | Jul 2008 | A1 |
20080192129 | Walker et al. | Aug 2008 | A1 |
20080212901 | Castiglia | Sep 2008 | A1 |
20080231714 | Estevez et al. | Sep 2008 | A1 |
20080247629 | Gilder et al. | Oct 2008 | A1 |
20080267510 | Paul et al. | Oct 2008 | A1 |
20090063431 | Erol et al. | Mar 2009 | A1 |
20090092322 | Erol et al. | Apr 2009 | A1 |
20090125510 | Graham | May 2009 | A1 |
20090141962 | Borgia et al. | Jun 2009 | A1 |
20090185241 | Nepomniachtchi | Jul 2009 | A1 |
20090185736 | Nepomniachtchi | Jul 2009 | A1 |
20090185737 | Nepomniachtchi | Jul 2009 | A1 |
20090185738 | Nepomniachtchi | Jul 2009 | A1 |
20090261158 | Lawson | Oct 2009 | A1 |
20100073735 | Hunt et al. | Mar 2010 | A1 |
20100074547 | Yu et al. | Mar 2010 | A1 |
20100080471 | Haas et al. | Apr 2010 | A1 |
20100102119 | Gustin et al. | Apr 2010 | A1 |
20100104171 | Faulkner et al. | Apr 2010 | A1 |
20100114765 | Gustin et al. | May 2010 | A1 |
20100114766 | Gustin et al. | May 2010 | A1 |
20100114771 | Gustin et al. | May 2010 | A1 |
20100114772 | Gustin et al. | May 2010 | A1 |
20100200660 | Moed et al. | Aug 2010 | A1 |
20100208282 | Isaev | Aug 2010 | A1 |
20110013822 | Meek et al. | Jan 2011 | A1 |
20110081051 | Tayal et al. | Apr 2011 | A1 |
20110091092 | Nepomniachtchi et al. | Apr 2011 | A1 |
20110280450 | Nepomniachtchi et al. | Nov 2011 | A1 |
20120030104 | Huff et al. | Feb 2012 | A1 |
20120070062 | Houle et al. | Mar 2012 | A1 |
20120230577 | Calman et al. | Sep 2012 | A1 |
20140040141 | Gauvin et al. | Feb 2014 | A1 |
Number | Date | Country |
---|---|---|
20040076131 | Aug 2004 | KR |
20070115834 | Dec 2007 | KR |
2006075967 | Jul 2006 | WO |
2006136958 | Dec 2006 | WO |
Entry |
---|
PDF417, Wikipedia: the free encyclopedia, Oct. 21, 2008, https://en.wikipedia.org/w/index.php?title=PDF417&oldid=246681430 (Year: 2008). |
International Search Report issued in related International Application No. PCT/US2011/056593 dated May 30, 2012 (3 pages). |
Office Action dated Jan. 9, 2020 for related U.S. Appl. No. 16/397,728 in 56 pages. |
Notice of Allowance for related U.S. Appl. No. 16/742,439, dated Sep. 18, 2020, in 39 pages. |
Number | Date | Country | |
---|---|---|---|
20190188464 A1 | Jun 2019 | US |
Number | Date | Country | |
---|---|---|---|
61802098 | Mar 2013 | US |
Number | Date | Country | |
---|---|---|---|
Parent | 13844748 | Mar 2013 | US |
Child | 15077801 | US |
Number | Date | Country | |
---|---|---|---|
Parent | 15077801 | Mar 2016 | US |
Child | 16282250 | US | |
Parent | 12778943 | May 2010 | US |
Child | 13844748 | US | |
Parent | 16282250 | US | |
Child | 13844748 | US | |
Parent | 14217192 | Mar 2014 | US |
Child | 16282250 | US |