Systems and methods for assessing standards for mobile image quality

Information

  • Patent Grant
  • 11157731
  • Patent Number
    11,157,731
  • Date Filed
    Monday, July 22, 2019
    4 years ago
  • Date Issued
    Tuesday, October 26, 2021
    2 years ago
  • Inventors
  • Original Assignees
  • Examiners
    • Rudolph; Vincent
    • Choi; Timothy
    Agents
    • Procopio, Cory, Hargreaves & Savitch LLP
Abstract
Methods and systems are provided for defining and determining a formal and verifiable mobile document image quality and usability (MDIQU) standard, or Standard for short. The Standard ensures that a mobile image can be used in an appropriate mobile document processing application, for example an application for mobile check deposit. In order to quantify the usability, the Standard establishes 5 quality and usability grades. A mobile image capture device can capture images. A mobile device can receive information associated with one or more image quality assurance (IQA) criteria; evaluating the images to select an image satisfying an image quality criteria based on the received information; and in response to the image satisfying the image quality score, sending the selected image to determine a set of image quality assurance (IQA) scores.
Description
BACKGROUND
1. Field of the Invention

The embodiments described herein relate to processing images captured using a mobile device, and more particularly to developing standards for processing the images and assessing an image processing system to determine whether it meets the standards.


2. Related Art

The use of mobile devices to capture images is ubiquitous in modern culture. Aside from taking pictures of people, places and events, users are utilizing the cameras on their devices for many different purposes. One of those purposes is capturing images of content that the user later wants to review, such as a description of a product, a document, license plate, etc.


Financial institutions and other businesses have become increasingly interested in electronic processing of checks and other financial documents in order to expedite processing of these documents. Some techniques allow users to scan a copy of the document using a scanner or copier to create an electronic copy of the document that can be digitally processed to extract content. This provides added convenience over the use of a hardcopy original, which would otherwise need to be physically sent or presented to the recipient for processing. For example, some banks can process digital images of checks and extract check information from the image needed to process the check for payment and deposit without requiring that the physical check be routed throughout the bank for processing.


Mobile devices that incorporate cameras are ubiquitous and may also be useful to capture images of financial documents for mobile processing of financial information. The mobile device may be connected with a financial institution or business through a mobile network connection. However, the process of capturing and uploading images of financial documents using the mobile device is often prone to error, producing images of poor quality which cannot be used to extract data. The user is often unaware of whether the captured document image is sufficient and capable for processing by a business or financial institution.


Attempts have been made to improve the quality of mobile images of financial documents to improve the accuracy of information extracted therefrom. There are numerous ways in which an image can be improved for extracting its content, some of which are implemented individually and some of which are implemented together. However, it is difficult to determine which methods are the best at improving image quality and content extraction. Of the methods often used, it is even more difficult to select a threshold of that particular method which will provide an accurate capture in as little time as possible. Finally, determining whether an image processing system is capable of performing adequate image capture, processing and content extraction has not been explored.


Therefore, there is a need for identifying image processing techniques which will provide optimal image correction and accurate content extraction.


SUMMARY

Embodiments described herein provide methods for defining and determining a formal and verifiable mobile document image quality and usability (MDIQU) standard, or Standard for short. The goal of this Standard is to ensure that a mobile image can be used in an appropriate mobile document processing application, for example an application for mobile check deposit, mobile bill pay, mobile balance transfer or mobile insurance submission and application. In order to quantify the usability, the Standard establishes 5 quality and usability grades: the higher grade images will tend to produce higher accuracy results in the related “mobile” application. A mobile image is first tested to determine if the quality is sufficient to obtain content from the image by performing multiple different image quality assessment tests. If the image quality is sufficient, one or more document usability computations are made to determine if the document or content in the image is usable by a particular application. A ranking is then assigned to the image based on the results of the tests which is indicative of the quality and usability of the image.


Systems and methods are provided for developing standards of image processing for mobile image capture and assessing whether mobile processing engines meet the standards. A mobile processing engine (MDE) is evaluated to determine if it can perform one or more technical capabilities for improving the quality of and extracting content from an image of a financial document (such as a check). A verification process then begins, where the MDE performs the image quality enhancements and text extraction steps on sets of images from a test deck of good and bad images of financial documents with known content. The results of the processing of the test deck are then evaluated by comparing confidence levels with thresholds to determine if each set of images should be accepted or rejected. Further analysis determines whether any of the sets of images were falsely accepted or rejected, and an overall error rate is computed. The overall error rate is then compared with minimum accuracy criteria, and if the criteria are met, the MDE meets the standard for mobile deposit


The goal of this Standard is to ensure that a mobile image can be used in an appropriate mobile document processing application, primarily Mobile Deposit [1], Mobile Bill Pay [2], Mobile Balance Transfer [3] and Mobile Insurance [4].


Other features and advantages should become apparent from the following description of the preferred embodiments, taken in conjunction with the accompanying drawings.





BRIEF DESCRIPTION OF THE DRAWINGS

Various embodiments disclosed herein are described in detail with reference to the following figures. The drawings are provided for purposes of illustration only and merely depict typical or exemplary embodiments. These drawings are provided to facilitate the reader's understanding and shall not be considered limiting of the breadth, scope, or applicability of the embodiments. It should be noted that for clarity and ease of illustration these drawings are not necessarily made to scale.



FIG. 1 illustrates a method of assessing, grading and ranking an image to determine an image quality and an image usability, according to one embodiment of the invention.



FIG. 2 illustrates a system for assessing, grading and ranking the image utilizing a mobile image capture device and image server, according to one embodiment of the invention.



FIG. 3 is an image illustrating an example of an out-of-focus (OOF) image identified during an OOF image quality test, according to one embodiment of the invention.



FIGS. 4A and 4B are images of plain skew and view skew images, respectively, identified during a skew image quality test, according to one embodiment of the invention.



FIGS. 5A and 5B are images of a low contrast image and poor darkness image, respectively, identified during a low internal contrast image quality test, according to one embodiment of the invention.



FIG. 6A is an image of a document where the corners are cut off, as would be identified during a cut corner image quality test, according to one embodiment of the invention.



FIG. 6B is an image of a warped document which would be identified during a warpage image quality test, according to one embodiment of the invention.



FIG. 7 is a block diagram that illustrates an embodiment of a computer/server system upon which an embodiment of the inventive methodology may be implemented





The various embodiments mentioned above are described in further detail with reference to the aforementioned figured and the following detailed description of exemplary embodiments.


DETAILED DESCRIPTION

I. Summary of Mobile Deposit Standards


Embodiments described herein define formal and verifiable mobile document image quality and usability (MDIQU) standard, of Standard for short. The goal of this Standard is to ensure that a mobile image can be used in an appropriate mobile document processing application, primarily mobile check deposit, mobile bill pay, mobile balance transfer or mobile insurance submission and application, although this list is only illustrative of the types of images and documents which may take advantage of the methods.


In order to quantify the usability, the Standard establishes 5 quality and usability grades: the higher grade images will tend to produce higher accuracy results in the related “mobile” application. The standards for various mobile document processing applications may use the ranking system established by this invention in order to define the various accuracy levels. For example, a mobile deposit standard may require MICR accuracy of 99% on mobile images of checks which are Grade 1 (the highest) according to this Standard, 97% on Grade 2 mobile images etc. The lowest grade (or 2 lowest grades, depending on application preferences) defines unusable images for which no meaningful prediction of accuracy could be made.


Given an application (e.g. Mobile Deposit) and a mobile image, MDIQU answers two basic questions:

    • 1. Does the mobile image have good image quality?
    • 2. Does the image contain a document which is valid for the particular application it's intended for (a valid US check in case of MD)?


Moreover, if the image quality is characterized as bad, MDIQU will detect a particular quality defect.


The answer on the first question (Image Quality) doesn't depend on the type of document or application in question. All that MDIQU is supposed to verify is that a human would easily read the relevant data off the mobile image. For example, a good quality mobile image of a check must contain the entire check (all 4 corners), be high contrast/sharp with all the characters being legible. The same will apply to mobile images of a bills, driver's licenses etc.


The answer on the second question (Document Validity) is supposed to ensure that the document within the image is indeed a check, a bill, a driver's license etc. Therefore, the validity factor is defined for each application independently.


II. MDIQU System and Workflow



FIG. 1 illustrates a flowchart of a workflow for assessing mobile image quality standards, according to one embodiment of the invention. The steps may include:



100—accept mobile image under examination and an expected document type



200—compute individual IQA scores



300—output individual IQA scores



400—compute compound IQA scores (see Section 4)



450—output compound IQA scores



600—make decision if image quality is sufficient to proceed



700—if “No”, assign image the lowest rank and exit workflow (in this case, the document can't be even extracted from the image).



750—if “Yes,” receive document category and optionally set of critical fields



800—compute usability score from document category and critical fields



850—output document usability score



900—compares usability score to pre-defined threshold



1000—if score is low, assign Rank 3 of Rank 4, then exit.



1100—if score is high, assign Rank 1 of Rank 2, then exit.


One embodiment of an MDIQU system is illustrated in FIG. 2, where a mobile image capture device 102 with an image quality assurance database 104 is connected with an image server 106 with a document validity database 108. Thus, the mobile device 102 may perform one or more of the IQA steps based on information stored in the IQA DB 104 to determine an image quality score, after which the image (if sufficient) is sent to the image server for performing a document validity test using information stored in the document validity DB 108. An image with a sufficient quality and usability may then be transmitted to an application server 110 where an application will utilize the image and its content for a particular task. Although not illustrated herein, the system may also perform the IQA tests at the image server 106, and even perform the document usability tests at the mobile device 102.


III. Individual Mobile Image Quality Tests


In one embodiment, the MDIQU system performs the following image quality tests (Mobile IQA tests) to detect the following image deficiencies: Out-of-focus (OOF), Shadow, Small size, View Skew (perspective distortion), Plain Skew, Warped, Low Internal Contrast, Reflection, Cut Corners, Cut Sides, Busy Background and Low External Contrast. Each score should be returned on the scale of 0 to 1000, when 0 means “No defect” and 1000 means “Severe Defect”. Alternately, MDIQU system may detect absence of the defects, thus swapping 0 and 1000.


U.S. Pat. No. 8,582,862, issued Nov. 12, 2013, gives a description of the Mobile IQA tests and corresponding algorithms, and is herein incorporated by reference in its entirety. Below is a brief description of IQA tests along with their relative importance.














IQA




Score
Importance
How it gets computed







OOF
High for all
OOF is computed on grey-scale snippets of documents based on



applications
frequencies of high-contrast local areas. The more high-contrast




areas is found, the higher score becomes (1000 means




“absolutely sharp”)


Shadowed
High for
“Shadowed” score is computed on grayscale snippets of



MD,
documents. The engine tries to break the entire snippet into 2



Medium
areas of very different brightness. For each such representation



otherwise
(could be several), the engine takes into account the delta in




brightness and the size of the “darker” area. If the delta is high




enough and the size of “darker” area is close to 50%, a “perfect”




shadow is registered, making the score = 0.1000 means “no




shadow”


Reflection
High for
Thus IQA score was designed to detect glare on the image and



Insurance,
assign a numeric value to its size so that 0 would mean “no



low
glare”, and 1000 would mean “significant” glare



otherwise


Low
Medium
Firstly, the engine computes the brightness histogram using


Internal

grayscale snippet of the document. The histogram is a 256-values


Contrast

vector H[ ], where H[0] is the number of pixels with brightness 0




(absolutely black) ... up to H[255] is the number of pixels with




brightness 0 (absolutely white). Secondly, having computed the




histogram H, the engine finds out where 1/3 of all pixels are




located counting from 0 upwards (G1) and from 255 downwards




(G2). Then weighted averages of brightness are computed for [0,




G1] and [G2, 255], which could be considered average




“darkness” (D) and average “lightness” (L) of the snippet




respectively. The difference L-D describes the contrast. After




proper normalization, 0 means very poor contrast and 1000




means very high contrast


Darkness
Medium
The goal of this score is to ensure that the image is not too dark..




Firstly, the engine computes the brightness histogram using




grayscale snippet of the document as explained above. Then a




weighted average grayscale value (G) is computed. After proper




normalization G = 255 is translated into 1000 (document is pure




white), G = 0 is translated into 0 (absolutely black)


Plain
Low
This IQA reflects 2D skew of the document (no skew means the


Skew

document's sides are parallel to the image sides). After




normalization, 1000 means 0 degrees skew while 0 means 45




degrees (45 degrees is the maximum: further 2D rotation will




mean that the document changed orientation). The importance of




detection view skew is low as Moobile SDK handles hight degree




of 2D skew nicely.


View
Mid
This IQA reflects 3D (view) angle between the camera and the


Skew

document No view angle means that the camera view is




perpendicular to the the document surface. When the view angle




becomes significant, the document becomes perspectively




distorted. After normalization, 1000 means 0 degrees skew while




0 means 22 degrees,


Cut
High
This IQA reflects presence of all four corners of the document


Corners

within the mobile image. The engine finds out if a corner is


(side)

located outside of the image as part of the automatic




framing/cropping process. Depending on how big is the missing




(cut-off) part of the corner, the engine computes a penalty per




corner. Zero penalty means that the corner resides inside the




image, maximum penalty is assigned when at least 25% (by area)




of the image quadrant related to this corner is cut-off. Then




maximum of 4 penalties computed for all for corners becomes the




overall cut-corner penalty. After normalization, 1000 means that




no penalty was computed for all corners; 0 means that at least 1




(unspecified) corner was penalized by max. penalty (see above)


Small
Low
This score is computed by comparing the document snippet size


Image

against the mobile image size (the latter becomes known after




cropping). When the camera moves away from the document, the




snippet becomes smaller, thus causing low values of this IQA




metric. After normalization, the score of 1000 means that the




document occupies at least 1/3rd of the entire mobile image area,




while 0 means that it occupies less than 1/6th of the entire mobile




image area.




The importance of detection Small Image is low as Moobile SDK




handles even small images quite nicely.


Warpage
None for
The score is computed as a measure of how flat the document is.



DLs,
The way to compute that is to measure how far the snippet sides



Medium
deviate from ideal straight lines. To normalize such deviation, its



otherwise
maximum value along side is divided by the perpendicular side's




length. For example, to compute “flatness” along the upper




horizontal side of the document, the largest deviation (in pixels)




is divided by the document snippet height. In the end, the largest




normalized deviation is taken, and then further normalized to




become IQA: if the document is absolutely flat, the score




becomes 1000; the deviation of 25% and more becomes 0.


Low
High for
There is insufficient contrast between the document and the


External
DLs,
background (making it harder to located the document)


Contrast
otherwise



medium


Busy
High for
Too many contrasting objects detected in the background


Background
DLs,
(making it harder to located the document)



otherwise



medium









  • Examples of Individual Mobile Defects




FIG. 3 illustrates an image which is out of focus (OOF) and would be identified with an OOF IQA test. The algorithm may be modified to be computed only “on characters”. This means that a special filter is used to detect positions of text characters, which then are used in the computation while the other areas are ignored. FIG. 3 illustrates an example of poor OOF (OOF score=90).


Reflection IQA testing doesn't work well on the image level and needs to be “localized” to the field level.



FIG. 4A illustrates an image of a document with poor plain skew (IQA=200), while FIG. 4B illustrates an image of a document with a view skew.



FIG. 5A illustrates an image with low contrast (specifically a low contrast IQA=210). This IQA should not be confused with Low Contrast background: the former describes contrasts inside the document whereas the latter outside (between the document and background). FIG. 5B illustrates an image which suffers from darkness, and therefore has a darkness IQA score of 290.



FIG. 6A illustrates an image of a document with cut corners, receiving a score of 900. FIG. 6B illustrates an image of a document with a warped image, receiving a warpage score of 0.


These IQA scores may be averaged or combined to determine a total score or weighted depending on the relevance of a particular IQA test. Nevertheless, a simple threshold value may be set to determine whether the image has sufficient quality to proceed to the document usability tests.


IV. Compound IQA Scores


Embodiments of the invention use individual mobile IQA scores to automatically create two compound IQA scores which will ensure that the mobile document could be detected in the mobile image as well as that the document is fully legible. The first compound IQAs will be titled “Crop IQA” and “Quality IQA.”

  • Benefits of Compound IQAs Over “Individual” IQAs


Simplicity: There are more than a dozen of individual IQAs and in essence they are being replaced by 2 compounded ones: “crop” and “quality”. It's much better to move two dials than a dozen.


Eliminating wrong messages: In reality, some individual IQAs are strongly correlated between themselves and with other defects. For example, shadow and darkness (2 different IQAs) may cause low OOF (another IQA) as also can small image (yet another IQA); wrong crop (has its own IQA) can cause wrong identification and therefore present the image as “unsupported” and so on. By compounding related IQAs we avoid potential misclassifications and get rid of this hierarchy and thresholds altogether. MIP will have a much smaller set of thresholds (3) and the error classifier will be well-tested and optimized in R&D.

  • Description of Compound IQA Scores


The compound IQA scores are linear combinations of individual IQA score. The coefficients in combination depend on application as different individual IQA score have different importance for different document categories, see table 2.


V. Document Usability


Embodiments of the MDIQU system provide document usability testing to ensure that the document within the image is indeed a check, a bill, a driver's license etc.


The following categories may be supported: US Checks (Mobile Deposit), Remittance Coupon (Mobile Bill Pay), Credit Card Bills (Mobile Balance Transfer), and Driver's License (Mobile Insurance).

  • Definition of Critical Fields


The usability testing is based on a definition of Critical (or Required) Fields, see FIG. 1. If the fields are not specified, the following ones are used by default.





















Document









Application
Category
Field1
Field2
Field3
Field4
Field 5
Field 6
Field 7







Mobile
Checks
MICR
Legal
Courtesy
Endorsement





Deposit


Amount
Amount


Mobile
Remittance
Account
Payee.ZIP
Biller


Bill Pay
Coupons
Number

Name


Balance
Credit
Account
Payee.ZIP
Biller
Balance


Transfer
Card
Number

Name
Due



Bills


Mobile
Driver
DL
Name
Address
DoB
Class
Sex
Expiration Date


Insurance
License's
Number









  • Computation of Usability Score



The usability computation involves data capture from the document image cropped out of the mobile image. Present MDIQU system uses a Mobile Preprocessing Engine and Dynamic Data Capture engine described in U.S. Pat. No. 8,379,914, which is incorporated herein by reference in its entirety.


The Document usability score is computed based on the set of captured critical fields or/and the confidence of each of such fields. As with the image quality test, the document usability tests results in a yes or no answer to the question of whether the document is usable, and will then further analyze the response to assign a grade value to the document based on how usable (or un-usable) it is.

  • Computer-Implemented Embodiment



FIG. 7 is a block diagram that illustrates an embodiment of a computer/server system 700 upon which an embodiment of the inventive methodology may be implemented. The system 700 includes a computer/server platform 701 including a processor 702 and memory 703 which operate to execute instructions, as known to one of skill in the art. The term “computer-readable storage medium” as used herein refers to any tangible medium, such as a disk or semiconductor memory, that participates in providing instructions to processor 702 for execution. Additionally, the computer platform 701 receives input from a plurality of input devices 704, such as a keyboard, mouse, touch device or verbal command. The computer platform 701 may additionally be connected to a removable storage device 705, such as a portable hard drive, optical media (CD or DVD), disk media or any other tangible medium from which a computer can read executable code. The computer platform may further be connected to network resources 706 which connect to the Internet or other components of a local public or private network. The network resources 706 may provide instructions and data to the computer platform from a remote location on a network 707. The connections to the network resources 706 may be via wireless protocols, such as the 802.11 standards, Bluetooth® or cellular protocols, or via physical transmission media, such as cables or fiber optics. The network resources may include storage devices for storing data and executable instructions at a location separate from the computer platform 701. The computer interacts with a display 708 to output data and other information to a user, as well as to request additional instructions and input from the user. The display 708 may therefore further act as an input device 704 for interacting with a user.


While various embodiments have been described above, it should be understood that they have been presented by way of example only, and not of limitation. The breadth and scope should not be limited by any of the above-described exemplary embodiments. Where this document refers to technologies that would be apparent or known to one of ordinary skill in the art, such technologies encompass those apparent or known to the skilled artisan now or at any time in the future. In addition, the described embodiments are not restricted to the illustrated example architectures or configurations, but the desired features can be implemented using a variety of alternative architectures and configurations. As will become apparent to one of ordinary skill in the art after reading this document, the illustrated embodiments and their various alternatives can be implemented without confinement to the illustrated example. One of ordinary skill in the art would also understand how alternative functional, logical or physical partitioning and configurations could be utilized to implement the desired features of the described embodiments.


Furthermore, although items, elements or components may be described or claimed in the singular, the plural is contemplated to be within the scope thereof unless limitation to the singular is explicitly stated. The presence of broadening words and phrases such as “one or more,” “at least,” “but not limited to” or other like phrases in some instances shall not be read to mean that the narrower case is intended or required in instances where such broadening phrases may be absent.

Claims
  • 1. A non-transitory computer-readable medium comprising instructions which, when executed by a processor, cause the processor to: receive an image of a document within a particular one of a plurality of document categories, wherein the image has been captured by a mobile device;for each of a plurality of image quality assurance (IQA) tests, execute the IQA test on the image to produce an individual IQA score, wherein each of the plurality of IQA tests detects a different image deficiency and is associated with a weight for each of the plurality of document categories, wherein each weight for one of the plurality of document categories indicates a predetermined level of importance of the associated IQA test to that document category;compute a compound IQA score based on a combination of the individual IQA scores and the weights for the particular document category of the document in the image;when the compound IQA score does not satisfy a threshold indicating acceptable image quality, reject the image for further processing; and,when the compound IQA score satisfies the threshold, determine a usability score based on a set of data fields that are extracted from the image of the document, andaccept or reject the image for further processing based on the usability score.
  • 2. The non-transitory computer-readable medium of claim 1, wherein the image deficiencies detected by the plurality of IQA tests comprises one or more of: Out-of-focus (OOF), Shadow, Small size, View Skew (perspective distortion), Plain Skew, Warped, Low Internal Contrast, Reflection, Cut Corners, Cut Sides, Busy Background, or Low External Contrast.
  • 3. The non-transitory computer readable medium of claim 2, wherein one of the plurality of IQA tests comprises generating a grey-scale snippet of the document, and using the grey-scale snippet to determine whether the grey-scale snippet is out of focus based on frequencies of high-contrast local areas.
  • 4. The non-transitory computer readable medium of claim 2, wherein one of the plurality of IQA tests comprises generating a grey-scale snippet of the document, and using the grey-scale snippet to determine whether the grey-scale snippet includes a shadow by breaking the entire grey-scale snippet into two areas of different brightness, determining a delta in brightness, and a size of a darker of the two areas.
  • 5. The non-transitory computer readable medium of claim 2, wherein one of the plurality of IQA tests comprises generating a grey-scale snippet of the document, and using the grey-scale snippet to determine whether the grey-scale snippet has low internal contrast by generating a histogram representative of brightness of pixels in the grey-scale snippet.
  • 6. The non-transitory computer readable medium of claim 2, wherein one of the plurality of IQA tests comprises generating a grey-scale snippet of the document, and using the grey-scale snippet to determine a darkness of the grey-scale snippet by generating a histogram representative of a brightness of pixels in the grey-scale snippet and then generating a weighted average grey-scale value based on the histogram.
  • 7. The non-transitory computer readable medium of claim 2, wherein one of the plurality of IQA tests determines whether a corner of the document is located outside of the image, and computing a penalty per corner when determining that one or more corners are outside of the image.
  • 8. The non-transitory computer-readable medium of claim 1, wherein determining the usability score comprises computing the usability score based on confidence values for one or more data fields in the set of data fields that are predefined as critical.
  • 9. The non-transitory computer-readable medium of claim 1, wherein the plurality of IQA tests comprises a shadowed test that indicates whether or not the image comprises a shadow, wherein the plurality of document categories comprises a mobile deposit, and wherein the shadowed test is associated with a higher weight for the mobile deposit than for one or more other ones of the plurality of document categories.
  • 10. The non-transitory computer-readable medium of claim 1, wherein the plurality of IQA tests comprises a reflection test that indicates an amount of glare in the image, wherein the plurality of document categories comprises an insurance submission, and wherein the reflection test is associated with a higher weight for the insurance submission than for one or more other ones of the plurality of document categories.
  • 11. The non-transitory computer-readable medium of claim 1, wherein the plurality of IQA tests comprises a warpage test that indicates a flatness of the document, wherein the plurality of document categories comprises a drivers license, and wherein the warpage test is associated with a lower weight for the drivers license than for one or more other ones of the plurality of document categories.
  • 12. The non-transitory computer-readable medium of claim 1, wherein the plurality of IQA tests comprises a low external contrast test that indicates a measure of contrast between the document and a background, wherein the plurality of document categories comprises a drivers license, and wherein the low external contrast test is associated with a higher weight for the drivers license than for one or more other ones of the plurality of document categories.
  • 13. The non-transitory computer-readable medium of claim 1, wherein the plurality of IQA tests comprises an out-of-focus test that indicates an amount of contrast in the image and one or more skew tests that each indicate a degree of skew of the document in the image, and wherein, for all of the plurality of document categories, the out-of-focus test is associated with a weight that is higher than the weight that is associated with each of the one or more skew tests.
  • 14. A mobile device comprising an image capture device, memory configured to store instructions, and a processor configured to, when running the instructions: receive an image of a document within a particular one of a plurality of document categories, wherein the image has been captured by the image capture device;for each of a plurality of image quality assurance (IQA) tests, execute the IQA test on the image to produce an individual IQA score, wherein each of the plurality of IQA tests detects a different image deficiency and is associated with a weight for each of the plurality of document categories, wherein each weight for one of the plurality of document categories indicates a predetermined level of importance of the associated IQA test to that document category;compute a compound IQA score based on a combination of the individual IQA scores and the weights for the particular document category of the document in the image;when the compound IQA score does not satisfy a threshold indicating acceptable image quality, reject the image for further processing; and,when the compound IQA score satisfies the threshold, determine a usability score based on a set of data fields that are extracted from the image of the document, andaccept or reject the image for further processing based on the usability score.
  • 15. The mobile device of claim 14, wherein the image deficiencies detected by the plurality of IQA tests comprises one or more of: Out-of-focus (OOF), Shadow, Small size, View Skew (perspective distortion), Plain Skew, Warped, Low Internal Contrast, Reflection, Cut Corners, Cut Sides, Busy Background, or Low External Contrast.
  • 16. The mobile device of claim 15, wherein one of the plurality of IQA tests comprises generating a grey-scale snippet of the document, and using the grey-scale snippet to determine whether the grey-scale snippet is out of focus based on frequencies of high-contrast local areas.
  • 17. The mobile device of claim 15, wherein one of the plurality of IQA tests comprises generating a grey-scale snippet of the document, and using the grey-scale snippet to determine whether the grey-scale snippet includes a shadow by breaking the entire grey-scale snippet into two areas of very different brightness, determining a delta in brightness, and a size of the darker of the two areas.
  • 18. The mobile device of claim 15, wherein one of the plurality of IQA tests comprises generating a grey-scale snippet of the document, and using the grey-scale snippet to determine whether the grey-scale snippet has low internal contrast by generating a histogram representative of brightness of pixels in the grey-scale snippet.
  • 19. The mobile device of claim 15, wherein one of the plurality of IQA tests comprises generating a grey-scale snippet of the document, and using the grey-scale snippet to determine a darkness of the grey-scale snippet by generating a histogram representative of a brightness of pixels in the grey-scale snippet and then generating a weighted average grey-scale value based on the histogram.
  • 20. The mobile device of claim 15, wherein one of the plurality of IQA tests determines whether a corner of the document is located outside of the image, and computing a penalty per corner when determining that one or more corners are outside of the image.
RELATED APPLICATIONS

The present application is a continuation of U.S. patent application Ser. No. 15/721,160, filed on Sep. 29, 2017, which is a continuation of U.S. patent application Ser. No. 15/397,639, filed on Jan. 3, 2017, now issued as U.S. Pat. No. 9,779,297, which is a continuation of U.S. patent application Ser. No. 14/217,170, filed on Mar. 17, 2014, now issued as U.S. Pat. No. 9,536,139, which claims priority to provisional application No. 61/802,039, filed on Mar. 15, 2013, all which are hereby incorporated by reference in their entirety.

US Referenced Citations (93)
Number Name Date Kind
4311914 Huber Jan 1982 A
5966473 Takahashi et al. Oct 1999 A
5999636 Juang Dec 1999 A
6038351 Rigakos Mar 2000 A
6125362 Elworthy Sep 2000 A
7072862 Wilson Jul 2006 B1
7426316 Vehvilainen Sep 2008 B2
7584128 Mason et al. Sep 2009 B2
7735721 Ma et al. Jun 2010 B1
7869098 Corso et al. Jan 2011 B2
7873200 Oakes, III et al. Jan 2011 B1
7876949 Oakes, III et al. Jan 2011 B1
7978900 Nepomniachtchi et al. Jul 2011 B2
8358826 Medina, III et al. Jan 2013 B1
8538124 Harpel et al. Sep 2013 B1
8540158 Lei Sep 2013 B2
8959033 Oakes, III et al. Feb 2015 B1
9058512 Medina, III Jun 2015 B1
9613258 Chen et al. Apr 2017 B2
9842331 Nepomniachtchi et al. Dec 2017 B2
10360447 Nepomniachtchi Jul 2019 B2
20020037097 Hoyos et al. Mar 2002 A1
20020085745 Jones et al. Jul 2002 A1
20030099379 Monk et al. May 2003 A1
20040017947 Yang Jan 2004 A1
20040037448 Brundage Feb 2004 A1
20040081332 Tuttle et al. Apr 2004 A1
20040109597 Lugg Jun 2004 A1
20040205474 Eskin et al. Oct 2004 A1
20040236688 Bozeman Nov 2004 A1
20050097046 Singfield May 2005 A1
20050125295 Tidwell et al. Jun 2005 A1
20050163362 Jones et al. Jul 2005 A1
20050180661 El Bernoussi et al. Aug 2005 A1
20050220324 Klein Oct 2005 A1
20050229010 Monk et al. Oct 2005 A1
20060045322 Clarke et al. Mar 2006 A1
20060045344 Paxton et al. Mar 2006 A1
20060177118 Ibikunle et al. Aug 2006 A1
20070009155 Potts et al. Jan 2007 A1
20070086643 Spier et al. Apr 2007 A1
20070110277 Hayduchok et al. May 2007 A1
20070114785 Porter May 2007 A1
20070154071 Lin et al. Jul 2007 A1
20070168382 Tillberg et al. Jul 2007 A1
20070206877 Wu et al. Sep 2007 A1
20070211964 Agam et al. Sep 2007 A1
20070288382 Narayanan et al. Dec 2007 A1
20070297664 Blaikie Dec 2007 A1
20080040280 Davis et al. Feb 2008 A1
20080086420 Gilder et al. Apr 2008 A1
20080152238 Sarkar Jun 2008 A1
20080174815 Komaki Jul 2008 A1
20080183576 Kim et al. Jul 2008 A1
20090063431 Erol Mar 2009 A1
20090185736 Nepomniachtchi Jul 2009 A1
20090185737 Nepomniachtchi Jul 2009 A1
20090198493 Hakkani-Tur et al. Aug 2009 A1
20100038839 DeWitt et al. Feb 2010 A1
20100102119 Gustin et al. Apr 2010 A1
20100114765 Gustin et al. May 2010 A1
20100114766 Gustin et al. May 2010 A1
20100114771 Gustin et al. May 2010 A1
20100114772 Gustin et al. May 2010 A1
20100200660 Moed et al. Aug 2010 A1
20100208282 Isaev Aug 2010 A1
20100254604 Prabhakara et al. Oct 2010 A1
20110013822 Meek et al. Jan 2011 A1
20110052065 Nepomniachtchi et al. Mar 2011 A1
20110091092 Nepomniachtchi et al. Apr 2011 A1
20110280450 Nepomniachtchi Nov 2011 A1
20110289028 Sato Nov 2011 A1
20120010885 Hakkani-Tur et al. Jan 2012 A1
20120030104 Huff et al. Feb 2012 A1
20120033892 Blenkhorn et al. Feb 2012 A1
20120070062 Houle et al. Mar 2012 A1
20120150773 DiCorpo et al. Jun 2012 A1
20120197640 Hakkani-Tur et al. Aug 2012 A1
20120201416 DeWitt et al. Aug 2012 A1
20120230577 Calman et al. Sep 2012 A1
20120278336 Malik et al. Nov 2012 A1
20130058531 Hedley et al. Mar 2013 A1
20130120595 Roach et al. May 2013 A1
20130223721 Nepomniachtchi et al. Aug 2013 A1
20130325706 Wilson et al. Dec 2013 A1
20140037183 Gorski Feb 2014 A1
20140040141 Gauvin et al. Feb 2014 A1
20140064621 Reese et al. Mar 2014 A1
20140108456 Ramachandrula et al. Apr 2014 A1
20140126790 Duchesne et al. May 2014 A1
20140133767 Lund et al. May 2014 A1
20140307959 Filimonova et al. Oct 2014 A1
20160092730 Smirnov et al. Mar 2016 A1
Foreign Referenced Citations (1)
Number Date Country
1020070115834 Dec 2007 KR
Non-Patent Literature Citations (4)
Entry
International Search Report issued in related International Application No. PCT/US2011/056593 dated May 30, 2012 (3 pages).
Office Action for related CA Patent Application No. 2,773,730, dated Aug. 21, 2017, in 4 pages.
Office Action for related U.S. Appl. No. 16/259,896, dated Dec. 12, 2019, in 22 pages.
Notice of Allowance for related U.S. Appl. No. 16/579,625, dated Jan. 13, 2020 in 27 pages.
Related Publications (1)
Number Date Country
20190347481 A1 Nov 2019 US
Provisional Applications (1)
Number Date Country
61802039 Mar 2013 US
Continuations (3)
Number Date Country
Parent 15721160 Sep 2017 US
Child 16518815 US
Parent 15397639 Jan 2017 US
Child 15721160 US
Parent 14217170 Mar 2014 US
Child 15397639 US