Patent Application
20110019243
1. Field of the Invention
The present invention relates to reading forms, and more particularly to optically reading forms, converting the optical information into digital data, and storing that digital data for processing.
2. Background Information
Printed documents, play slips, lottery scratch tickets, instant tickets and the like are collectively defined herein as “forms.” Often forms have man made-marks at locations indicating a specific human intent. Correctly identifying a form and reading or processing the printed and man-made markings are important non-trivial tasks.
Some of these tasks include: detecting the presence of a form, determining that the form is motionless, locating and identifying marks on the form, and then interpreting the meaning of the marks.
Forms may be identified by printed markings that are read and interpreted, or a human may indicate the form type. The printed markings normally include logos or other special marks. For example, registration marks may be printed and used by processing equipment to accurately identify the type of form and locations on the form. Herein “registration” is defined to include alignment, orientation, scaling and any other operations performed on an image of a form wherein the individual pixels in the image of a form may be directly compared to pixels in other images or a model image of the same form type. Herein “model” refers to the stored digital image of a particular flat form.
Typically, reading an example of a form begins with a photo-sensitive device or camera or the like capturing an image of the form. The captured image may be digitized, downloaded, stored and analyzed by a computing system running a software application, firmware embedded in a hardware framework, a hardware state machine, or combinations thereof as known to those skilled in the art.
Some form reading systems include an open platen upon which a form is simply laid. The side where the form is inserted may be open, but access to the platen may be open on three or even all four sides. Other types of readers include tractor-type readers that deliver the form to a controlled environment for reading.
One continuing problem with form readers is that if the form is not flat, the location and therefore the meaning of a mark or a series of marks may be misinterpreted possibly causing unacceptable errors, including misreading the form.
The present invention is directed toward creating and reading stereoscopic views of the same scene, for example, the scene may be a form. The parallax ability of the present invention allows a determination that the example form is flat or not, or a determination that the form is not reliably readable. Raw or unprocessed digital data from the stereoscopic views of the same scene may be processed to convert the raw digital data into digital data that would have been gathered if the form were flat. The present invention may provide a virtual flat form from the raw data of a bent form.
The present subject matter includes previously stored models of known forms, in which the location information and characteristics of boundaries, logos, registration and alignment marks and any other relevant areas of interest on the form are stored in a computer system. The characteristics may include, for example, the center of mass of a mark, the radius of gyration of the mark, the number of pixels in the mark, and the shape of the mark. In addition, the characteristics may include the length, thickness and shape of lines, logos, registration marks or other such artifacts. The type of form may be indicated by an agent, or one or more identifying marks may be read wherein the processing system knows the type of form being processed.
It is presumed that the two views of captured digital data and the model have been registered to each other. That may be accomplished by presenting a flat target as the scene for both optical paths. Specific locations on the target may indicate the origin for an X, Y coordinate system that applies to both stereoscopic views and the model for that form.
The location of a mark is defined as the center of mass (COM) of the mark, where the center of mass is: COMx=(ΣXpixels)/(# of X pixels); COMy=(ΣYpixels)/(# of Y pixels). The radius of gyration is Rgyr=(Σmr2/mass)1/2. Here mass is the number of pixels in the mark and m is the mass of one pixel, here assumed to be 1. The Rgyr is independent of orientation. Other characteristics may include mass alone, shape, and other geometric moments may be used.
Illustratively, two stereoscopic optical images of a form are captured and referred to as “captured optical images.” Since the images are taken from two different views, parallax techniques may be used in processing these images. Each view may be received by separate photo-sensitive surfaces (within a camera, etc.) that may be separate areas of one photo-sensitive surface or that may be two photo-sensitive surfaces, but both within one camera. The two captured optical images are digitized forming “captured digital images” that are registered and stored in a memory wherein the captured digital images may be compared to each other and to the stored model. The digitalization may occur in the camera electronics or in the processor. An application in a computer system coordinates the digitization, registration, storing, comparing, and thresholding of acceptable differences (discussed below) to determine to further process or reject the form. The further processing may include reading all the relevant, including man-made marks on the form and forming a virtual flat form by correcting the differences and then reading all the relevant marks on the form. The information of the marks may then be sent to a central controller that may authorize a payout to the form holder or otherwise process the information.
It is noted that optical filters and analog-type processing may be accomplished in embodiments of the present invention, although they are not discussed hereinafter.
The processing of the two captured digital images may entail comparing marks on the images to each other and to marks on the model of the form. Discrepancies will become readily apparent. For example, if the two stereoscopic views are properly registered and aligned with each other, and a straight line segment at a known location is on the form, that straight line segment on each of the captured digital images will be congruent (within acceptable tolerances, see below) to each other if the form is flat. The two captured digital images and the model will all have marks with identical characteristics of location (COM), size, Rgyr, line length, line thickness, etc., if the form is flat.
If the form is not flat, in the above example, the two captured digital images of a straight line (or any other such mark or artifact), when compared, will not be congruent with each other or with the model of the straight line. Illustratively, if a form is bent upward, a straight line traversing the bend will not be congruent on the two captured digitized images of the line or to the model of the straight line. Moreover, a mark that is raised on the bent portion of a form will have a different location in each of the captured images and both of these locations will be different from the location on the model (flat) form. The mark will be of a different size compared to each of the captured images and to the stored model.
If enough known marks are distributed on the form, parallax-type corrections may be applied to the entire form, or conversely, the parallax-type correction may indicate that the form should be rejected as not flat enough. The granularity (the closeness of the known marks) may allow corrections to the entire captured images of the form and, thus, the entire form may be read and processed. Parallax correction refers to comparisons of locations in the two captured images to each of and to the model. The comparisons refer to geometric, trigonometric calculations from the known locations and characteristics of the marks and the measured locations and characteristics. In one embodiment, if the correction calculations indicate that the raised portion of a form exceeds about 0.5 inches (heuristically determined), the form should be rejected.
The model includes locations and characteristics of marks such as lines, symbols, logos, alignment, registration and/or other printed information on the form. The system may compare the model to the captured digital images and detect differences. For example, when a known single straight line on a form is captured as something other than a single straight line, the system may determine that the form is not flat and it may be rejected. For example, the orientation of the straight line on a bent form with respect to the cameras may capture the line as straight but with differing lengths compared to the straight line in the model form or as a bent line. Such differences are indications that the form is not flat.
The differences, however, may be used to indicate that the form is flat. For example, thresholds may be developed and applied to the differences that might determine that the example form is flat enough to be further processed.
The differences between and among the two captured digital images and the stored model digital image may allow correction for the non-flatness of the form wherein a virtual flat form results. Projection algorithms have been developed that will correct for a known form that is bent. For example, Mercator Projections and similar projections are known to those skilled in the art.
Herein, if the thresholds are “met,” the differences may be judged to be too great and the form may be rejected. If the thresholds are not met, the differences are judged to be small enough to allow further processing of the form. The thresholds may be applied after projection processes have been applied. Illustratively, known marks distributed over the entire surface of the form may be used to determine that the entire surface of interest on a form is flat enough to process any other marks on the example form. The surface of interest includes any location on the form where known or man-made marks may exist to convey relevant information of the form type.
The invention description below refers to the accompanying drawings, of which:
The following presumes that the form type is known (an agent may so indicate), and that the two areas 9a and 9b have been registered with respect to each other and to a model of the form.
Note that the lenses 7a and 7b and the photo-sensitive surface areas 9a and 9b are representative and may be quite different in practice. For example, one or no lenses may be used, but alternatively, optic modules with lenses and mirrors may also be used, and photo-sensitive surface areas 9a and 9b may, as mentioned above, be separate surfaces within one camera, as well as different areas on a single surface.
The form 2 is located on a platen 5 that is positioned below the camera 18. Two captured optical images of the same scene are formed on the photo-sensitive surface areas 9a and 9b that may be downloaded (e.g., scanned, read-out) by electronics 8 to produce video signals 10a and 10b for each surface 9a and 9b, respectively. The video signals 10a and 10b are digitized and stored as pixels (or pixel data) of two captured digital images in memory 18 on a computer system 12.
The computer system 12 includes a processor 14 that operates on the pixels, the memory 18 and I/O drivers 16 that handle, at least, displays, keyboards, buttons, and communications. The computer system 12 may be connected to a network 17 that communicates with a central controller 15.
Memory 18 may include one or more image buffers, other buffers, cache, etc. An operating system and software applications may be stored in memory 18. An image processing application 13, discussed below, processes the image data for both of the stereoscopic captured digitized images. Removable flash memory 19, as preferred, may contain the application programs wherein removing the flash for software security leaves no application programs in the computer system 12.
Although the photo-sensitive surface 26 is shown as a single surface, the images are directed onto separate sections that can be addressed and downloaded independently. The captured optical image data on the photo-sensitive surfaces represents the light intensity striking the photo-sensitive surface. As described above, the camera electronics 8 reads out the image intensity data from the photo-sensitive surface 26. The video signals 10a and 10b are downloaded to the processing system 12 where it is digitized and processed.
Since the same scene is input to the photo-sensitive surfaces, in order to compare the two views of the same scene, as mentioned above, the corresponding locations on each photo-sensitive surface must be registered with each other. Known registration marks may be recognized and located on each image such that the “IMAGE a” pixels and the “IMAGE b” pixels correspond directly to each other. That is, all the corresponding locations within each image can be directly overlaid and match each other within each captured digital image. Typically, the captured digital images and the model are registered on an X,Y plane coordinate system, but other systems may be used.
Parallax effects are well-known in the art and represent one method of measuring distances, for example, astronomical distances to other heavenly bodies. The angle to a heavenly body from two different locations may be compared and the difference in the measured angle is a function of the distance to that heavenly body. Parallax calculation, however, also allows, as in the present application, the ability to detect a form that is bent, and then project the marks on a bent form to locations and sizes as if the marks were on a flat form—a virtual flat form.
Form 40′ reflects form 40 with a bend at location 58 through the angle 56. The point B rotates upwards 60 to location B′. On the maps of
Note that the distances of the movement from B to B′ on each map are not of the same length. This is obvious from inspection of the ray tracings of
But the movement from B to B′ in each surface of
The locations, characteristics and meanings of marks on the model for form 40 are known to the processor 12. The marks A and B may have parameters (shapes, size, etc. as mentioned above) that are known to the processor 12. The processor 12 may process the captured digital images and recognize the marks A and B, and know where they should be located on SURa and SURb. When the processor finds the location of the mark B to be different on SURa and SURb, the processor may than apply a correction factor that moves the locations from B′ of B″ to B on each map. This process may be expanded by locating other known recognizable marks on the form, like C and D, where C is at its model location but where D is at D′ due to the fold at 58. With enough marks distributed over the entire surface of the form 40, correction factors may be developed and applied for the entire surface of the form 40. The result is a virtual flat form where the marks on the form can be interpreted for meanings.
Known marks on the form may be used to calculate corrected locations for other known marks on the form. Difference errors may be calculated for these known marks, and, if there are enough distributed over the surface of the form, errors may be calculated for areas over the entire surface of the form.
For example, in
Thresholds may be developed heuristically and if the calculated errors fall within thresholds, all the marks, including man-made marks, on the form may be read and processed. In one application, a threshold of 0.5 inches of a rise of a mark from a virtual flat form to the actual form may be applied. For example, from the
On
The corrections may include, but are not limited to, location and/or parameters of the mark including location, orientation, size or scale, line thickness, degree of congruency (how much of the mark is congruent among the two captured images and the model image), etc.
It should be understood that above-described embodiments are being presented herein as examples and that many variations and alternatives thereof are possible. Accordingly, the present invention should be viewed broadly as being defined only as set forth in the hereinafter appended claims.
