Patent Application
20120301009
The present invention relates to an apparatus and method for identifying coins, more specifically identifying the denomination, type, date, and mint of coins which may be used for the discrimination of coins by said attributes and the promotion of a coin counter.
Coin identification methods are often used for the purposes of determining the denomination and authenticity of coins and often for the purposes of mechanically discriminating coins based on that information. The most common coin discrimination devices, such as those used in automatic vending machines, coin-to-currency changers, gaming devices such as slot machines, bus or subway token “fare boxes”, and the like, generally employ inductive coin testing methods to determine the denomination and authenticity of coins. These methods typically work by measuring the effect of a coin on an alternating electromagnetic field produced by one or more coils disposed at a passage through which a coin passes. The effect of the coin on the impedance of the coil(s) is dependent on one or more of the properties of the coin such as diameter, thickness, conductivity and permeability. The detection signals output from coil sensors of this type are concentrated in a basic pattern representative of these characteristics of the coin. By comparing the measured pattern with patterns established in advance, the genuine or counterfeit nature of the coin, and the denomination of the coin, can be determined.
More recently, optical sensors have been implemented to provide another method, or additional criteria, by which the denomination and authenticity of a coin may be determined. Optical sensor methods have been primarily directed towards the discrimination among coins of similar electromagnetic and physical properties, yet not authentic with respect to a specific sovereignty, such as coins originating from a foreign country or entity. In such methods, an optical sensor typically captures a two-dimensional image of a coin surface such as one of the faces, the periphery, or the ridge of the coin which is then used to perform pattern matching by comparing the acquired coin image to patterns of known coins to produce a discrimination signal. However, little effort has been directed towards the automated identification of coinage features deliberately minted, yet not universally present on coins of the same denomination or type, such as details indicating the date and the location of mint of a coin. Such information is desirable as it can be a source of novelty, entertainment and appreciation. Additionally, certain coins of particular date and mint are considered “rare” and are thus more valuable than coins of similar denomination yet produced with a differing date or mint. Currently, identifying and retrieving coins of specific date and mint from general circulation is difficult and time consuming. Date and mint information is typically determined “by eye,” sometimes with the aid of magnification, and can often be taxing on the individual as the examination of a large number of coins can be tedious and time consuming. There is currently no device which automates the identification of these coin attributes, nor one which can do so at high speed and low cost.
Prior art has been directed towards capturing an image of a side of a coin, generating a binary image and discriminating the coin based on geometric relations among patterns detected in the binary image. In one such method, identification is based on the radius, number and area of connected regions and the distances between those connected regions; by comparing these measured values with those of known coins the authenticity and denomination of the coin is determined. However, methods of this type are insufficient for the robust identification of patterns not universally present on the denomination or type of coin detected, such as patterns indicative of the date and mint, which can have a plurality of shapes and features which subtly differ. For similar reasons, methods in which coin image data is highly abstracted, often in order to reduce computational complexity, prove insufficient to extract the desired coin attributes.
Much of the prior art makes use of the fact that coins can have authenticity and denomination specific information on the edge, periphery, or on both sides (obverse and reverse) of a coin, and thus the coin only needs to be imaged from one vantage point to determine the denomination and genuine nature of the coin. However, when date and mint information are present on a coin, that information tends to be present on only one side of the coin, thus both sides of a coin often need to be imaged to extract the desired information from the proper side of the coin. The need to capture and process images of both sides of a coin produces non-trivial difficulties which are not adequately overcome by the prior art, which are addressed by the invention described herein.
Prior art has been directed towards the use of MOS-type image sensors to capture coin images. MOS-type image sensors often suffer from blurring effects and geometrical distortion caused by the ‘rolling shutter’ of such sensors. One method overcomes these limitations by using an image acquisition method in which the image capture phase begins in advance, before a coin reaches a prescribed position, at which point the coin is briefly illuminated and the image capture is concluded. In several embodiments presented herein, rolling shutter issues arising from the use of MOS-type sensors are circumvented using a different, simpler method.
Prior art has been directed towards measuring the damage, or wear, of a coin using captured images of the sides of the coin. In one method, coins are advanced using a conveyor system; magnetic and image sensor data is then acquired of the coins and compared to data patterns of known coins. Other methods are aimed at the replication and automation of the grading processes used in the collectables industry to determine the quality of known coins. The methods and apparatuses described therein are generally unsuitable for the purposes of the present invention described herein.
Prior art has been directed towards converting circular images of coins into rectangular images and comparing those rectangular images to reference images for the purpose of determining the genuine or spurious nature of the coins. However, such methods produce non-linear spatial distortions that make robust identification difficult, especially for subtle details such as date and mint information. The method described herein does not require the transformation of circular images to rectangular images.
Prior art has been directed towards verifying the embossed nature of an imaged coin using special illumination and image processing methods. Such methods are also not necessary for the purposes of the invention described herein.
Devices capable of extracting denomination, type, date and mint information from coins may be used for the sorting of coins by such attributes as well as used to augment current devices that employ coin discrimination such as coin counters which typically aid untrained members of the general public in the conversion of their coins to cash. Such an augmented coin counting device could provide the return, compensation or redemption of users' coins deemed “rare” or valuable as well as provide entertainment for users of such devices and a means for promotion and loyalty for such devices. Such an augmented coin counting device may provide a sweepstakes-like experience for users as they are made aware of, or rewarded for, coins with additional value, be it collectible value, promotional value, monetary value, or otherwise, that the users were previously unaware of. Such an augmented coin counting device may provide entertainment to users which may be used to distinguish the device from that of competing products or services.
Prior art has been directed towards coin identification for the purpose of promotion and encouraging the use of coin counting kiosks. However the method described requires the minting and distribution of non-government issued promotional coins for which the winning/losing nature of the promotional coins cannot be visually determined. In said method, the winning/losing nature of a coin is made manifest only upon deposit into a coin counting kiosk, which detects, and discriminates on, the unique inductive signature of the promotional coin. The promotional methods described herein use the visual features of government issued coins, which do not require the additional minting and distribution efforts as the promotional coins described in the prior art, and for which the winning/losing nature, or relative place in a spectrum of rewards of the coin, can be visually determined prior to deposit.
Other uses for devices capable of extracting denomination, type, date and mint information from coins may be the aid in “vintage surveys” of coins in circulation conducted by central banks, minting agencies, government and academic authorities, etc. in which a large sample of coins is taken and the date and mint data is collected to determine statistics about the circulating money supply. Other areas of use may include sorting, entertainment, promotion or gaming.
In one embodiment, the method and apparatus described herein is implemented in conjunction with publicly used coin counting kiosks. Such coin counting devices are typically used for processing and/or discriminating coins or other objects, such as discriminating among a plurality of coins or other objects received all at once, in a mass or pile, from the user, with the coins or objects being of many different sizes, types or denominations. These coin counting devices typically have a high degree of automation and high tolerance for foreign objects and less-than-pristine objects (such as wet, sticky, coated, bent or misshapen coins), so that the device can be readily used by untrained members of the general public, requiring little or no human manipulation or intervention, other than inputting the mass of coins.
One aspect of the method and apparatus described herein is to identify the denomination, type, date, and mint of coins, or a subset of those coin attributes. In one embodiment, a plurality of coins are dropped into a hopper which then funnels the coins to a position where a carousel or other advancing mechanism can pick up individual, or a plurality of coins. The coin advancing mechanism is mechanically connected to a computer controlled stepper motor which allows the coins to be advanced along a coin sliding surface in discrete or continuous motion. The coin sliding surface, or a portion thereof, is transparent and coins passing over a specified region are illuminated by lighting sources. Imaging devices, such as cameras using CCD or CMOS type image sensors, then acquire digital images of both sides (or faces) of the coins, those which are adjacent to the coin sliding surface and those which are opposing.
A central computer or dedicated image processor then proceeds to process the two acquired digital images. A global threshold is applied to the acquired images resulting in black and white (binary) images; the white (positive) regions are then summed and if the resulting value is below a set threshold value, the images are discarded. If the resulting value is above the threshold value, the images are considered to be good candidates for containing coins or other objects. The images are then corrected for noise, background artifacts, geometric distortion, and camera orientation. The images then undergo an adaptive binary threshold and contours are detected in the resulting binary images. Contours with length smaller than a threshold value are rejected and ellipses are fit to the remaining contours using a least-squares fitting method. Ellipses with low eccentricity are considered good candidates for coins, and ellipses with an effective radius within the range of a valid coin radius are considered for further processing. For US coins, the effective radius typically indicates the denomination candidate of the coin imaged, which is further confirmed or disconfirmed upon subsequent processing. The location of the ellipse fitted to the contour of a valid coin is then used to crop the image in order to isolate the image of the individual coin for further processing. In the case of multiple coin processing, prior camera calibration and location coincidence criteria allows for images of the obverse and reverse sides of valid coins to be properly paired for further processing.
The binary image resulting from the adaptive threshold stage provides information indicative of the embossed detail of the coin due to a lighting configuration in which the coins are illuminated at a large angle relative to the normal of the sides of the coins. This binary image is then fit to templates of coins of known denomination and type at a plurality of rotational orientations. The template exhibiting the best fit identifies the orientation, type and respective face of the coin depicted in each image as well as provides further confirmation of the denomination of the coin. The acquired images are then corrected for the orientation of the coin.
Subsections of the rotationally corrected binary images are then taken from regions where date and mint information should approximately be located. These cropped images containing date and mint information are then matched to templates of all possible date and mint information for the particular coin denomination and type identified. The best match renders the date and mint information contained in the images. Various metrics and machine-learning algorithms can be further applied to the images and template matching results in order to improve recognition accuracy.
In one embodiment, the user of the coin counting kiosk is made aware of the denomination, type, date and mint data collected from their deposited coins using a monitor, or touch-screen, connected to the kiosk. This collected coin data and the natural rarity of specific types, dates and mints of coins in present circulation is used as the basis for entertainment, loyalty and promotion of the coin counting kiosk. Points, prizes, coupons, merchandise, badges, honors or publicity may then be awarded to the user based on the user's coin data and the likelihood of specific coins, groups of coins or other derivative events. Users' coin data is saved to a central database, via a modem or other communications facility connected to the coin counting kiosk, to allow users to access their coin data, and any derivative data, from auxiliary platforms such as computers, social networking platforms, social media outlets, mobile devices, Point-of-Sale (POS) systems, customer loyalty systems as well as from the same or a different coin counting kiosk.
In one embodiment, the denomination, type, date and mint of each processed coin is compared to a database of “rare” and/or user-defined coins. The user may then be informed of coins processed which match the database criteria, upon which the coins may then be returned to the user or the user may be credited for the deposit of their coin.
The detailed description set forth below in connection with the appended drawings is intended as a description of presently-preferred embodiments of the invention and is not intended to represent the only forms in which the present invention may be constructed or utilized. The description sets forth the functions and the sequence of steps for constructing and operating the invention in connection with the illustrated embodiments. However, it is to be understood that the same or equivalent functions and sequences may be accomplished by different embodiments that are also intended to be encompassed within the spirit and scope of the invention.
The coin identification method and apparatus described herein can be used in connection with, or as an enhancement to, a number of devices and purposes. One such implementation is illustrated in
Another implementation of the method and apparatus described herein is illustrated in
The general coin path for the implementation depicted in
In the implementation depicted in
The coin pickup assembly 205, referred to hereafter as the carousel, is comprised of a circular plate 224 with machined holes or sockets 212, referred to hereafter as sockets, and a protrusion, extending axially outward along the circumference of the circular plate 224, referred to hereafter as the lip 208. In some embodiments, bumps or grooves may be placed on the lip 208 to facilitate the agitation of coins such that coins may position themselves into the sockets 212. Bumps or grooves may also be placed on the circular plate 224 of the carousel to facilitate coin agitation. The sockets 212 are cut through the circular plate 224 of the carousel 205 at regularly spaced intervals around and adjacent to the circumference of the circular plate 224. The sockets 212 are shaped such that they are conducive to capturing coins in the recess of the sockets 212.
For example, in one embodiment this shape consists of a circular region on the leading edge 290 with a flat portion on the trailing edge 218. The shape of the sockets 212 may also be sized such that only one coin can fit laterally in the recess at any one time. The thickness of the circular plate 224 of the carousel 205 is such that the recess formed by the sockets 212 allows for only one coin to sit on the ledge of the sockets 212 without sliding out. The axially oriented thickness of the lip 208 is such that coins which fall onto the carousel 205 can rest, or roll, on the lip 208 until they enter one of the sockets 212. The carousel 205 is affixed to an axle 210, about which the carousel 205 rotates. The front face 254 of the carousel 205 is parallel with the plane of the coin sliding surface 213; the opposing face of the carousel 205 is adjacent, or flush, with the coin sliding surface 213. The carousel 205 slides against the (stationary) coin sliding surface 213 upon rotation of the carousel 205. The carousel 205 may be made from any type of hard material, such as plastic, thermoplastic, glass, metal, wood or some composite. In one embodiment, the carousel 205 is constructed of hard plastic treated or painted such that it has low reflectivity of light. The underside of the carousel 205, that which is in physical contact with the coin sliding surface 213, may be composed of a different material conducive to low friction sliding against the coin sliding surface 213, such as a cloth or plastic, which may be attached to the underside of the carousel 205 with industrial glue, epoxy, mechanical fasteners, or the like. The carousel 205 may also have a calibration mark 211 placed at a radius such that it can enter the imaging area of the top camera 207a which allows for the calibration of the angular orientation of the carousel 205 with respect to the imaging devices 207a,b.
In another embodiment calibration marks are placed adjacent to every socket 212. The calibration mark 211 can be painted or be a separate material embedded in the carousel 205 such that it is flush with the circular plate 224 of the carousel 205. The calibration mark 211 can also be colored to produce a high contrast to the surface of the carousel 205, such as white or yellow.
The carousel 205 may be affixed to an axel 210 with a spring loaded coupler or any other device that provides a biasing force to keep the carousel 205 pressed flush against the coin sliding surface 213, preventing gaps between the carousel 205 and the coin sliding surface 213 through which coins may otherwise fall. Alternatively, or in addition to, a piece of material or biasing device around the circumference of the carousel 205 may apply uniform pressure, or provide a boundary, to the lip 208 of the carousel 205 to keep the carousel 205 flush to the coin sliding surface 213.
The axle 210 on which the carousel 205 is affixed is connected to a motor 241 shown in
Discrete advancement of the carousel 205 may be achieved mechanically, with a dedicated circuit, or more preferably via a stepper type motor and a micro-controller 240 (
As the carousel 205 rotates (counter-clockwise in
Along the annular path, a captured coin 223 passes over a transparent surface 219 aligned in parallel (or flush) to the elevation of the coin sliding surface 213 such that the coin 223 can easily slide onto the transparent surface 219. In another embodiment, the entire coin sliding surface 213 is transparent such that there is no precipice, or edge, over which the coin 223 must pass over. The transparent surface 219 may be made of transparent plastic, or thermoplastic such as Plexiglas® or Lexan®. In one embodiment, the transparent surface 219 is constructed of scratch resistant, optical grade glass such as Corning® Gorilla® Glass. The transparent surface 219 may be easily removed to allow for cleaning and replacement to maintain the optical integrity of the transparent surface 219 throughout operation. Additionally, the underside of the carousel 205 may serve to wipe or buff the transparent surface 219 as the carousel 205 rotates, thus maintaining the optical integrity of the transparent surface 219 during operation.
Behind the transparent surface 219 is an imaging device 207b (
External lighting 206a,b,c (reverse side lighting not shown in
After having both sides of a coin imaged, the images are processed by a central computer 152, the details of which are described in detail below. In one embodiment, a second image of a coin is captured only if the first side captured is not sufficient to extract all the information necessary to extract the desired attributes of the coin, thus conserving computation time and resources.
After image capture and image processing, the carousel 205 advances the coin to the apex of the coin sliding surface 213 where a hole in the coin sliding surface produces a ledge 209 that causes the coins to slide over and fall behind the plane of the coin sliding surface 213 onto a coin rail 203 which guides coins, e.g. coin 204, behind the plane of the coin sliding surface 213. The hole in the coin sliding surface 213 is sufficiently large such that coins of all sizes can pass through and fall onto the coin rail 203 due to their own gravity. The coin rail 203 behind the coin sliding surface 213 is spaced sufficiently such that a coin can pass freely behind the coin sliding surface 213. The face of the coin 204 rests adjacent the face of the coin rail 203 and sits on a protrusion, or ridge 216 along which the coin rolls due to the inclination of the coin rail 203.
The coin rail 203 may be made of any hard material such as plastic, thermoplastic, glass, metal, wood or some composite. In one embodiment, the coin rail 203 is constructed with a hard plastic such as high density polyurethane such that it does not electromagnetically interfere with the workings of an auxiliary sensor 202. Additionally, ridges 221 may be on the rail protruding slightly towards the plane of the coin sliding surface 213 to reduce surface contact with a coin 204 to avoid jams. Coins that fall off the coin rail 213 may be caught by a protrusion 215 of the hopper 214 and returned to the bottom position of the carousel 205 due to the curvature of the hopper 214. In some embodiments, a second ridge may rise perpendicular to ridge 216 to protect the coin from falling off ridge 216.
The coin may then pass through an auxiliary sensor 202 such as an inductance coil which can provide information regarding a coin's secondary attributes such as size, diameter, conductivity and weight. In one embodiment, these qualities are measured by applying a multi-frequency oscillating electromagnetic field.
As a coin 204 or object passes through the sensor 202, changes in inductance (from which the diameter of the object or coin can be derived), and the quality factor (Q factor), related to the amount of energy dissipated (from which conductivity of the object or coin can be obtained) are measured. Those skilled in the art will understand that a variety of methods and sensors can be employed to achieve discrimination based on secondary attributes, such as non-image based measurements. This data may be used in conjunction with the processed image data to decide the fate of the coin as well as the user data to be registered and displayed by the central computer 152. The sensor 202 can be connected to auxiliary electronics such as a micro-controller 240 which can perform the necessary processing tasks as well as serve as an interface between the sensor 202 and the central computer 152.
A means for mechanically discriminating the coins, depending on the processed image data and/or auxiliary sensor data, can be employed to separate coins based on predetermined factors. For example, coins may then be mechanically discriminated by a servomechanism 239 (
The entire apparatus 200 may be enclosed such that ambient light is insulated from the imaging devices 207a,b.
In one embodiment, the paddles 309a,b,c,d are pivotally mounted on tension disk pins 321 so as to permit the paddles 309a,b,c,d to pivot in directions 326 parallel to the plane of the tension disk 323. Such pivoting 326 is useful in reducing the creation or exacerbation of coin jams since coins or other items which are stopped along the coin path will cause the paddles 309a,b,c,d to flex, or to pivot around pins 321, rather than requiring the paddles 309a,b,c,d to continue applying full motor-induced force on the stopped coins or other objects. Springs resist the pivoting, urging the paddles 309a,b,c,d to a position oriented radially outward, in the absence of resistance (e.g. from a jammed coin). In another embodiment, a different number of paddles are implemented, such as 6 or 8 paddles, which may cause a smaller number of coins to be advanced such that the entirety of the coins may be imaged by a minimal number of imaging devices.
Similar to the embodiment depicted in
The paddles 309a,b,c,d may be composed of any type of hard material, such as plastic, thermoplastic, glass, metal, wood or some composite. In one embodiment, the paddles 309a,b,c,d are composed of a plastic that prevents the degradation or scratching of the transparent portion of the coin sliding surface 313. In another embodiment, the radially inward portion of the paddle head 317 is composed of a cloth or rubber material that aids in the cleaning and polishing of the transparent portion of the coin sliding surface 313 to maintain the optical integrity of the transparent portion of the coin sliding surface 313. The transparent portion of the coin sliding surface 313 may also be treated with an anti-reflective coating to reduce reflections from lighting.
Coins which are not positioned in the space with their faces adjacent the coin sliding surface 313 (such as coins that may be tipped outward or may be perpendicular to the coin sliding surface 313) may be struck by the paddles 309a,b,c,d as they rotate, agitating the coins and eventually correctly positioning the coins in the annular space with the edge of the coins adjacent the face of the annular space defined by the circular recess 308.
Once coins are positioned along the annular path 308, for example coin 312, the leading edge 350 of a paddle head 317 contacts the trailing edge 352 of the coin 312, forcing the coin 312, and any adjacent coins such as coin 327, along the coin path. In one embodiment, each paddle 309a,b,c,d can move a plurality of coins, such as up to 10 coins. The paddles 309a,b,c,d are connected to a tension disk 323 which is rigidly affixed to a shaft 310 which is connected to a means for generating a rotational force such as a motor, or a computer controlled stepper motor. The motor may be used in conjunction with a gearbox or gear reducer to increase the torque applied to the tension disk 323. The motor may rotate the paddles 309a,b,c,d continuously or in discrete “steps” of specific angular displacement. In one embodiment, the steps are spaced such that the angular distance subtended by each advancement of the paddles 309a,b,c,d is equal to the angular spacing of the paddles 309a,b,c,d. For example, for the particular paddle configuration depicted, the paddles 309a,b,c,d would be advanced in 90 degree increments. In such an embodiment, coins travel in discrete angular advances, such as 90 degrees, then briefly pause for a fixed or variable amount of time. Coins which pause over the imaging area 307 are then captured by opposing imaging devices (not shown) above and below the plane of the coin sliding surface 313.
In one embodiment, the imaging devices and the respective lighting 306a,b,c (for the front imaging device), which can be similar in make and orientation to that of the embodiment depicted in
In another embodiment, multiple imaging devices may be used above and below the coin sliding surface 313 to enlarge the area of the imaging region 307, such that all coins which may be pushed by the paddles 309a,b,c,d through the imaging area 307 can be imaged simultaneously. The imaging device may also be staggered with respect to the coin path as in the embodiment depicted in
Preferably the imaging area 307 is as close to the apex of the annular coin path 308 as possible such that coins stacked edge-on-edge like coins 324 will be singulated along the coin rail 303 in a determinable succession allowing for the mechanical discrimination of coins based on their respective image data by discriminating device 301. This also allows time for any coin which may be stacked on top of another coin side-to-side (or face-to-face) to fall and return to the bottom position of the hopper 314 so that the faces of the coins entering the imaging area 307 are not obstructed upon being imaged.
The coins are eventually forced to travel onto and along the linear portion 325 of the rail disk 322 and subsequently roll onto the coin rail 303, such as coin 304. As the paddles 309a,b,c,d continue to move along the circular path, they contact the linear portion 325 of the rail disk 322 and flex axially outward facilitated by a tapered shape of the radially inward portion of the paddle pad 317 to ride over (i.e. in front of) a portion of the rail disk 322. Singulation of the coins occurs along the linear portion 325 of the rail disk 322 and the coin rail 303, and various design features can be implemented to further facilitate the singulation of coins. In one embodiment, the coin rail 303 may be designed with a wall and gap such that coins cannot fall off the coin rail 303 upon entering the gap; such an embodiment would prevent errors in attributing image data to specific coins for the purpose of mechanical discrimination. The remainder of the coin path (and the embodiment depicted in
In one embodiment, the coin rail 402 comprises a first wall 416, a protrusion, or lip 418, connected to and extending from the first wall 416. The rail 402 may be made of any type of hard material, such as plastic, thermoplastic, glass, metal, wood, or some composite. In one embodiment, the rail 402 is made from hard plastics, with transparent sections made from optical grade, scratch resistant Plexiglass®. In another embodiment, the transparent sections are made of optical grade, scratch resistant glass such as Corning® Gorilla® Glass.
The lip 418 is sufficiently wide so as to allow coins 422 of various shapes and sizes to pass parallel and along the wall 416, as the coin 422 rolls along its edge, without falling over. In the same respect, the wall 416 should be sufficiently high so as to prevent a coin 422 resting on its edge on the lip 418 from falling behind the rail 402. Due to the backward (or transverse) inclination of the coin rail 402, the sides of coins 422 are biased against the wall 416 of the coin rail 402 by gravity. Coins 422 which fall off in front of the rail 402 may be redirected back to the coin pick up device, for example by a hopper, and placed onto the rail 402 again by the coin pick up device.
A first image capture device 404a is positioned adjacent to and directed towards the transparent portion of the first wall 416 to allow the first imaging device 404a to capture the image of a first side (or the obverse side) of a coin 422 passing through the imaging region 424 of the rail 402. A second image capture device 404b is positioned adjacent to and directed towards the transparent portion of the first wall 416 to allow the second image capture device 404b to capture the image of a second side (or reverse side) of the coin passing through the imaging region 424 of the rail 402.
In embodiments in which the material cannot be made transparent, such as wood or metal, a hole may be centered in line with the imaging device 404a or 404b so that the image capture device 404a or 404b can take a picture of the coin 422 as it passes by the hole. In some embodiments, the hole may be covered with a transparent material such as plastic, thermoplastic, glass, and the like to prevent the coin 422 from falling out as it passes by the hole. The transparent material may be easily removed to facilitate cleaning and replacement of the transparent material to maintain the optical integrity of the imaging system. Further, an automated wiping or cleaning system may be employed to maintain the optical integrity of the transparent portion of the coin rail 402.
To facilitate passage of the coin 422 on the rail 402, the rail 402 may be tilted downward from the from the first end 410 of the rail 402 to the second end 412. This allows gravity to pull a coin 422 deposited at the first end 410 of the rail 402 to roll towards the second end 412 of the rail 402. Other means for transporting the coin 422 from the first end 410 to the second end 420 can be utilized, such as a conveyor system as shown in
In some embodiments, the rail 402 may further comprise coin stops. Coin stops are obstructions within the rail 402 that provide a means for slowing or temporarily stopping the coin 422 when it enters the imaging region 424 of the image capturing devices. This will minimize any blurring of the coin image.
The coin stop may be an obstruction created on the lip 418, on the wall 416, or coming down from the top that disrupts the natural traveling rate of the coin 422 through the rail 402. An obstruction may be any deviation from the smooth surface of the wall 416 or lip 418. By way of example only, a bump or void may be placed on the lip 418. A coin 422 traveling over the bump will naturally slow down. A coin traveling into the void may either slow down or become completely immobilized. In some embodiments, friction creating protrusions, such as brushes, may project into the coin path to slow down the rolling coin at the image field 424. If the obstruction is placed within the image field 424, the image capture device 404 can capture the image of the coin as it slows down or stops, allowing for a clear shot.
In embodiments utilizing friction creating obstructions, such as bumps, protrusions, brushes, and the like, the obstructions may be adjustable so that if the coin is stopped, the obstruction can be moved out from the pathway of the coin to allow the coin to resume forward. In embodiments utilizing the void, a movable member may be positioned to penetrate through the void so that if a coin is stuck in the void, the movable member can be inserted into the hole so as to push the coin out and back rolling again.
Movements of the obstructions can be coordinated with the coin advancing mechanism such as the stepper motor so as to avoid multiple coins jamming at the obstruction. For example, as a coin is being deposited onto the rail 402 from the advancing means, an obstruction that has slowed or stopped a coin 422 already in transit can be removed to allow the coin 422 to continue transit.
In some embodiments, no obstructions or coin stops are utilized. The imaging devices 404a,b may be high speed cameras that can capture a clear image of a moving coin 422. Furthermore, the speed of the coin 422 may be adjusted by adjusting the angle of the rail relative to the ground to slow the coin 422 down as necessary depending on the quality of the imaging devices 404a,b.
In some embodiments, a trigger may be set up to time the image capturing process to acquire an image just as the coin 422 passes in front of the imaging region 424 of the image capturing device 404. For example, a beam of light may be directed transversally through, or onto, the wall 416 within the path of the coin 422. When the coin 422 passes through the beam of light to disrupt the transmission of the light, a signal can be sent to the camera 404a to acquire the image immediately or within a specified time. A similar trigger can be set up for, or shared with, the second camera 404b.
To assure the imaging devices 404a,b can capture the entire image of a coin 422, the image field 424 may be broad. However, this can result in a loss of resolution. In some embodiments, once the trigger 426 is actuated, the imaging devices 404a,b can begin capturing a series of images in rapid succession for a period of time. Alternatively, a stop trigger can be positioned downstream of the image capture device such that actuation of the stop trigger stops the image capturing process. The stop trigger, like the acquisition trigger, may be a beam of light, disruption of which causes the image capture device to stop taking pictures. During the processing stage, images in which the entire coin 422 was not captured can be discarded. In another embodiment, the imaging devices 404a,b continuously acquire images.
The lighting 408a,b can be of similar type and variation of make, orientation and triggering as that described above for the embodiments depicted in
The emission source for the lighting elements 421 may be fluorescent, halogen, xenon gas, light emitting diode (“LED”) or the like. In one embodiment, the lighting elements 421 are high current, high intensity, flash-LEDs, due to their longevity, physically robust design, and low heat dissipation.
The lighting elements 421 may be affixed to the hoop 401 by solder, glue, epoxy, mechanical fasteners, or the like. The electrical leads of the lighting elements may be connected in series, parallel or some combination, to an external power source, and/or triggering source, via wires 403.
The hoop 401 may be affixed to an external mounting bracket to fix the position of the lighting 408a,b relative to coin rail 402.
Diffusers may be used in conjunction with the lighting 408a,b to produce greater uniformity of illumination across the imaging region 424. During operation, only a subset of the lighting elements 421 may be operated for a period of time. Upon the detection of a malfunction, expiration or burn-out of a certain number of lighting elements 421 within in a first subset, another, second subset may then be used during subsequent operation. In this way, human intervention is reduced in the replacement and maintaining of the uniform illumination of the imaging area.
This lighting technique, sometimes referred to as “dark field illumination”, is particularly useful as the information to be extracted from the coin's image, e.g. the coin's primary attributes, tends to be embossed and thus highlighted in the acquired images. Examples of a coin's primary attributes include, but are not limited to, date of mint, place of mint or mint mark, inscription or legend (i.e. the portion of the coin on the obverse or reverse sides that tell us important things like who made the coin, Statehood, commemoration information, and denomination), the motto, the portrait, and the like.
Due to the extra space between the lighting 408b and the coin 422 induced by the wall 416, the angle at which light 406b is incident upon the bottom surface of the coin 422 may be different from the angle at which light 406a is incident upon the top surface of the coin 422. The positioning, configuration or manufacturing of the lighting 408b may be different from that of the lighting element 408a so as to correct for the presence of the wall 416 and allow light to be incident upon the coin 422 at substantially the same angle independent of the side of the coin being imaged.
The transparent sections of the wall 416 may be treated with an anti-reflective coating to mitigate reflections from the wall 416.
An auxiliary sensor and discrimination means may be placed downstream of the imaging region 424 of the coin rail 402 similar to that described above for the embodiments depicted in
In one embodiment, the wall 416 may have transverse protrusions, grooves, or be “ribbed”, so as to reduce the contact surface of the wall 416 with the coin 422. These ribs may or may not continue, or extend through the imaging region 424.
The embodiment is comprised of a transparent conveyor belt 501 which may be guided by, and rolls along, rollers 504a,b, in addition to auxiliary rollers (not shown) which can tension, clean and redirect the belt 501, around other hardware so as to complete a loop allowing for the use of an “endless” belt. The rollers 504a,b may be made of any hard material such as plastic, thermoplastic, wood, metal, rubber or a composite. In one embodiment, the drive roller 504a is made of a material which can grip the belt 501 such as rubber. The auxiliary rollers, such as roller 504b, may be on bearings, bushings, or the like, to allow the rollers to rotate freely about mounted shafts or drive axles. The conveyor belt 501 may be made of a pliable, transparent surface such as a high quality, scratch resistant plastic. The drive roller 504a is connected to a drive means, such as a computer controlled stepper motor 505.
Coins, e.g. coin 502, are placed on the conveyor belt 501 by a user, or after having been pre-processed, conditioned, cleaned, etc. by a trommel device, passed over a coin tray, dropped from a chute, vacuumed or the like. The belt 501, driven by stepper motor 505, then advances the coins to the imaging region 507. The belt 501 may be advanced continuously or in discrete “steps” of fixed or variable displacement, with pauses of fixed or variable time between advancements. The imaging region 507 is illuminated by lighting 508a,b which can be of a similar type and variation of make, orientation and triggering as that described above for the embodiments depicted in
In another embodiment, an additional conveyer belt is used to apply pressure to the top surface of coins entering the conveyor system 500. This effectively “pins” or “presses” coins to the lower conveyor belt 501 which may be useful in embodiments in which the belt is advanced in discrete steps. In such discrete-step embodiments, the additional conveyor belt would prevent coins from sliding, which would otherwise do so due to the inertia of the coins, the low friction of the belt 501, and the rapid stop-start motion of the belt.
The coins may be passed through an auxiliary sensor and a means for mechanically discriminating the coins similar to that described for the embodiments depicted in
The stepper motor 703 controls the means of coin advancement (for example, the carousel 203 in FIG. 2A,B, the paddles 307a,b,c,d in
Imaging Devices 704a,b can be connected to the central computer 701 by a variety of interfaces, such as those listed above, in addition to composite, coaxial, and s-video interfaces, or the like. The imaging sensor of the imaging devices 704a,b may be of MOS-type or CCD-type architecture, monochromatic or color, with a plurality of resolutions and frame rates. In one particular embodiment, two Imaging Source DFK-31BUO3 cameras are used as imaging devices, which implement a 1024 by 768 pixel color Sony CCD imaging sensor, capable of capturing 30 frames per second. For some imaging devices, dedicated hardware may be required, such as a frame grabber, to serve as an interface between the imaging devices 704a,b and the central computer 701.
Two distinct apparatuses, lighting 712a,b, are used with the distinction referring to the side of the transparent surface the lighting is disposed towards. The lighting 712a,b are powered by a lighting power supply 710 which may allow for setting the operation and relative intensities of individual lighting elements, or a subset of individual lighting elements, of each of the lighting 712a,b to help achieve uniform illumination across the imaging region.
The auxiliary sensor 706 may consist of a core material with a wire winding about the core, such as a low-frequency and a high-frequency wire winding about the core. The core is disposed along the passageway of coins and is capable of measuring changes in inductance as coins pass the sensor. By analyzing the resulting signal, the denomination and authenticity of the coin can be accurately identified. The auxiliary sensor 706 may be connected to an auxiliary controller 709 which may include the necessary electronics (micro-controllers, oscillators, etc.) to execute the inductive measurements on passing coins. The auxiliary controller 709 serves as an interface between the auxiliary sensor 706 and the central computer 701, and allows for information to be conveyed to the central computer 701 regarding the data obtained from the auxiliary sensor 706 and auxiliary controller 709.
The servo-mechanism 707 activates the mechanical discriminator used to direct coins to different chutes, return trays, etc. The servo-mechanism 707 is connected to a servo controller 708 which serves as an interface between the central computer 701 and the servo-mechanism 707. The central computer 701 can trigger the servo 707 based on data collected from the imaging devices 704a,b and/or auxiliary sensor 706. In one embodiment, the servo controller 708 is connected directly to the auxiliary controller 709 as in
The central computer 701 may be a PC type computer such as one employing an Intel Pentium processor or the like. The computer may run a variety of operating systems such as Windows XP or a Linux based operating system. In one embodiment, the means for capturing and processing the image data collected from imaging devices 704a,b is performed by the central computer 701 using image processing algorithms. Image processing speed may be improved through the use of software optimization libraries such as Intel's Integrative Performance Primitives or Intel's Thread Building Blocks. Those skilled in the art will recognize that the processing tasks (described in detail below) can be executed using a variety of programming languages such as C++, Java, Python, etc. as well as other dedicated computer vision software such as VisionPro© software by Cognex. Processing performance may also be accelerated through the use of multiple (parallel) processors, multi-core processors, graphics processing units (GPUs), and other hardware. In one embodiment, a Dell Optiplex GX620 PC is used with a Pentium 4 HT processor, 2 gigabytes of RAM, running the Windows XP operating system.
The calibration of the electronic components can be done using a variety of methods, procedures and sequences. The calibration settings of certain electronic components may be interdependent on the calibration settings of other electronic components, thus certain steps in the calibration procedure may need to be repeated multiple times until the desired refinements are achieved. In one calibration procedure, the parameters associated with the lighting 712a,b are calibrated first. Depending on the particular lighting configuration, these parameters may include the overall lighting intensity, the relative intensities for multi-element lighting, as well as the pulse duration, intensity and delay for flash-type lighting, and the physical orientation for lighting with adjustable mounting. In many cases an “active” histogram can be used as an aid for achieving uniform illumination across the imaging area. An active histogram is a plot of the number of pixels in an image having a particular value, for example values ranging from 0 to 255. The histogram is updated repeatedly through a succession of images like that from the video source of a imaging device viewing the particular imaging area for which the lighting elements are being calibrated for. By placing testing targets, or “test patterns”, in the imaging area and generating an active histogram for the entirety of, or from various regions of interest within, the images acquired the lighting levels can be adjusted such that the histograms generated show little variation over the imaging area.
If the apertures (the opening through which light is focused onto the imaging sensor) of the imaging devices 704a,b are adjustable, they may also be calibrated. A large aperture will allow more light to be incident on the imaging sensor but will produce a narrower depth of field (the portion of the imaging region that appears acceptably focused in the acquired image). However, a large aperture may also cause significant geometric distortion in the acquired image. A small aperture will typically provide a larger depth of field and less spatial distortion but may require a longer exposure time (the amount of time during which the imaging sensor samples incident light) to produce an adequate signal-to-noise ratio. The optimal setting will depend on the lighting, optics and imaging sensor used.
After setting the aperture, the focus of the camera can then be calibrated. The focus may be adjusted manually (“by hand”) or with an electronically controlled assembly. It might not be possible to bring the entire imaging region into focus due to the aperture setting, thus the aperture may need to be readjusted (generally, made smaller) and the focus calibration repeated. A test pattern containing contrasting regions of various spatial frequency, such as the USAF 1951 Test Pattern, may be placed on the imaging plane and used as an aid for finding the optimal focus. Optimal focus can be achieved “by eye” by examining images acquired successively as the focus setting is altered. In another method, an algorithm can aid in calibration by measuring the contrast of the acquired images of the test pattern. By adjusting the focus setting to maximize the contrast measurement for the test pattern, the image can be brought into optical focus. If the focus setting is electronically controlled, this process may be automated.
The optimal exposure time generally depends on lighting levels, the quantum efficiency of the imaging sensor, and the aperture setting. For embodiments where coins are discretely advanced and thus brought to rest before imaging, the exposure time can be set to acquire the largest amount of light without significant pixel saturation (the point at which pixels cannot register any more incident light). By maximizing the exposure time, the aperture may be reduced which will tend to improve the depth of field and minimize geometric distortion in the acquired images. However, the exposure time should not be set so long that overall processing time is unacceptably lengthened. For embodiments where coins are advanced continuously, the need to mitigate blurring in the acquired images may dictate the optimal exposure time, in which case the aperture may need to be readjusted to achieve the desired signal-to-noise ratio. An active histogram can assist in setting the optimal exposure time such that the highest signal to noise ratio is achieved without significant pixel saturation. For flash-type lighting, the optimal pulse duration and delay time may be dependent on the exposure time and may have to be re-calibrated.
For some imaging sensors the resolution of the acquired images can be changed, typically causing the imaging sensor to operate at a different frame rate. It may be desirable to decrease the resolution of the images being acquired by the imaging sensors to increase the frame rate of the imaging sensors and thus reduce the total processing time. The optimal balance between processing speed and resolution may be set empirically and the processing software can be designed to account for changes in resolution and scaling appropriately.
After operating the imaging devices 704a,b under operating conditions for a period of time, “dark frame” images can be taken by acquiring multiple images with lens caps on the imaging devices 704a,b. The images acquired will produce an estimate of the fixed pattern noise generally arising from the thermal noise and amplifier noise of the imaging sensors. By taking the pixel-wise median of the group, or “stack”, of acquired dark frame images, an estimate of the fixed pattern noise is obtained and can be subtracted from images acquired during operation. This may not be necessary for some imaging sensors due to their quality or design.
For some embodiments, it is desirable for the optical axis of the imaging devices 704a,b to be perpendicular to the imaging plane so consistent images of coins can be acquired regardless of the position or orientation of the coins within the imaging plane. An off-axis camera will generally cause distortion such that circular coins will appear approximately elliptical in the acquired images. In one embodiment, the cameras 704a,b are mounted on fixed hardware which precisely aligns the cameras 704a,b with respect to the imaging plane. In another embodiment, the cameras 704a,b are mounted on hardware which allows for fine adjustments to be made to the positioning of the camera with respect to the imaging plane. To aid the calibration process, multiple images of different coins can be acquired and ellipses can be “fit” to the periphery of the coins (the method by which to do so is described in detail below). For a misaligned camera, the ellipses fit will have some average eccentricity and an average angle of orientation. Using the average angle of orientation of the ellipses, adjustments can be made to the position of the misaligned camera. The process may be repeated several times and as the camera becomes aligned, the average eccentricity of the fitted ellipses should approach zero indicating that the coins are approximately circular and thus the camera is perpendicular to the plane.
Due to imperfections in the manufacturing process, imaging devices may produce spatial distortion in the images acquired, and for some cameras this can significantly affect photogrammetric processing. Reducing the size of the aperture can reduce some distortion, however corrections may still need to be made “in-software'”, this is especially the case if an extreme wide-angle or “fish-eye” lens is used in conjunction with the imaging sensor. A common technique for correcting this distortion is to use multiple images of a test pattern, such as a checkerboard pattern composed of contrasting squares or an array of solid dots arranged with regular spacing in a grid pattern. By comparing points in the acquired images to points in the known geometry of the test pattern, a model of the distortion can be extracted. One common distortion model used is that of Brown (D.C. Brown, “Close-range camera calibration,” Photogrammetric Engineering 37 (1971): 855-866), which assumes the distortion contains a radial and a tangential component. After appropriately modeling the distortion, a geometric transform can be used to correct the distortion from subsequent images acquired during normal operation. This distortion calibration method can also produce estimates of the extrinsic parameters of the imaging device (those pertaining to the orientation of the imaging sensor with respect to the imaging plane) and can be used to make further physical corrections to the orientation of the imaging device as well as in-software corrections via another geometric transform.
The imaging devices 704a,b may become misaligned after prolonged operation of the apparatus due to mechanical vibration, impulses, etc. It is thus beneficial to have a method for correcting the alignment of the imaging devices 704a,b without direct intervention from the user or a technician. One such method for automating alignment correction is to store in memory the parameters of the ellipses fitted to the periphery of coins in the acquired images during normal operation (described in more detail below). As the camera drifts out of alignment, the ellipses which are fit to the images of the approximately circular coins will become more eccentric. By knowing that the ellipses are in fact representations of an approximately circular surface on a plane, one can define a mapping, or geometric transformation, to correct the images taken from the misaligned cameras. For small deviations in the alignment, an affine-type transformation can be applied. For large deviations, a projective-type transformation maybe required, which can be estimated using known methods (see Q. Chen et al., “Camera Calibration with Two Arbitrary Coplanar Circles”, Proc. European Conf. Computer Vision, 2004, pg. 521-532 and M. Lourakis, Plane Metric Rectification from a Single View of Multiple Coplanar Circles, Proc. Of IEEE ICIP, Cairo, Egypt, 2009) which make use of the fact the imaged coins are coplanar circles. Further, the eccentricity of the ellipses fit to the periphery of coins during operation can be used to signal or alert an operator that the imaging device is in need of realignment.
For some embodiments in which the imaging devices 704a,b are triggered by the central computer 701, the position of the coin advancing means may need to be known so images encompassing the complete coin(s) can be captured. One method for calibrating the position of the coin advancing means involves placing a calibration mark (such as a circle, ring, ellipse, star etc.) of known dimensions and possibly color, and with high contrast, on the coin advancing means. Before normal operation of the apparatus, the stepper motor advances the coin advancing means. For embodiments in which the coin advancing means is advanced in discrete steps, the coin advancing means is advanced in small steps (typically subtending smaller displacements than the normal operating steps). As the coin advancing means is advanced, images are acquired and processed such that if the calibration mark is detected in an image, using the known geometric properties of the calibration mark, the location of the calibration mark (its center) in the image is recorded and the coin advancing means is no longer advanced. By having previously determined the trajectory of the calibration mark, the measured location of the calibration mark can be used to determine the orientation of the coin advancing means. For a carousel embodiment like that depicted in
The acceleration and speed of the stepper motor during operation can be set empirically, such that the operation is as quick as possible without causing coins to become dislodged, jammed, jerked, or slide past the imaging area due to the inertia of the coins.
For two imaging device embodiments (one imaging device on each side of the transparent surface on which coins are imaged), it may be difficult to position the imaging devices at precisely the same vertical distance from the imaging plane, thus images taken from one camera may display a scene at a different scale or magnification than images taken from the other imaging device. These differences can be corrected by measuring and comparing the radius of multiple known coins imaged by both imaging devices, to produce an accurate scaling factor. For example imaging device 704a may measure a US Quarter to have an average radius of 270 pixels, whereas imaging device 704b may measure the average radius of a US quarter to be 255 pixels. Subsequent images from imaging device 704a can then be scaled down by a factor of 0.9444 (255/270) during processing to match the scale of images produced by imaging device 704b. This calibration process also determines the overall scale factor used in the image processing of US coins. For example, if in the particular configuration US Quarters have been determined to have an average radius of 255 pixels, this can be considered the standard scale and the radius for other valid coins can then be determined; for example, the radius of US Nickels would then be known to be approximately 223 pixels (smaller than a US Quarter by a factor of 0.874). Similarly, the expected radius of dimes, pennies, etc. can be determined Using the scaling factor, images can be scaled appropriately so they can be compared to templates of fixed resolution. Alternatively, templates can be resized to the scaling factor for the particular setup. Determining changes in the scaling factor during operation can be automated by tracking the drift in the parameters of the ellipses fit to the periphery of known coins. This can help mitigate errors due to changes in the imaging device alignment, which may be a result of mechanical vibration or the temporary removal of the imaging device for cleaning or maintenance.
If more than one imaging device is used per side of the transparent surface the images may need to be “stitched together” into one larger image before being processed. This stitching may be calibrated by placing a test pattern on the imaging plane and using well known point-set image registration methods where points common to the acquired images of the imaging devices are used to determine the proper geometric transformation.
After the imaging devices 704a,b have been calibrated, “scratch images” can be acquired which are images of the transparent surface viewed in the imaging region in the absence of coins. These scratch images can be used to subtract the effects of physical scratches on the transparent coin sliding surface during operation. The scratch images may be used to notify the apparatus, user or service personnel that the transparent surface may need to be cleaned, replaced or toggled such that another portion of the transparent surface is brought into the imaging region. Additionally, the scratch images may be used for background subtraction during operation.
After imaging device and lighting calibration, image processing parameters can be set for the various algorithms used. These may include values dictating processes such as binary thresholding, adaptive thresholding, edge detection and smoothing levels, the specific details of which will be described in more detail below. These parameters may be set empirically by passing coins of known denomination, type, date, and mint and adjusting the parameters to maximize the accuracy in identifying those parameters.
Calibration of the auxiliary sensor 706 and servo-mechanism 707 can be accomplished with known methods specific to the particular devices used.
In the following description, images are considered two-dimensional arrays, or matrices, with each individual element in the array referred to as a pixel. The “depth” of the image is the number of bits used to represent the value of each pixel. A binary image is an image in which pixels can have only two values such as 0 or 1 (black or white, respectively) in the case for images with a depth of 1-bit. For images with a depth of 8-bits, the pixels in so called “binary” images can only have values 0 or 255 (black or white, respectively) which is the convention used for the description set forth below. Grayscale images have a larger range of pixel values than binary images, namely values between 0 to 255 (inclusive) for pixels with a depth of 8-bits. It is worth noting that in the description below, the processes described are not necessarily destructive, which is to say image data is not necessarily lost after undergoing a process, and typically new memory is allocated for the new data, or image, output from a process. Thus images or data input into a process can still be refereed to after the process has occurred, as opposed to the process “writing over” the input image or data.
Processing begins with grabbing images (step 801) from the imaging devices for processing. In some embodiments, only one image may be grabbed and processed at a time because for many coins there is a 50 percent chance that the first image processed will contain all the information needed, namely the denomination, type, date and mint of the coin. If it is determined that the first image grabbed and processed does not contain all the necessary information, then the second image is processed. This technique is useful as it decreases the average processing time per coin. In another embodiment, both images are processed in parallel, and this method will be assumed for the remainder of the description.
Before further processing, it is assumed that the images are in grayscale format. If the images are taken with a color imaging sensor, the resulting color image may need to be de-bayered and/or converted to a grayscale format. A global threshold 802 is then applied to the images where each pixel value of the acquired image is compared to a constant threshold value (typically set in calibration). Pixels with values above the threshold value are set to a high value (255) and those pixels below the threshold value are set to a low value (0), thus producing a binary image.
All the pixel elements in the global threshold images are then summed 803 and the sum is compared to a threshold value 804 (typically set in calibration). If the sum is above the threshold, the image is considered to contain an object, if the sum is below the threshold, then no object is considered to be in the image and no further processing is done. In the case where there is no object detected in the acquired images, the images are discarded (cleared from memory, or “freed”), the stepper advances 805 and the processing chain restarts with a new set of acquired images.
For the case in which the sum of the global threshold images is greater than the set threshold 804, the original acquired grayscale images are corrected for background artifacts, artifacts arising from scratches in the transparent surface if applicable and noise 806. Further, any geometric distortions determined in the calibration process are then rectified 807 in the images.
Images then undergo adaptive (also known as “dynamic” or “local”) thresholding 808, the resulting binary image is used for, among other things, finding the periphery of the object in the image so an ellipse or circle may be fit to the boundary.
Images then undergo contour detection 809, in which boundaries between black and white (0 and 255, respectively) regions are found in binary images. A contour is a list of pixel elements which represent a curve in an image corresponding to a boundary. Contour detection 809 produces a list of contours, which can then be filtered according to the length of each contour such that only relatively long contours are used for the next process of fitting ellipses to each contour 810. Filtering by length of contour saves computation time as small contours generally correspond to noise, reflections, and artifacts in the image as opposed to the periphery of an object such as a coin.
Ellipses are then fit to the length-filtered contours 810 by a least-squares method, rendering a list of the parameters for the “best fit” ellipse for each contour, these parameters include: center of ellipse, semi-major axis length, semi-minor axis length (all measured in pixels) as well as the angle of orientation of the semi-major axis (in degrees) with respect to the horizontal axis of the image. Only ellipses with a ratio of semi-minor axis to semi-major axis near unity are considered good candidates for coins 811 and only ellipses with an effective radius, (semi-minor+semi-major)/2, within the tolerance of a valid coin are considered for further processing 812. If no such ellipses exist, the images are discarded and the stepper motor is advanced 827, and the process is repeated by grabbing the next image 801.
For images containing an object with valid effective radius and eccentricity, the adaptive threshold process may be repeated using different processing parameters. This may be useful as coins of some denominations, and thus radii, may be composed of materials that exhibit different reflectivity and imaging properties. A more robust binary image may be extracted by using parameters in the adaptive thresholding process optimized for such coins. For example a US Penny may have different optimal adaptive thresholding parameters than a US Quarter.
For images containing an object with valid effective radius and eccentricity, the parameters of the fitted ellipse are recorded 813 and are later used for calibration purposes. An elliptical mask is then generated 814 with a region of interest identical to the fitted ellipse and applied to the adaptive threshold image. A mask is a binary image where a region of interest is set to one value (255) and the rest of the image to another value (0). The mask is then applied to an image (such as a grayscale image) creating an new image in which pixels corresponding to pixels of value 255 in the mask image take on the value of the image the mask was applied to. Pixels corresponding to pixels of value 0 in the mask image are set to 0 in the new image. Applying a mask aids in the removal of background artifacts that might still be in the image of the coin after being cropped.
Images are then cropped 815 into images of dimension specific to the coin believed to be in the images. For example, if an object in an acquired image was measured to have an effective radius of 183 pixels, and this effective radius was within the tolerance range for a US dime which has and effective radius of 188 pixels (determined from calibration), the acquired image would be cropped to an 376 by 376 pixel image, the standard set for US dimes, as opposed to a 366 by 366 pixel image.
Another method for determining the location and radius of circular objects in an image is the circular Hough Transform. This method can be used in place of fitting ellipses to contours 810 and may be particularly useful for embodiments in which multiple coins can be contained in one image. The circular Hough Transform can use either an edge detection algorithm (such as algorithms to be described in more detail below) or contour detection to produce a binary image representative of boundaries in the image. In one instance of the circular Hough Transform, an “accumulator space” is created which is a three dimensional array of size m by n by r. Where m and n are the dimensions of the binary image to be processed and r is the number of different radius circles tested to be in the image. For each r, one can imagine a circle of some fixed radius being centered on each (pixel) element in the input binary image. All non-zero (positive) pixels one radius distance away from each element in the m by n binary image are summed and that number is recorded to the respective element in the accumulator space. In this way, edge (or contour) pixels which lie along the outline of a circle of the given radius all contribute to the accumulator space at the center of the circle. In this way, peaks in a plane (one of the m by n sub-spaces) of the accumulator space correspond to the centers of circular features of a given radius in the original image. This method can be robust against noise; however, it generally requires a large amount of computation time and memory. There are variations to the circular Hough Transform which can improve efficiency, and bounding the radius range and resolution can dramatically improve speed.
For embodiments in which multiple coins can be contained in one image, the pixel coordinates for centers of the circles detected in images from one imaging device may be different from the pixel coordinates for centers of the circles detected in images from the opposing imaging device. A grouping method may be needed in order to appropriately group images of the top and bottom of a particular coin. Many grouping methods can be executed; in one grouping method the distance between centers of circles of similar radius from both images are measured and the pairing which minimizes the distance is the considered the correct pairing.
For many of the processes described above and below, computation time may be reduced by using “pyramidal” techniques. By downsampling an image by some set factor before applying a process such as circle detection, the computation time is reduced because there are less pixels which need to be processed. For processes in which geometric parameters are found such as the radius and position of a fitted circle, the parameters may be scaled up by the reciprocal of the factor used to scale down the image before processing. Processing downsampled images typically reduces the accuracy of fitting parameters, thus pyramidal processing may be used for iterative processes in which small scale images are used as first approximations and serve to confine the parameter space for processing at higher resolutions, or at full scale.
Edge detection algorithms may be used for contour detection instead or in addition to adaptive thresholding for subsequent contour detection or other stages of image processing. Those skilled in the art will recognize that a variety of edge detection and edge enhancement techniques can be used such as the use of Sobel or Laplacian operators. In one embodiment, the Canny edge detection algorithm is used for edge detection. The Canny algorithm typically works by first convolving an input grayscale image with a Gaussian or averaging filter to reduce noise in the image. Horizontal and vertical derivatives of the resulting image are then computed using operators such as the Roberts, Prewitt or Sobel operators. From these gradient images the direction and magnitude of edges in the input image are found. The gradient direction is rounded to one of four angles representing vertical, horizontal and diagonal directions; the pixels where these directional gradients are local maxima are candidates for assembling into edges. The Canny algorithm then tries to assemble individual edge candidate pixels into contours. These contours are formed by applying a hysteresis threshold to the pixels of the gradient image, where there are two thresholds, an upper and lower. If a pixel has a gradient larger than the upper threshold, then it is accepted as an edge pixel; if a pixel has a value below the lower threshold, it is rejected. If a pixel's gradient is between the thresholds, then it will be accepted only if it is connected to a pixel that is above the high threshold. Typically good high-to-low threshold ratios are between 2:1 and 3:1. Other algorithm variables to be set include the size of the smoothing filter as well as the size of the derivative operators; larger operators may give better approximations of the directional derivatives. These parameters may also be specific to coins of particular radii or iteratively varied such that a sufficient level of edge detail is produced. Edge detail may be measured by summing all the edge pixels and comparing the sum to a denomination-specific threshold. The resulting image is a binary image with positive regions typically representative of contours of the image.
Images then undergo rotational fitting 817 where the binary edge images (such as those in FIGS. 11A,B) are compared to templates in order to identify the type of coin, which face of the coin is in which image (obverse or reverse) and the rotational orientation of the coin. This also serves to determine whether the object in the image is a valid coin or merely a “slug” or other circular object with the same radius as a valid coin. The effective radius measured in the ellipse fitting process 810 determines what denomination of the coin (e.g. nickel, dime, quarter, etc.) the circular object is a candidate for and thus which set of templates should be used for comparison to the binary edge images.
Within each denomination, templates are produced in advance for obverse and reverse sides of each type of coin expected to be processed. For example, for US Quarters between 1932 and 2008, we have templates for the Obverse and Reverse sides of the US Washington Quarter (
A variety of methods may be used to create templates. In one method, templates are created using control point image registration, where multiple cropped binary edge images of coins of the same denomination, type and face (obverse or reverse) are visually inspected for points corresponding to common features among the images. A program, such as MATLAB, can be used to generate a geometric transform based on the selected points and apply that transform to the group of images such that the images all align with one another. After a group of images for a particular coin denomination, type and face have been registered the images are then “stacked” such that corresponding pixels from each registered image are summed to form a new intensity image, which is then normalized to form a grayscale image. The grayscale image will have high pixel values for features (positive regions) occurring in many of the images and have low pixel values for less common features. The template can then be threshold such that only features occurring more than a set number of times remain in the template image, and those occurring less are removed. This process reduces noise and anomalies in the template image. Alternatively, the template image may be used as a grayscale image or thresholding may be applied to convert the template image into a binary image.
A “rotational set” is produced for each template in advance. A rotational set is composed of multiple images of a template rotated about the center of the template in discrete angular displacements. The range of the angular displacements can vary from 0 to 360 degrees and various sizes of angular spacing between displacements can be used, for example, in one embodiment the rotational sets consist of 180 images of each template rotated in 2 degree steps, in another embodiment the rotational sets consist of 90 images of each template rotated in 4 degree steps. In the embodiment described herein, all US Washington Statehood Quarter rotational templates consist of templates rotated in 4 degree steps; the reverse US Washington Quarter, reverse US Washington Bicentennial Quarter, and the obverse US Washington Statehood Quarter templates depicted in
For templates which are binary images (images with pixels having values of only 0 and 255), some algorithms which produce artificial rotations render grayscale images (images with pixels having values between 0 and 255) due to the interpolation method used, such as bi-linear or bi-cubic interpolation. Other interpolation methods can be used to preserve the binary nature of the templates such as nearest-neighbor interpolation, however, interpolation methods producing grayscale images provide better matching results. By creating rotational sets of the templates in advance, processing time is saved because computationally intensive interpolation does not need to be performed during operation. In one embodiment, the rotational sets for each template are loaded into memory, such as RAM, to improve computation time as opposed to using hard-disk memory storage which tends to have longer access times.
Each image in the rotational set for each template appropriate to the measured radius is then matched to the binary edge images produced in the edge detection step 816. The image with the best match renders the rotational orientation, the type and face of the coin in each image. Template matching can be done a variety of ways, in one embodiment a normalized correlation coefficient method is used. The normalized correlation coefficient matching method operates such that a perfect mismatch between the template and binary edge image will result in a match index of −1, a perfect match will result in a match index of +1; and a value of 0 means there is no correlation between the template and image (i.e. there are only random alignments among the pixels).
For normalized correlation coefficient matching each image in the rotational sets for each template are converted to “signed” grayscale images which allow pixels to have values ranging from −255 to 255. For each image, the mean pixel value of the entire image is calculated and subtracted from each individual pixel such that the resulting image is an intensity map relative to the mean of the original image. The preparation of such “mean-corrected” template images may be done in advance to conserve computational resources during normal operation. Similar to templates, during operation a mean-corrected image is produced of the acquired binary edge image to be matched, referred to hereafter as the “mean-corrected target image”. For each mean-corrected template in a rotational set, the match index is found as a function of rotational orientation of the mean-corrected template using the equation (Eq.1):
where θ is the angle of template rotation, match(θ) is the measured normalized correlation coefficient, or “match-index”, T(θ) is the mean-corrected template image from the rotational set for the particular coin type being fit, I is the mean-corrected target image, and the multiplication operator * denotes pixel-wise multiplication between two images and denotes scalar multiplication when between two scalars.
During operation, the rotational sets of template images may be loaded into memory (such as RAM) in one contiguous “image vector”, such that all template images can be called, or retrieved, using a single index. All the templates of all the rotational sets can be loaded into one contiguous space of memory, which allows for faster processing. For example, the image vector for the US Quarter contains 4950 images from all the template images in all the rotational sets for that coin denomination. Further, to increase processing time, the templates and the target images can be downsampled to lower pixel resolution to increase computational efficiency, for example all the US Quarter templates and target image are reduced by a factor of 8, from 756 by 756 images to 90 by 90 images.
For embodiments in which the pair of acquired images are processed in series, if it is determined 820 that the first processed image contains the face of the coin which contains no information, such as date and mint, then the processing chain can be restarted with the other, opposing image 821.
For embodiments in which the pair of acquired images are processed in parallel, or both images are processed serially prior to further processing, “coin logic” or redundancy can be implemented such that if one image is a heads the other should be a tails, and the rotations of those coins should be strongly correlated, if not the best matches that make logical sense can be used instead of the best matches selected independent of the other match results. For certain embodiments, it is conceivable that coins can be stacked on top of each other during imaging, in which case there can be contradictory matching results, such as a two headed US Quarter, in these cases the images may be rejected, or if possible, information such as date and mint may still be retrieved.
In the example shown, the images (correctly) correspond to the obverse and reverse side of a US Washington Quarter at 84 and 264 degrees respectively. Therefore, the rest of the relevant information sought (date and mint) is on the first image,
The digits target,
In one embodiment, a template matching method is used to achieve character recognition. For this method, digits templates for each coin denomination and type can be formed using a template creation method similar to the control point image registration method described above for building the templates used in rotational fitting. The digits templates may be modified to eliminate any features which may be shared with other template digits. The digits templates, such as those shown in
The digits target is compared with each of the digits templates, for the date ranges of the respective coin denomination and type, using a normalized correlation coefficient method. Further, for each digits template there are additional digits templates of the same digits template, only rotated by some tolerance. For example, in one embodiment each digits template consist of a set of the original digits template and four additional digits templates rotated by −4, −2, +2, +4 degrees, for example, the rotation sets for the “98” digits template are shown in
In one embodiment, the digits target and mint mark target are “padded” with zero value pixel elements around the border of the images to allow for greater translational variations between the templates and the target during the matching process. In another embodiment, the digits target and mint mark target are generously cropped, with the region cropped significantly larger than the desired features contained within the cropped image.
where R(x,y) is the matching array in which each element indicates the normalized correlation coefficient, or match-index, between a particular mean-corrected digits template I and a mean-corrected digits target T at the relative displacement x,y. The elements of R(x,y) can take on values between +1 for a perfect match and −1 for a perfect mismatch; x′ and y′ are “dummy” variables for the purposes of referencing pixel elements in the summation, and the multiplication operator * denotes pixel-wise multiplication between two images and scalar multiplication when between two scalars.
For each matching array created from matching a digits template to the particular digits target, the maximum element in the array is extracted indicating the best fit achieved for each digits template. These values are complied into a vector, the resulting vector from matching the digits templates of
To decrease the likelihood of misclassification of target digits various augmentations can be made to the template matching method. For example, likelihood criteria can be applied to the results in the matching vector, weighted by the empirically determined likelihood of finding a particular date in circulation, for example a 1944 US Washington Quarter is more unlikely to be found than a 1995 US Washington Quarter.
In one embodiment, grayscale images of coins are enlarged to higher resolution using an interpolation method, such as bi-cubic interpolation, and then the enlarged grayscale images undergo adaptive threshold to become binary images. These binary images are then used for segmentation and character recognition, typically achieving more accurate recognition. This method can be used in addition to processing at normal scale. In one embodiment, if the match levels among template digits are close to one another, or certain threshold parameters are not met such as signal-to-noise ratios, then processing occurs at higher resolution and with a subset of template digits. These methods can be particularly useful for smaller coins, such as US Dimes, which may exhibit smaller features than larger coins, such as US Quarters.
Topological features of the digits target can be used to further weight certain digits templates in the matching vector or reduce the subset of possible digits templates. Such features may include topological “holes” which are closed loops such as those found in “8”s, “6”s, “9”s, “4”s, and “0”s.
A corner detection algorithm may be applied to the image, such as a version of the Harris corner detection algorithm, and corners (number of, sharpness of, location of, etc.) can be used as another classification feature.
The “moments” of the digits templates and digits target may be compared such as centers of “mass” and distribution of “mass” of the images or collections of moments can be compared, such as “Hu moments”, to define another metric for measuring the quality of match between a digits template and digit target.
In one embodiment, the results in the matching vector, as well as any additional information such as the distances between centers of mass, distance between other moments, topological measurements, etc. are to be put into a “master” table and a machine learning algorithm is used to determine an appropriate weighting scheme for each feature to produce robust digits recognition. There are many such machine learning methods which may be implemented, many involve having a large “training set” of images with previously identified digits from which the algorithm iteratively determines the most effective weighting scheme to maximize matching accuracy. The “trained” weighting scheme is then used during normal operation.
Similar matching algorithms as those described above can be used for matching and identifying the mint mark target. It is possible for some coins for there to be no mint mark, thus the match-index for mint mark matching or some other criteria may have to be above a certain threshold to indicate that there is in fact a mark. The result of the image processing described is an ASCII string containing the denomination, type, date and mint of each imaged coin. For example, the ASCII string produced from processing the images in
In one embodiment, the ASCII string containing the coin attributes is used by a “front-end” graphical user interface to present the coin attributes to the user on a touch-screen display. Examples of one particular graphical user interface is shown in
Absent from current coin counting and sorting devices is a means for informing the user of the primary attributes of coins deposited, nor is there a means for presenting such information in an entertaining and engaging manner. The coin display feature 2001 is used to organize and communicate the coin data acquired during operation to the user in an intuitive, entertaining and engaging manner. In one embodiment, the coin display feature 2001 consists of a grid with a plurality of coin vacancies 2002 with a date 2040 of the respective coin below each coin vacancy 2002. In this particular embodiment, each row of coin vacancies 2002 corresponds to a particular decade of coin dates 2040. In one embodiment, within each coin vacancy 2002 there is a loyalty point value 2003 or graphic 2004, such as a corporate, charity or organization logo indicating an award, bonus, coupon, donation, merchandise or prize, or other promotional value, that the user would receive for having deposited a coin with the corresponding date of the coin vacancy 2002 enclosing the graphic 2004.
When a coin is deposited and the coin's primary attributes extracted (denomination, type, date, and mint) using the methods described above, the coin is “registered” and the corresponding coin vacancy 2002 is filled, notifying the user that that particular coin has been deposited. In the embodiment depicted in
In the embodiment shown in
The user may also select to view other coins of the same denomination, such as older or newer dates and types of coins, by using the navigation buttons 2009a,b to toggle between other coin grids. For the particular screen shown, the user may select the left most navigation button 2009a to view older US Pennies or select the right most navigation button 2009b to view different types of Lincoln Bicentennial Pennies. A coin grid page indicator 2041 may be used to indicate the current coin grid being viewed relative to the other coin grids. The user may also view and explore other denominations of coins by selecting one of the denomination tabs 2010a,b,c,d. Each denomination tab 2010a,b,c,d indicates the denomination 2011 and the total number of coin of that denomination registered 2012. The user may gather further information about each coin shown by selecting or actuating the coin image 2013. Such information may include the origin of mint, images of both sides of the user's actual coin, how many of the selected coin were minted, how many times the select coin was registered during the deposit, or during the history of the apparatus, the odds or probability of finding the selected coin in circulation, the number of loyalty points awarded for the selected coin, etc. Different coin images 2013 may be used to represent and communicate the quality of the coin registered, for example a more worn coin may be represented by an image of a coin with less luster and of a different general color. The total monetary value of the coins registered may be indicated in a separate coin total box 2014. Users may select the information button 2015 during the coin deposit process to view information such as an explanation of the features of the user interface screen 2000. When a user has deposited all of their coins, the user may select the exit button 2016 to indicate the completion of depositing coins. A user may identify themselves using a loyalty card, or the like, prior to, or during coin deposit. In one embodiment the username 2042 of the user is displayed on the screen 2000 during coin deposit.
From the Start Screen 2101, users who wish to initiate a transaction are taken to a Pre-Transaction screen 2103 where the user is notified of any options, fees, terms and conditions for the service. If the user proceeds, the user is taken to a Transaction Screen 2104, such as the screens shown in
Users who are logged-in patrons may view their coin progress in a Progress Screen 2108, which allows users to view the various denomination, types, dates and mints of the coins deposited over the course of their transactions. In one embodiment, the progress information is organized in the form of a virtual coin book similar to that shown in
The data stored in the Central Data Base 2208 may also be accessed by users while not at the kiosk 2201, for example users may use a computing device, such as mobile devices 2209 or computers 2210, to access the Central Data Base 2208 via the internet 2208 to view their coin data, find out about promotions, trade virtual coins with other users, post versions of their coin data, or progress, badges, awards to social media outlets, social networks and websites, view statistics, leader boards, and the like.
The kiosk 2201 may also be connected to a host retailers' loyalty system or Point of Sale (POS) System 2206, which may also be accessed by registers 2207 or tellers at the same location as the kiosk 2201. This may be used to register the amount of the transactions as promotional value, such as points, prizes, awards, coupons, vouchers, or the like, earned at the kiosk 2201.
The kiosk 2201 may collect user's information and identify users using a unique or already issued loyalty card 2202, identification Card, bar code, magnetic strip, RFID, password 2203, electronic mailing address, mobile device 2204, near-field technology device, or the like. The kiosk 2201 may interact with a user's mobile device 2204 to update information, such as transaction data, coin data or any derivative data, for example via a software application running on the mobile device 2204. Information acquired from users (including information regarding the coins deposited) will allow users to interact with each other with their respective computing devices or mobile devices 2204 to exchange information, such as transaction data, coin data, any derivative data, contact information, and the like to foster discussion, trading, etc.
This invention may be industrially applied to the development, manufacture, and use of a coin identification apparatus and method for identifying and sorting coins based on primary attributes and/or secondary attributes, and displaying the results in an entertaining and engaging manner. The apparatus comprises a tray 101 into which coins are loaded; a coin pick-up assembly 107 operatively connected to the tray 101 into which the coins are deposited from the tray 101; an imaging device 207 to acquire an image data selected from the group consisting of a denomination, a type, a date, and/or an origin of mint; a computer specially programmed for processing the image data; a means for mechanically discriminating the coins based on the image data, causing the coins to be routed into one of a plurality of bins 109; and an output device to display at least one primary attribute of the coins in graphical form. The graphical representation of the coin data can be presented in animated form to entertain the user as the data is updated in real time.
| Filing Document | Filing Date | Country | Kind | 371c Date |
|---|---|---|---|---|
| PCT/US11/50719 | 9/7/2011 | WO | 00 | 8/8/2012 |
| Number | Date | Country | |
|---|---|---|---|
| 61383298 | Sep 2010 | US |
