Patent Grant
9400909
This invention relates generally to the identification of objects using printed identifiers, and more particularly to the identification of small, uneven, differently shaped, or other objects, such as medication pills, using one or more identifiers embedded in a fractal-based printed identifier to create a visually complex fingerprint.
A counterfeit medication or a counterfeit drug is a medication or pharmaceutical product which is produced and sold with the intent to deceptively represent its origin, authenticity or effectiveness. A counterfeit drug may contain inappropriate quantities of active ingredients or none at all, may be improperly processed within the body (e.g., absorption by the body), may contain ingredients that are not on the label (which may or may not be harmful), or may be supplied with inaccurate or fake packaging and labeling. Counterfeit medicinal drugs include those with less or none of the stated active ingredients, with added, sometimes hazardous, adulterants, substituted ingredients, completely misrepresented, or sold with a false brand name. Otherwise legitimate drugs that have passed their date of expiry are sometimes remarked with false dates. Low-quality counterfeit medication may cause any of several dangerous health consequences including side effects, allergic reactions, in addition to their obvious lack of efficacy due to having less or none of their active ingredients. Medicines which are deliberately mislabeled in order to deceive consumers—including mislabeled but otherwise genuine generic drugs—are counterfeit.
Since counterfeiting is difficult to detect, investigate, quantify, or stop, the quantity of counterfeit medication is difficult to determine. Counterfeiting occurs throughout the world, although there are claims that it is more common in some developing countries with weak regulatory or enforcement regimes. It is estimated that more than 10% of drugs worldwide are counterfeit, and in some countries more than 50% of the drug supply is counterfeit. In 2003, the World Health Organization estimated that the annual earnings of counterfeit drugs were over US$32 billion.
The considerable difference between the cost of manufacturing counterfeit medication and price that counterfeiters charge is a lucrative incentive. Fake antibiotics with a low concentration of the active ingredients can do damage worldwide by stimulating the development of drug resistance in surviving bacteria. Courses of antibiotic treatment which are not completed can be dangerous or even life threatening. If a low potency counterfeit drug is involved, completion of a course of treatment cannot be fully effective. Counterfeit drugs have even been known to have been involved in clinical drug trials.
Medication Identification
In addition to the problem with counterfeit medications, simple identification of medication is also an extremely large problem. More than 80% of adults in the U.S. take at least one pill a week, whether prescription, OTC, vitamin or herbal. Yet the pills they are taking are difficult to identify based on their visual characteristics alone. Pill identification, or the inability to correctly visually identify a pill, is a large contributing factor to medication errors. These errors can occur anywhere along the drug-taking process. Difficulty with pill identification is further exacerbated when patients are older, have some form of impairment, take multiple drugs or have limited health literacy. 1.5 million people are harmed each year because of medication errors The cost of treating drug-related injuries in hospitals is approximately $3.5 billion per year. The actual number of medication errors is presumably much higher since not all medication errors lead to injury or death. A pill's poor labeling and packaging are thought to cause one third of medication errors, while studies have also shown that a pill's shape and color are important factors in drug identification.
Existing Identification and Anti-Counterfeiting Technologies
There are several technologies that have been employed in an effort to combat the counterfeit drug problem, and to allow for identification of medication. An example is radio frequency identification which uses electronic devices to track and identify items, such as pharmaceutical products, by assigning individual serial numbers to the containers holding each product. The U.S. Food and Drug Administration (FDA) is working towards an Electronic pedigree (ePedigree) system to track drugs from factory to pharmacy. This technology may prevent the diversion or counterfeiting of drugs by allowing wholesalers and pharmacists to determine the identity and dosage of individual products. Some techniques, such as Raman spectroscopy and Energy Dispersive X-Ray Diffraction (EDXR) can be used to discover counterfeit drugs while still inside their packaging. Other more traditional systems may be applied to such medication identification, such as barcoding being provided on medication packaging (either one or two dimensional). For such a use, however, any damage to the barcode, difficulty in printing the barcode (such as deformation based upon printing surface), or obscuring a portion of the barcode may render the barcode inoperative.
Marking individual pills with one or more identifiers is considered a useful method for identification, but has been traditionally thought of being cost prohibitive while offering only minimal improvement over packaging marking. One or more barcodes may be employed (either one or two dimensional) and may be printed to individual medication pills, instead of, or in addition to being printed to the medication packaging. Such a printing process may be implemented by employing one or more appropriate printing apparatuses, such as a pad printing apparatus provided by Printing International® N.V./S.A., for example. Thus, each pill may be individually printed with the use of such a pad printing apparatus. Laser marking has also been used to print high-resolution images or barcodes directly onto pills. In consumable products, Mars®, Inc. utilizes inkjet or pad printers to print images cheaply onto individual pieces of candy. Indeed, U.S. Pat. No. 7,311,045 describes a system for printing multi color images on a candy by maintaining a directional registration of the candy between printing steps. In each instance, holding each individual medication pill or candy is performed by vacuuming the pieces in place, and holding the piece firmly in place between steps so that orientation of the piece during printing does not change. Other patents and applications assigned to Mars®, Inc. describe a number of systems and methods for printing food grade inks onto shaped candy elements.
While one or two dimensional barcodes have been used to serialize individual pills and verify authenticity and identity, but as recognized by the inventors of the present invention, their designs are relatively easy to replicate, require fixed surface areas and specific alignment for printing, and are rendered unusable if occlusion occurs due to handling or if the barcode is damaged. Unlike forensic features, which are embedded into an item, in barcode technology the item's physical attributes are completely distinct from the barcode itself. Further, whether using such a pad printing process, or employing other printing methods such as ink jet printing or laser marking for imparting markings to candy or medication pills, the inventors of the present invention have recognized that the need for purposefully handling individual pills may be time consuming and expensive. Further, the described printed elements may fail to provide robust images sufficient to act as a unique identifier for a particular batch of processed elements. For such a use, as noted, any damage to the barcode, difficulty in printing the barcode (such as deformation based upon printing surface), or obscuring a portion of the barcode may render the barcode inoperative. Additionally, barcodes may be easily copied and applied to counterfeit objects. None of these systems are sufficient for imparting robust identification information to a pill or other candy object.
Similar problems of identification of other products or objects, such as consumer products and the like, may also arise. While holographic printing on hang tags and the like has been employed in an effort to mark such objects, and to perhaps stop counterfeiting of these objects, these tags may be removed and possibly copied as printing on a single tag may not be seen as a particularly difficult deterrent. Thus, not only is secure identification impossible, varying levels of desired security cannot be employed.
Therefore, it would be desirable to provide a method and apparatus that overcome the drawbacks of the prior art.
In accordance with one or more embodiments of the present invention, a standardized process for labeling and identifying medication and other objects is provided, and in particular may comprise a system and method for printing an identification pattern to a plurality of regularly or irregularly shaped and arranged objects. In particular, a fractal pattern is preferably printed as the identification pattern onto a plurality of medication pills or other objects. At a later time, computer vision may be applied to read these applied fractal labels to automatically confirm identification and authenticity of the pills or other objects, irrespective of orientation, partial occlusion, or partial damage of the printed fractal image. Finally, different levels of fractal dimensions (the number of times the fractal pattern is recursively printed within itself) may be printed and therefore read from these fractal images employing different resolution imaging devices, providing varying levels of security and precision in the identification process while allowing for ease of identification and a reduced usability burden. Thus, imaging devices with lower resolution imaging capabilities may be able to resolve one or two fractal dimensions and provide a lower, less expensive, consumer oriented level of authentication, while higher resolution imaging devices may be able to resolve seven or more fractal dimensions, thus providing additional security where desired.
Security Approaches
The inventors of the present invention have recognized that four schemes have typically been used by pharmaceutical manufacturers to identify and authenticate medication.
Fractal Encoding
A fractal is a rough or fragmented geometric shape that can be split into parts, each of which is (at least approximately) a reduced-size copy of the whole, a property called self-similarity. A fractal often has the following features:
It has a fine structure at arbitrarily small scales.
It is too irregular to be easily described in traditional Euclidean geometric language.
It is self-similar (at least approximately or stochastically).
It has a Hausdorff dimension which is greater than its topological dimension (although this requirement is not met by space-filling curves such as the Hilbert curve).
It has a simple and recursive definition.
Because they appear similar at all levels of magnification (at each printed dimension), fractals are often considered to be theoretically infinitely complex (in informal terms).
By encoding one or more pieces of identification information into such a fractal through the use of one or more predetermined fractal type, color, or other fractal parameter, and printing or otherwise etching the resulting fractal onto a plurality of medication pills or other objects, a robust and secure medication or object identification scheme may be provided. Varying resolutions and complexities of such fractals may be employed in order to impart desired levels of security. More complicated fractals having a greater number of dimensions (and thus allowing an imaging device with higher levels of resolution to recognize the fractal pattern more completely at these greater levels of resolution may be employed for use with objects needing higher levels of security, as these more complex fractals are more difficult (or impossible) to copy, requiring at least higher quality printers and readers that may not be easily available. As further recognized by the inventors of the present invention, a low-cost tool such as a webcam combined with computer vision software may be provided to a patient to properly image and identify a medication at a lower level of security.
Once applied, the fractal image provides a robust identification system that is resilient against identification when partially occluded, or when printing is imperfect because of object shape, position, surface texture or the like. Thus, one or more predetermined characteristics of a pill or other object may be employed to be used as part of an identifier for the object. In such a manner, not only is the fractal image used to prevent against counterfeiting as it is difficult or reproduce, but further various characteristics of the object, such as color, shape, texture, markings and the like, may combine with such a fractal image to produce a unique fractal/object characteristic combination. As the characteristics of the fractal alone are known, the fractal image may act as a calibration tool to determine any influence the color, for example, of the pill may have on the actual fractal color, thus allowing for an accurate determination of pill color. In such a manner, these noted characteristics of the object may cause one or more distortions in the shape, color, or other attribute of the printed fractal image, these distortions being potentially resolvable and recognizable at differing desired imaging resolutions, providing differing levels of security. While a pill may also distort barcodes and other printed images, barcodes include thick lines and may hide any details of geometry changes. It is the unique structure of a fractal, and its multi-dimensional, intricate structure, that makes it ideal for performing this task as less of the pill surface is obscured, allowing for additional opportunities for measuring such distortion and other subtle geometric changes in a medication pill or other item. The use of reference points in such a printed fractal image, and expected distances therebetween or orientations thereof, may thus allow for precise determination of distortions of the image based upon printing techniques and pill shape.
Furthermore, because of the self replicability of fractal images, the application of these unique fractal identifiers may be generated that may be applied substantially simultaneously to a large group of medication pills or objects without regard to orientation or relative positioning of the objects during printing. The resulting identification images are robust even if portions of the printed images are not properly printed, or are damaged, obscured or otherwise occluded. Thus, during a preferred printing process, these pills or other objects need only be maintained in approximately a single layer during the printing process. Strict orientation and arrangements of the medication pills during printing or subsequent imaging is not required (in that fractal patterns are recursive patterns that may be viewed at any level of detail and give the same information). Additionally, the entire portion of the fractal need not be properly printed on the pill, providing additional robustness in the printing process. Thus vacuuming of the pills in a particular orientation for printing is not necessary, allowing a relatively disorganized set of pills to be printed at one time.
Additionally, multiple fractal patterns may be overlaid, thus producing a more complex identification image. These overlaid fractals may be electronically combined before printing, thus requiring a single printing pass, or may use multiple printing passes, thus, printing multiple fractals at different times on the pills. These overlaid fractal patterns may also be selected to provide different imaging results, such as a first fractal image allowing for a more accurate measurement of shape and color of the medication pill, and another fractal image providing various medication information. Further, by providing more complete coverage of a pill surface, while simultaneously perhaps allowing substantial portions of the pill surface to be viewable along with the fractal image, the geometry of the pill and any unique or identifiable geometric characteristics may be more accurately determined through a measurement of distortion of the fractal images by the shape of the pill.
Therefore in accordance with one or more printing mechanism embodiments of the invention, because any portion of the fractal image is sufficient to provide all of the information, a conveyer mechanism may be provided for forwarding, in either a continuous or batch processing manner, a plurality of medication pills to a printing area, in addition to the use of the individual pill printing schemes of the prior art. If forwarded in a batch manner, a fractal image is preferably printed on the area containing the medication pills in a manner employing, by way of example only, ink jet printing technology of the type described above with reference to the '045 patent, the contents thereof being incorporated herein by reference. Other printing or etching technologies, such as laser etching, laser marking, photographic exposure, chemical etching, photolithography, solid ink printing or the like may be employed, each preferably providing different and varying possible resulting resolutions for offering differing levels of security. Additionally, one or more medication pills may be coated with a laser or light sensitive material that allows for marking of the pill, while being consumable by humans. However, unlike the '045 patent, rather than managing objects piece by piece, in accordance with the present invention, because of the nature of fractal images, a single image may be provided to the plurality of medicine pills at one time. In such a manner, each pill will receive only a portion of such an image. Because of the self referential nature of fractals, this part of the image will be sufficient to provide an identifiable amount of information that may be recovered with computer visual recognition technology. Of course, batches of such pills may be manually or otherwise placed in a printing area, and removed after printing.
If forwarded in a continuous manner, a fractal pattern that may have bounds in the direction across the direction of travel of the plurality of pills, but is continuously repeatable in the direction of travel, may be applied. In this manner, a very large number of such pills may be processed continuously and inexpensively, and therefore in accordance with other continuous manufacturing and processing of such pills.
While the invention is described as relating to medication pills, the invention may be applied to any objects in which such an identification application may be beneficial, including any type of small, uneven, or irregularly-shaped object part, including for pricing, and anywhere a traditional bar code or two dimensional bar code may be employed, and where the integrity of the item makes it important to track and identify the object.
Still other objects and advantages of the invention will in part be obvious and will in part be apparent from the specification and drawings.
The invention accordingly comprises the several steps and the relation of one or more of such steps with respect to each of the others, and the. apparatus embodying features of construction, combinations of elements and arrangement of parts that are adapted to affect such steps, all as exemplified in the following detailed disclosure, and the scope of the invention will be indicated in the claims.
For a more complete understanding of the invention, reference is made to the following description and accompanying drawings, in which:
The detailed embodiments of the invention will now be described making reference to the following drawings in which like reference numbers denote like structure or steps.
In accordance with an embodiment of the invention, an ink jet, laser marking system, or other printing or etching system may be employed in order to print a plurality of medication pills with a predetermined fractal image. Other acceptable printing systems may also be employed, and may include, by way of example only, and without limitation, laser printing, laser etching, photographic imaging, photolithography techniques, solid ink printing, or the like. One or more known edible ink products may also be employed in the printing process. Invisible, UV sensitive, heat sensitive, and other appropriate inks may be employed. Each medication pill may further be coated with a laser or other light sensitive or otherwise sensitive coatings that, when caused to react, may be employed to mark the medication pills, while remaining edible by humans. Furthermore, single or multiple color printing may be employed.
Fractal Printing
Referring first to
As is further shown in
The printing of fractal patterns is not limited to pill-shaped medication. As is shown in
Fractal Resolution
In accordance with one or more embodiments of the present invention, the use of high resolution printing and imaging techniques may be employed when more robust security measures are required, increasing the difficulty of copying such an image, and leading to greater accuracy in identification of each item. Determinations of the maximum resolution of such printed fractals may be identified by building such fractals from a smallest possible printed pattern, thus bounding the lowest fractal resolution. Alternatively, such fractal images may be generated in a traditional form, by starting with a largest fractal image, and then dividing these larger images into smaller and smaller objects, until a predetermined limit or printing resolution is reached. Such increased resolution may also allow for more precise measurements of the details of the fractal image, such as one or more distances between various portions thereof, ratios of one or more various measured lengths or angles of various portions thereof, etc., texture of the medication pill surface, color of the medication pill surface, etc. which may then be compared to expected values to confirm authenticity and identification.
Fractal Complexity
Furthermore, various types of fractals may be employed based upon a desired level of security. Thus, more complex versions of fractal images may be employed where a more secure identification system is desired. Variations in such fractal images may include changes in angles, length, number of pixels employed, distribution of one or more characteristics or pixel density, purposeful omission of particular pixels, use of particular color combinations on a planned or randomized basis. Not only may generally more complex images be used, but a higher resolution printing process may also be employed, thus allowing for more precise printing of multiple fractal dimensions, and eventual recall and analysis of the fractal images by higher resolution imaging devices at deeper acquisition resolution. Furthermore, as noted above, combination fractals may be employed to provide additional robustness against counterfeiting, and for determining identity. These combined fractals may be particularly chosen to allow for determination of different types of information. Thus, a first or more fractals may be employed to measure for detection of distortion that may be a result of the shape of the pill, while a second or more fractals may be employed for encoding information and for prevention of replication of the fractal images. Additionally, various color gradient application may allow for the calibration of the fractal image, the pill, or other object.
Referring next to
After printing such a medication, in order to properly identify the pill, imaging steps may be employed. As is further shown in
Thus, as is further shown in
Fractal Deformation
As is next shown in
Thus, recognition of predictable distortion of the fractal image and additional measures to avoid such copying may also be employed in accordance with various embodiments of the invention. In particular, a fractal pattern may be calibrated to include one or more of object pattern, shape, texture, markings, line thickness (such as through the use of thicker inkjet lines, or by altering a wavelength of a marking laser, for example), or the like. As noted above, by including one or more aspects of the object in the coding scheme and training an imaging system to recognize these expected fractal distortions, in order to counterfeit the object, not only must the counterfeiter precisely copy the fractal image, but must also produce an object nearly identical to the genuine object with respect to any number of attributes, any of which may pose difficulty. As is further noted above, various calibration lines or the use of symmetrical fractal patterns may be employed for distortion detection. Thus, for example, incorrect object color may change an overall color of the fractal image applied thereto, thus indicating a non-authentic object. If printing resolution is reduced based upon curvature of a pill or other object (and therefore a change in distance from the print head), ink may be distributed in a known manner, creating a unique signature and allowing for any recognition system to better determine the shape of the object more accurately. In an additional attempt to provide a difficult to copy image, as noted above, a plurality of fractal images may be overlaid on an object, thus making copying even more difficult as various interactions between the various images may be more difficult to determine.
In addition to including various features of the object in the fractal definition, these attributes may influence perception of the fractal image by an imaging system. Thus, by being printed on a curved surface, for example, a fractal image may be deformed or otherwise influenced in predictable ways, thus allowing for the user of such shape to be employed to further differentiate authentic objects. Because orientation of objects is not necessary in accordance with embodiments of this invention, deformation of such a fractal pattern may be determined in a number of likely orientations of an object, and then may be so classified and found on printed objects. Further, if orientation of the object can be controlled for printing, then precise fractal deformation may be determined. As such imaging systems may learn such expected fractal deformation, the deformation may be employed as part of the object identification system.
Fractal Coding
In accordance with one or more embodiments of the invention, information may be coded into the fractal, by placing such coding information into one or more parameters that may be stored in a parameter file used to generate the fractal. Thus, a batch number or the like may be used in the place of particular parameters to be chosen by the user. Recognition of the fractal, and reverse engineering thereof to recreate the parameter file may then provide access to the batch number or the like by the user, for example. It is anticipated that only a predetermined number of parameters may be employed for particular coding, others of the parameters being adopted to vary in a pseudo-random or other predetermined manner in order to make it more difficult to predict future likely parameter combinations. Other information that may be encoded in such a fractal may comprise one or more of Medication Name, Dose, Manufacturer, Date of Manufacture, Expiration Date, Location or the like. More personalized information may also be encoded into the fractal, including Patient Name, Prescription Regimen, Physician Name and the like. Furthermore, fractal images may be printed at manufacture, at distribution, or in combination. Thus, a manufacturer printed fractal may be provided for counterfeit prevention, while a second fractal, printed on top of the manufacturer fractal, or alongside thereof, may be provided and printed by, for example, a pharmacist or the like. These fractal images may include particular prescription, patient, prescribing doctor, date, and other patient and administration specific information. Thus, by allowing for such a combination fractal application, general and specific identification information may be provided, resulting in a robust, personalized pill marking. Additionally, it may be possible that one or the other of the manufacturer or local information may be provided by other than a fractal image. Thus, by way of example, the manufacturer information may be provided in accordance with a fractal information, while the particular patient information may be provided by a one or two dimensional barcode, or other information providing process. Of course, just the local or manufacturer fractal images may be used.
Alternatively, randomly generated fractals may be employed and recognized from a look-up table to be associated with a particular batch processing unit. Other methods may provide a library of fractal images, and indications of which fractals are to be applied to different type, shaped, or colored objects. Thus, it may be determined that a particular type of fractal image is best applied to a particularly shaped object. Next, from this subset, a further subset may be determined as best for the particular color of the object. A fractal image from this subset may then be used or encoded further, and then applied. As will be further described below, such a hierarchical selection process may also speed the eventual fractal acquisition and recognition process.
Colors may be omitted from the fractal printing process deliberately in order to increase the number of variations of the code. For example, omission of specific areas may indicate batch number or date. Furthermore, a range of colors may be included with absolute colors such as black, white, etc. acting as reference points. Use of such a range of colors allows for more patterns to be created and utilized, thus increasing a range of possible unique fractal images. Selection and/or omission of particular pixels in such a fractal image may be further used for variation to allow for randomization of predetermined fractal images.
In accordance with an embodiment of the invention, one may embed codes into the fractal image that are distributed that can be properly resolved and interpreted up in reasonable lighting conditions by a low resolution camera. If higher resolution is available in the fractal images, but cannot be precisely determined by the low resolution camera, that low resolution camera may further be employed to determine likely distributions of color or shape across such a fractal that, while perhaps not being precisely distinctive, do provide an additional level of security above the simply lower resolution components of the fractal image. Since these patterns are replicated, the system may decision fuse multiple instances of the same uncertain distribution to come up with a much higher probability of confirmation.
Printing Techniques
Referring next to
It is contemplated in accordance with one or more embodiments of the invention that each batch processing unit 520 contain a plurality of individual objects, and in accordance with a preferred embodiment of the invention, a plurality of medication pills or the like. Such a plurality of medication pills 610 are shown in
Thus, as is shown in
It may further be desirable to determine where the one or more pills or other objects are located in a single dimension (when printing is in accordance with a raster type mechanism) or in a two dimensional arrangement (such as a screen printing system or the like). In such a manner, ink may only be applied where such objects are present, thus saving ink and improving longevity of the system. Furthermore, by determining location of objects, and thus potentially batch size, particular fractal images may be employed that are properly suited to such batch size or arrangement of objects.
In addition to employing the batch processing method of
Hardware Signature
It has been determined by the inventors of the present invention that a particular image processing apparatus 530 may affect how the printing ink is distributed on the surface of the medication pill or other object, thus, in combination with the printed fractal pattern, producing a printer signature, i.e. a printer specific rendering of the particular fractal pattern. The particular characteristics of the printer, including nozzle tolerance, humidity and a host of other factors may influence an output fractal image. Such a printer signature may be further used as an identifying feature of the printed fractal image. Such a signature may be similarly determined when other imaging techniques, such as those described above, are employed.
This idea of printer signature may also be extended to product signature and camera (or more broadly, imaging apparatus) signature. For example, a product signature may be based at least in part on how the product absorbs the ink, how the pattern at higher resolution distributes the ink, texture and reflectivity of the object, shape of the object, etc. Unique texture, shape, color specific to the object (pill) will “code” or distort the fractal into a unique ID. A camera or other imaging apparatus used to eventually image the printed fractal image may also have a unique signature in distorting the fractal image that may also act as an increase in security as it may be difficult to anticipate a camera that is to be used for imaging, if not an authentic system. These signatures, as opposed to being deficiencies of the system, may be embraced to strengthen the robustness of the system. Such image influences resulting from unrelated characteristics of systems used to implement the system will be difficult/impossible to replicate. Through a decision fusion process combining the results of analysis of any one or more of these attributes, an overall picture and confidence of authenticity or counterfeit may be determined.
Thus, the different hardware and pill interaction signatures, including shape, texture, color of the object, or the like will all help to further “code” or distort the fractal in a unique way. Hence, the unique attributes of the product/item will help to create a unique fingerprint for the fractal. The inventive system is therefore able to learn the unique characteristics of the product through computer vision training or the like, and not simply apply an out-of-context code to the item.
At higher resolutions of inkjet printing or other printing, marking or etching processes, codes may be embedded that higher resolution cameras are able to read as well. In the event that inkjet printers can no longer print at a high enough resolution, then feathering or expected feathering distribution may also be picked up based on a distribution that may be unique to the printer (printer's signature). Alternatively, other higher resolution printing or etching techniques may be employed. In the event that a particular camera, such as a webcam or the like, does not have sufficiently high resolution for acquisition of a particular fractal image, then expected blurring patterns may be read. This blurring pattern may therefore be provided as a signature in itself, and may be learned through computer vision and machine learning or the like by teaching with the lower resolution camera. Multiple instances of the feathering signature may suggest likelihood of identification. Any such recognition system may rely on confidence levels of detection and confirmation. Thus, even if identification is confirmed, a threshold may determine confidence over suspicion of counterfeit (i.e. how confident the system is that the item is authentic). Many instances of low confidence levels (even if above threshold levels), as received and accumulated over time from any number of different users at a centralized location, may indicate a potential counterfeit issue and raise a flag remotely to anti-counterfeit authorities to double check a medication source, or alert a user to report the possibility of a counterfeit medication source.
When printing, and thereafter requesting image acquisition, at higher levels of resolution, the actual printing method may be employed to code information when continuous lines or images may not be able to be printed at these higher resolutions. For example, in the case of inkjet dots at very high resolution, the relative positioning of the ink jet dots may be changed, in order to be arranged, for instance, in a form of a proximity to a center of some printed object or other marker or attribute. Thus, similar to notes on a scale, these same dots may be use to allow for the encoding information even in cases when only low printing resolution is available, but high magnification image acquisition may be available when the pill or object is to be identified.
Robust Imaging Solution
As noted above, in all cases, because of the inherent replicability of fractal images, the solution is effective even if only part of the fractal is printed on the pill. By the nature of fractals, any encoded ID is repeated within the shape. Similarly, when reading, if part of the fractal is obscured or otherwise unreadable, the ID may still be determined. Many varieties of data can be encoded in the fractal (date, time, batch number, location, manufacturer, dose, item etc). Furthermore, writing or reading of the fractal image does not require alignment of the object perfectly as only part of the fractal may be good enough (may need a percentage of the fractal to be printed on a surface for a specific webcam resolution and inkjet resolution).
Upon reading of a printed fractal identification image through the user of a standard barcode laser scanning device, image acquisition, or other method for reading information from the surface of the medication pill, various reference points may be used to aid in determining authenticity. Upon the determination of one or more of such reference points preferably defined by one or more of a pattern, color, combination thereof or the like in a fractal image, distances and ratios to other reference points may be determined and used to confirm authenticity, or a level of confidence in that authenticity, in a manner as described above. Thus, an object may be scanned, and existence of at least a portion of such a fractal may be determined. If there is enough information in the portion of the fractal image to confirm identity, then identity is confirmed. If however, not enough information is available, various pieces of fractal images may be pieced together to determine enough information. Alternatively, an occlusion or the like may be effectively disregarded through such piecing together of the fractal image. Such information to be pieced together may be taken from one or multiple fractal dimensions. Further, a user may be instructed to bring such an object closer to an imaging apparatus (or otherwise zoom in on the object), or be asked to switch to another medication pill for re-identification, in order to improve capture resolution. This may be particularly important in difficult imaging environments, such as in the existence of bad lighting conditions or the like, which may reduce a confidence of precision of imaging. Such improved resolution may be employed alone, or in conjunction with anticipated effects from one or more object attributes, as noted above, in order to identify an authentic printed fractal image.
Any applicable imaging system, such as a high resolution system, or a webcam system, will benefit greatly over the use of barcoding. As orientation of the medication pill is not important, imaging of the fractal image can be performed at any angle of the pill. Furthermore, in accordance with various embodiments of the invention, various augmented reality solutions may be employed in order to properly image the medication pill and fractal image, thus truly freeing up the user to image the fractal image without any real issues regarding orientation or placement of the pill. Such augmented reality solutions may also provide additional information regarding the medication pill, including patient name, medication administration schedule information, prescribing doctor's name, contact information, or any other information that may be useful for the user to view.
Counterfeit Mapping
Once an authentication, or counterfeit is determined, such information may be provided to a remote location to accumulate such information. Each pill identification instance will result in an authenticity confidence score. With the user's consent, instances of low authenticity confidence may be reported to a centralized location, along with a medication image, GPS data, as well as time and date stamps. Higher-level authentication tests may be carried out at local pharmacies using higher resolution imaging devices. When sufficient notifications of potential counterfeit medications have accumulated (confidence flags), a geographic nexus of particular counterfeits will be determined alongside likely illegal distribution channels, thereby aiding anti-counterfeit officials. Thus, if a high concentration of counterfeit items is found in a location, investigations may be employed in that area. Further, proper identifications can be confirmed. Variation in batch coded information may be employed in order to further allow for confirmation of particular medication generation time and location stamps. Such information may be forwarded over the Internet or other transmission system, such as transmission over a cellular telephone connection or the like, to a centralized location for analysis and accumulation, for example.
Fractal Pattern Selection
Selection of the actual patterns to be employed may be performed in accordance with consideration of one or more parameters to be encoded into the fractal image, and further based upon a surface or medication pill upon which the fractal image is to be printed. As different information may be encoded into each fractal pattern, the selection and encoding of this information will make changes to the fractal pattern in subtle manners. Based upon a printing surface, expected distortion, amount and type of information to be encoded, printing technology to be employed, or level of resolution in printing and imaging desired, different fractal patterns may be preferred and employed. In fact, each such printed fractal provides a multi-dimensional pattern that comprises the above noted fractal signature. These dimensions may include one or more of fractal image, texture of the surface, color of the medication pill and shape and contours of the medication pill. These features may be employed to aid in object recognition.
Furthermore, selection of particular types of base patterns (to be modified by coding) may be performed in accordance with one or more particular tasks, pills or desired results. Thus, for example, one or more simple fractal patterns, such as a Cantor fractal patterns may be employed or lower security situations where identification is most important. More complex types of fractal base images may be employed for other, security intense applications.
One or more possible applications are outlined below. This list should be considered exemplary, and should not be construed as limiting the application of the inventive technology to other applications.
Such a fractal image may be applied as a security labeling system to any item that is created in batches and may be varied in shape, such as medication pills in the manner as describe above. When combined with a facial recognition system, matching of patient and medication can be performed. The fractal image may be applied as a game on candy as a replacement to a “scratch and win” system, thus creating a show and win application, requiring the showing of the candy with the fractal image thereon being shown to an imaging device such as a web cam on a computer, mobile device or the like.
Such a fractal image may be applied to handbags or other fabric/clothing on the inside of a garment or label (as difficult to replicate directly onto a 3d texture/surface).
Such a fractal image system may also be employed with an identification systems employing facial recognition, or other biometric identification system. Thus, identification of a patient or other user may be made employing one or more known identification systems, such as those noted above or others. Pill identification may them be performed, and a confirmation that the particular identified user is to take the identified pill. If customized fractal images are to be used, the system may be able to determine whether the particular pill being imaged is being taken at the right time by the correct person. Thus, through the ability to personalize such medication pills by batch number, patient, or particular pill dosage, a link between the pill and user may be established and confirmed. Release of such personalized information may therefore be predicated on proper biometric or other identification.
The following features may also be provided in accordance with the inventive system, as related to reporting of various results of identification determinations. Use of the inventive fractal identification systems may be employed to provide an audit trail for a pill or object manufacturer. Thus, upon use of the inventive fractal recognition system by a consumer, seller, or other individual, it may be possible to log results and alert the manufacturer if imaged fractal identifiers show low confidence (based on fractal integration with object—shape, color, texture, curvature), thus perhaps indicating an intent to replicate or otherwise provide a counterfeit product. Such information, along with location data, may be provided to authorities or other systems for tracking such counterfeiting, and in order to determine or identify counterfeit drug distribution points, or other areas with such high counterfeit drugs. Consumers may be provided with an incentive to check the identity of such fractal images, thus increasing availability of such widely spread identification information.
Similarly, to the extent that imaging of the fractal images determines that exact matches are present, similar information indicating positive results, and a likely absence of such counterfeiting may be provided to the manufacturer. Such information may also include coded information, such as batch, time/date, location and other information that may be available. Consumers may also be provided this positive match information so that they can be sure that their pill or other item is genuine, and that their security is being safeguarded.
System Benefits
Benefits of employing the inventive fractal imaging system are myriad. The use of an ink jet printing process is easily available, and relatively inexpensive while remaining flexible. Other printing or etching processes, such as one or more of those noted above, may be employed when other combinations of cost and security are to be considered, or when mass printing is to be employed. Mass batch and continuous processing avoids costs associated with properly printing images on individual pills or other objects. Thus, the fractal images may be quickly modified, by including changes to the parameters for generating the fractal images, thus being indicative of various coding included in the fractal image.
A fractal library database that changes over time may be employed, in the manner noted above, so that changes over time may be documented and later confirmed. Such a fractal database may also be tailored to webcam resolutions commonly used in smartphones, thus providing fractal images with resolution acceptable and able to be imaged by standard webcams in smartphones. Since the ID can be confirmed via a consumer with a mobile device and smart phone, it means that no special scanning hardware is needed. Thus, manufacturers or others may educate consumers to image and verify authenticity of an image, such as through imaging using a webcam on a mobile device or the like.
Confidence levels will allow for different security options. Thus different levels of resolution may be employed based upon a level of desired security. Higher resolution images, requiring higher resolution imaging devices may be employed for higher security applications, while lower resolution imaging and printing may be employed for lower security options. If the inkjet is a low quality printer, then distribution at higher resolutions (as measured by the camera) may suggest that fraud was committed if the ink distribution (or one or more other printing attributes) is different to what is expected.
As noted above, since inkjet or other etching or printing technologies may allow for rapid changing of patterns, one can quickly update a fractal pattern to be printed, and link the pattern to specific dates of production and/or batch numbers. This may be especially useful for perishables or medications that go out of date. Also may be useful for fashion items where dates of production are important. Use of older fractal patterns on newer objects may suggest counterfeiting, for example Inks that fade over time may be employed, thus indicating passage of substantial time, or the like. Alternatively, inks that may be wiped off or otherwise removed may be employed to allow for maintenance in the integrity of the item, while still providing desired levels of security.
As described above, it is easy to print whole or part of the fractal image onto the medication as alignment is not critical. The inventive technology allows pills and objects of different shapes, curvature, and sizes to be labeled uniquely. In fact the unique shape, color and texture of the item allows for a unique ID to be printed and can help to differentiate close but not identical items. The shape of the pill or item may have an influence on the way the fractal is printed allowing for expected distortions in the pattern to be recognized. This robust system may be particularly useful in identifying counterfeit items that may have slightly different shapes.
As further described above, one or more fractal patterns may have a number of different colors built in that act as a calibration code. When pills or other objects appear to be slightly different in color when compared to the fractal-pill combination (a known color gradient and range of the fractal and ratio with the pill), then a warning sign can be issued. Thus, comparison of the color of the pill or other object to the fractal, or consideration of an effect the color of the pill or other object may have on the color of the fractal may aid in the determination of status of the pill or other object. Furthermore, as each object surface will have a unique texture, the fractal may be distorted in a predictable, measurable manner in accordance with this known object surface texture. This allows for texture to be identified as well of a specific surface and differences flagged.
Fractal identification labeling is far superior to existing imaging labels because it does not require the whole fractal necessarily to be printed on the medication or other object. Use of the inventive printed fractal image will allow for occlusion by the user (due to fingers blocking image or poor environmental conditions) as only a portion of the image may be required to reach desired confidence levels. Such a fractal identification label does not suffer from occlusion problems (e.g. traditionally, one number hidden or obscured may inhibit use of the ID system). Computer vision may therefore be used to learn and to identify replicating blurred images in the fractal and/or expected ratios of patterns, colors or shapes. In the event that occlusion occurs due to the environment and/or finger occlusion, then the system may be able to “piece together” different parts of available fractal images/sections, to create a complete code within a certainty range. Furthermore, fractal identification may operate better than a number, or more traditional barcode, as it is more difficult to replicate.
This is the first universal pill labeling system developed through geometry. The problem with present day technology is that it does not label the pill, or if it does label the pill the information is inaccessible to the public or can be easily copied or destroyed. The present invention is novel because it concentrates solely on the printing of the pill and the organic nature of the fractal patterns themselves, and the complexity embedded within them, to make the system extremely difficult to replicate. The fractal patterns may blanket the entire surface of the pill. Any random segment of the pill, no matter how small or how large, will be able to be used to identify it. This is a tremendous improvement over barcodes, which are static fixed-form labels that do not lend themselves to different resolutions, and are also easily copied, damaged, and do not adapt to the physical configuration of a pill. Progressively higher levels of fractal resolution will also allow for progressively higher levels of security authentication. It is the first solution that addresses high security pill identification needs with public accessibility.
It will thus be seen that the objects set forth above, among those made apparent from the preceding description, are efficiently attained and, because certain changes may be made in carrying out the above method and in the construction(s) set forth without departing from the spirit and scope of the invention, it is intended that all matter contained in the above description and shown in the accompanying drawings shall be interpreted as illustrative and not in a limiting sense.
It is also to be understood that this description is intended to cover all of the generic and specific features of the invention herein described and all statements of the scope of the invention which, as a matter of language, might be said to fall there between.
This application is a continuation application of U.S. patent application Ser. No. 13/338,602 filed Dec. 28, 2011 to Hanina, titled “Method and Apparatus for Fractal Identification”, which in turn claims the benefit of U.S. Provisional Patent Application Ser. No. 61/544,056, filed Oct. 6, 2011 to Hanina, titled “Method and Apparatus for Fractal Identification” the contents of each of these applications being incorporated herein by reference.
| Number | Name | Date | Kind |
|---|---|---|---|
| 4728783 | Brass et al. | Mar 1988 | A |
| 4941193 | Barnsley et al. | Jul 1990 | A |
| 5065447 | Barnsley et al. | Nov 1991 | A |
| 5845264 | Nellhaus | Dec 1998 | A |
| 5992742 | Sullivan et al. | Nov 1999 | A |
| 5995673 | Ibenthal et al. | Nov 1999 | A |
| 6002794 | Bonneau et al. | Dec 1999 | A |
| 6661904 | Sasich et al. | Dec 2003 | B1 |
| 6799725 | Hess et al. | Oct 2004 | B1 |
| 7059526 | Sullivan et al. | Jun 2006 | B1 |
| 7152206 | Tsuruta | Dec 2006 | B1 |
| 7304228 | Bryden et al. | Dec 2007 | B2 |
| 8422792 | Kruglick | Apr 2013 | B2 |
| 8720790 | Hanina | May 2014 | B2 |
| 20040034579 | Xu et al. | Feb 2004 | A1 |
| 20070074030 | Quine | Mar 2007 | A1 |
| 20070194034 | Vasiadis | Aug 2007 | A1 |
| 20080106726 | Park | May 2008 | A1 |
| 20080138604 | Kenney et al. | Jun 2008 | A1 |
| 20080290168 | Sullivan et al. | Nov 2008 | A1 |
| 20130127959 | Hanina | May 2013 | A1 |
| Entry |
|---|
| Kiani, K.; Kosari, E.; Simard, M.R.; A New Method of Fractal Barcodes Identification; Proceedings of the World Congress on Engineering and Computer Science, 2011 vol. 1 WCECS 2011, Oct. 19-21, San Francisco, USA. |
| Kiani, S.; Moghaddam, M.E.; A Multi-Purpose Digital Image Watermarking Using Fractal Block Coding; Journal of Systems and Software, vol. 84, Issue 9, Sep. 2011, pp. 1550-1562 (Abstract only). |
| Kiani, S.; Moghaddam, M.E.; A Fractal Based Image Watermarking for Authentification and Verification; Image and Signal Processing, 2009, CISP 2009, Oct. 17-19, 2009 (Abstract only). |
| PCT Search report and written opinion, Cited in AI-0019-PCT1 (PCT/US12/59139 (Dec. 18, 2012),1-8. |
| Number | Date | Country | |
|---|---|---|---|
| 20140209686 A1 | Jul 2014 | US |
| Number | Date | Country | |
|---|---|---|---|
| 61544056 | Oct 2011 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | 13338602 | Dec 2011 | US |
| Child | 14228305 | US |
