Patent Application
20210142436
COPYRIGHT© 2016-2017 Alitheon, Inc. A portion of the disclosure of this patent document contains material which is subject to copyright protection. The copyright owner has no objection to the facsimile reproduction by anyone of the patent document or the patent disclosure, as it appears in the Patent and Trademark Office patent file or records, but otherwise reserves all copyright rights whatsoever. 37 C.F.R. § 1.71(d) (2017).
Centralized databases storing digital fingerprints of objects enabling enhanced security, rapid searching, and high reliability. Methods and apparatus to identify, track, and authenticate any physical object utilizing a suitable database. In particular, event-triggered authentication of objects utilizing digital fingerprints.
Many different approaches are known to uniquely identify and authenticate physical objects, including labeling and tagging strategies using serial numbers, barcodes, holographic labels, RFID tags, and hidden patterns using security inks or special fibers. All currently known methods rely on applied identifiers that are extrinsic to the object and, as such, may fail to detect introduction of counterfeit or otherwise unknown objects. In addition, many applied identifiers add substantial costs to the production and handling of the objects sought to be identified or authenticated. Applied identifiers, such as labels and tags, are also at themselves at risk of being damaged, lost, stolen, duplicated, or otherwise counterfeited.
The following is a summary of the present disclosure in order to provide a basic understanding of some features and context. This summary is not intended to identify key or critical elements of the disclosure or to delineate the scope of the disclosure. Its sole purpose is to present some concepts of the present disclosure in simplified form as a prelude to a more detailed description that is presented later.
There are many known approaches to establishing or reestablishing the authenticity of an object, including secure supply chains, expert assessment, and counterfeit detection. What is lacking, however, and is provided by the current disclosure, is the ability to perform event-triggered authentication utilizing digital fingerprints and fingerprint templates for both overt and covert authentication, counterfeiting, conformity, and non-conformity assessments.
Additional aspects and advantages of this disclosure will be apparent from the following detailed description of preferred embodiments, which proceeds with reference to the accompanying drawings.
In order to describe the manner in which the above-recited and other advantages and features of the present disclosure can be obtained, a more particular description follows by reference to specific embodiments thereof which are illustrated in the appended drawings. Understanding that these drawings depict only typical embodiments of the disclosure and are not therefore to be considered to be limiting of its scope, the disclosure will be described and explained with additional specificity and detail through the use of the accompanying drawings in which:
Reference will now be made in detail to embodiments of the inventive concept, examples of which are illustrated in the accompanying drawings. The accompanying drawings are not necessarily drawn to scale. In the following detailed description, numerous specific details are set forth to enable a thorough understanding of the inventive concept. It should be understood, however, that persons having ordinary skill in the art may practice the inventive concept without these specific details. In other instances, well-known methods, procedures, components, circuits, and networks have not been described in detail so as not to unnecessarily obscure aspects of the embodiments.
It will be understood that, although the terms first, second, etc. may be used herein to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another. For example, a first machine could be termed a second machine, and, similarly, a second machine could be termed a first machine, without departing from the scope of the inventive concept.
It will be understood that when an element or layer is referred to as being “on,” “coupled to,” or “connected to” another element or layer, it can be directly on, directly coupled to or directly connected to the other element or layer, or intervening elements or layers may be present. In contrast, when an element is referred to as being “directly on,” “directly coupled to,” or “directly connected to” another element or layer, there are no intervening elements or layers present. Like numbers refer to like elements throughout. As used herein, the term “and/or” includes any and all combinations of one or more of the associated listed items.
The terminology used in the description of the inventive concept herein is for the purposes of describing particular embodiments only and is not intended to be limiting of the inventive concept. As used in the description of the inventive concept and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that the term “and/or” as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed objects. It will be further understood that the terms “comprises” and/or “comprising,” when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
The methods described in the present disclosure enable the identification of an object without the need for attaching, applying, or associating physical tags or other extrinsic identifying materials with the object. A system does this by creating a unique digital signature for the object, which is referred to as a digital fingerprint. Digital fingerprinting utilizes the structure of the object, including random and/or deliberate features created, for example, during manufacturing or use of the object, to generate a unique digital signature for that object—similar to the way in which a human fingerprint references the friction ridges on a finger. Also, like a human fingerprint, the digital fingerprint can be stored and retrieved to identify objects at a later time.
Eliminating the need to add extrinsic identifiers or any physical modifications to an object offers a number of advantages to manufacturers, distributors, buyers, sellers, users, and owners of goods. Forgoing the addition of extrinsic identifiers reduces the cost of manufacturing and offers greater security than physical tagging. Moreover, physical identifiers can be damaged, lost, modified, stolen, duplicated, or counterfeited whereas digital fingerprints cannot.
Unlike prior art approaches that simply utilize a comparison of pixels, a system in accordance with the present disclosure utilizes the extraction of features to identify and authenticate objects. Feature extraction enables users to take a large amount of information and reduce it to a smaller set of data points that can be processed more efficiently. For example, a large digital image that contains tens of thousands of pixels may be reduced to a few locations of interest that can be used to identify an object. This reduced set of data is called a digital fingerprint. The digital fingerprint contains a set of fingerprint features or locations of interest which are typically stored as feature vectors. Feature vectors make image processing more efficient and reduce storage requirements as the entire image need not be stored in the database, only the feature vectors need to be stored. Examples of feature extraction algorithms include—but are not limited to—edge detection, corner detection, blob detection, wavelet features, Gabor, gradient and steerable output filter histograms, scale-invariant feature transformation, active contours, shape contexts, and parameterized shapes.
While the most common applications of the system may be in the authentication of physical objects such as manufactured goods and documents, the system is designed to be applicable to any object that can be identified, characterized, quality tested, or authenticated with a digital fingerprint. These include but are not limited to mail pieces, parcels, art, coins, currency, precious metals, gems, jewelry, apparel, mechanical parts, consumer goods, integrated circuits, firearms, pharmaceuticals, and food and beverages. Here the term “system” is used in a broad sense, including the methods of the present disclosure as well as apparatus arranged to implement such methods.
In this application, the term “scan” is used in the broadest sense, referring to any and all means for capturing an image or set of images, which may be in digital form or transformed into digital form. Images may, for example, be two dimensional, three dimensional, or in the form of a video. Thus a “scan” may refer to an image (or digital data that defines an image) captured by a scanner, a camera, a specially adapted sensor or sensor array (such as a CCD array), a microscope, a smartphone camera, a video camera, an x-ray machine, a sonar, an ultrasound machine, a microphone (or other instruments for converting sound waves into electrical energy variations), etc. Broadly, any device that can sense and capture either electromagnetic radiation or mechanical wave that has traveled through an object or reflected off an object or any other means to capture surface or internal structure of an object is a candidate to create a “scan” of an object. Various means to extract “fingerprints” or features from an object may be used; for example, through sound, physical structure, chemical composition, or many others. The remainder of this application will use terms like “image” but when doing so, the broader uses of this technology should be implied. In other words, alternative means to extract “fingerprints” or features from an object should be considered equivalents within the scope of this disclosure. Similarly, terms such as “scanner” and “scanning equipment” herein may be used in a broad sense to refer to any equipment capable of carrying out “scans” as defined above, or to equipment that carries out “scans” as defined above as part of their function.
In this application, different forms of the words “authenticate” and “authentication” will be used broadly to describe both authentication and attempts to authenticate which comprise creating a digital fingerprint of the object. Therefore, “authentication” is not limited to specifically describing successful matching of inducted objects or generally describing the outcome of attempted authentications. As one example, a counterfeit object may be described as “authenticated” even if the “authentication” fails to return a matching result. In another example, in cases where unknown objects are “authenticated” without resulting in a match and the authentication attempt is entered into a database for subsequent reference the action described as “authentication” or “attempted authentication” may also, post facto, also be properly described as an “induction”. An authentication of an object may refer to the induction or authentication of an entire object or of a portion of an object.
Because digital fingerprinting works with many different types of objects, it may be useful to define what regions of digital images of objects are to be used for the extraction of features for authentication purposes. The chosen regions may vary for different classes of objects. In some embodiments, a chosen region may be the image of the entire object; in other embodiments chosen regions may be one or more sub-regions of the image of the object.
For instance, in the case of a photograph, a digital image of the entire photograph may be chosen for feature extraction. Each photograph is different and there may be unique feature information anywhere in a photograph. In such a case, the authentication region may be the entire photograph.
In some embodiments, multiple regions may be used for fingerprinting. In some examples, there may be several regions where significant variations take place among different similar objects that need to be distinguished while, in the same objects, there may be regions of little significance. In other examples, a template may be used (see
In one embodiment, an object, such as a bank note, may be deemed authenticated if a few small arbitrary regions scattered across the surface are fingerprinted, possibly combined with one or more recognitions of, for example, the contents of a region signifying the value of the bank note or one containing the bank note serial number. In such examples, the fingerprints of any region (along with sufficient additional information to determine the bank note value and its purported identity) may be considered sufficient to establish the authenticity of the bill. In some embodiments, multiple fingerprinted regions may be referenced in cases where one or more region may be absent from an object (through, for example, tearing) when, for example, a bank note is presented for authentication. In other embodiments, however, all regions of an object may need to be authenticated to ensure an object is both authentic and has not been altered.
In one embodiment, a passport may provide an example of feature extractions from multiple authentication regions; see
Next, the system processes the image data to determine an authentication region. In this example, the authentication region is the biographic data page of the U.S. Passport, located in the lower portion of image 150, identified by dashed box 154. Next, the process generates an authentication image for feature extraction, block 106. The authentication image is illustrated at reference 156. Next, at block 108, the process defines one or more locations of interest for feature vector extraction. The locations of interest in this example are, as shown in image 158 by dashed boxes 160, the surname, the given name, the passport number, and the passport photo.
Finally, at block 110, the process 100 comprises creating a fingerprint template 120. In this example, template 120 identifies an object class (U.S. Passport), defines an authentication region (for example, by X-Y coordinates), and lists one or more locations of interest within that authentication region. In this instance, the list comprises passport number, photo, first name, and last name.
In some embodiments, an ability to define and store optimal authentication regions for classes of objects may offer benefits to a user. In some embodiments, it may be preferable to scan limited regions of objects rather than to scan entire objects. For instance, in the case of an article of designer clothing, scanning a clothing label may be preferable to scanning an entire garment. (To be clear, the label or a portion of it is scanned for fingerprinting, not to recognize text on the label.) Further, defining such regions may enable detection of partial alteration of an object.
Once an authentication region is defined, specific applications may be created for different markets or classes of objects that may assist users in locating and scanning an optimal authentication region. In some embodiments, for example when utilizing a mobile device, a location box and crosshairs may automatically appear in the viewfinder of a smartphone camera application, to help the user center the camera on an authentication region, and automatically lock onto a region and complete a scan when the device is focused on an appropriate area. It should be noted that, although some examples suggested above are two-dimensional objects (passport, bank note), the present disclosure is fully applicable to three-dimensional objects as well. As previously noted, scanning may be of any kind, including 2-D, 3-D, stereoscopic, HD, etc. and is not limited to the use of visible light or to the use of light at all (as previously noted, sonar and ultrasound are, for example, appropriate scanning technologies).
In some embodiments, objects may have permanent labels or other identifying information attached to them. In addition to the objects themselves, these attachments may also be referenced as features for digital fingerprinting, particularly where the label or other identifying information becomes a permanent part of the object. In one example, a permanent label may be used as an authentication region for the object to which it is affixed. In another example, a label may be used in conjunction with the object itself to create a fingerprint of multiple authentication regions referencing both a label and an object to which the label is affixed.
In one example, wine may be put into a glass bottle and a label affixed to the bottle. Since it is possible that a label may be removed and re-applied elsewhere merely using the label itself as an authentication region may not be sufficient. In this case, the authentication region may be defined so as to include both a label and a substrate it is attached to—in this example some portion of a label and some portion of a glass bottle. This “label and substrate” approach may be useful in defining authentication regions for many types of objects, such as various types of goods and associated packaging. In other instances, authentication may reveal changes in the relative positions of some authentication regions such as in cases where a label has been moved from its original position, which may be an indication of tampering or counterfeiting. If an object has “tamper-proof” packaging, this may also be included in the authentication region.
In some embodiments, multiple authentication regions may be chosen from which to extract unique features. In a preferred embodiment, multiple authentication regions may be selected to enable the separate authentication of one or more components or portions of an object. For example, in one embodiment, features may be extracted from two different parts of a firearm. Both features may match the original firearm but since it is possible that both parts may have been removed from the original firearm and affixed to a weapon of different quality, it may also be useful to determine whether the relative positions of the parts have changed. In other words, it may be helpful to determine that the distance (or other characteristics) between Part A's authentication region and Part B's authentication region remains consistent with the original feature extraction. If the positions of Parts A and B are found to be consistent to the relative locations of the original authentication regions, the firearm may be authenticated. Specifications of this type may be stored with or as part of a digital fingerprint of an object.
In an embodiment, when a new type or class of object is being scanned into a system for the first time, the system can create a fingerprint template (as shown in
A fingerprint template is not required for the system to authenticate an object, as the system can automatically extract features and create a digital fingerprint of an object without it. However, the presence of a template may optimize the authentication process and add additional functionality to the system.
The uses of the fingerprint template include but are not limited to determining the regions of interest on an object, the methods of extracting fingerprinting and other information from those regions of interest, and methods for comparing such features at different points in time. The name “fingerprint template” is not important; other data with similar functionality (but a different name) should be considered equivalent.
In an embodiment, four different but related uses for this technology are particularly in view in the present disclosure. These are illustrative but are not intended to be limiting of the scope of the disclosure. These applications may be classified broadly as (1) authentication of a previously scanned original, (2) detection of alteration of a previously scanned original, (3) detection of a counterfeit object without benefit of an original, and (4) assessing the degree to which an object conforms with a predetermined specification, such as a manufacturing specification or other applicable specification.
The uses of the fingerprint template include but are not limited to determining the regions of interest on an object, the methods of extracting fingerprinting and other information from those regions of interest, and methods for comparing such features at different points in time. The name “fingerprint template” is not important; other data with similar functionality (but a different name) should be considered equivalent.
In an embodiment, four different but related uses for this technology are particularly in view in the present disclosure. These are illustrative but are not intended to be limiting of the scope of the disclosure. These applications may be classified broadly as (1) authentication of a previously scanned original, (2) detection of alteration of a previously scanned original, (3) detection of a counterfeit object without benefit of an original, and (4) assessing the degree to which an object conforms with a predetermined specification, such as a manufacturing specification.
In example (1), an object is fingerprinted preferably during the creation process (or at any time when its provenance may be sufficiently ascertained) or at a point where an expert has determined its authenticity. Subsequently, the object is later re-fingerprinted, and the two sets of fingerprints are compared to establish authenticity of the object. The fingerprints may be generated by extracting a single fingerprint from the entire object or by extracting multiple sets of features from multiple authentication regions. Fingerprinting may also involve reading or otherwise detecting a name, number, or other identifying characteristics of the object using optical character recognition or other means which may be used to expedite or facilitate a comparison with other fingerprints. For instance, in cases where manufacturing (or other object) databases use serial numbers or other readable identifiers, such identifiers may be utilized to directly access the database record for the object and compare its digital fingerprint to the original that was previously stored, rather than searching an entire digital fingerprinting database for a match.
In case (2), a fingerprinted object is compared, region by region, with a digital fingerprint of an original object to detect low or nonexistent matching of the fingerprint features from those regions. While case (1) is designed to determine whether the original object is now present, case (2) is designed to detect whether the original object has been altered and, if so, how it has been altered. In some embodiments, authentication regions having poor or no matching fingerprint features will be presumed to have been altered.
In case (3), an object may not have been fingerprinted while its provenance was sufficiently ascertainable. One example would be bills or passports created prior to initiating the use of a digital fingerprinting system. In such examples, digital fingerprints of certain regions of interest on an object may be compared with digital fingerprints from known, or suspected, counterfeit objects or with both those and fingerprints of properly authenticated objects. In one example, a photograph may be spuriously added to a passport and, as an artifact of the counterfeiting, the edge of the added photo may tend to be sharper than an edge of an original, unaltered, photograph. In such a case, fingerprint characteristics of known authentic passports and those of passports that are known (or suspected to) have been altered by changing a photograph may be compared with the passport being inspected to estimate whether the passport exhibits indications of alteration.
In an embodiment, once an object has been scanned and at least one authentication region has been identified, the digital image, which will be used to create the unique digital fingerprint for the object, is generated. The digital image (or set of images) provides the source information for the feature extraction process.
In the present disclosure, a digital fingerprinting feature is defined as a feature or a location of interest in an object, which feature is inherent to the object itself. In some embodiments, features preferably are a result of a manufacturing process, other external processes, or of any random, pseudo-random, or deliberate process or force, such as use. To give one example, gemstones have a crystal pattern which provides an identifying feature set. Every gemstone is unique and every gem stone has a series of random flaws in its crystal structure. This pattern of random flaws may be used for the extraction of feature vectors for identification and authentication.
In the present disclosure, a “feature” is not necessarily concerned with reading or recognizing meaningful content, for example by using methods like optical character recognition. A digital fingerprint of an object may capture both features of the object and features of any identifiers that are affixed or attached to the object. Feature vectors extracted from authentication regions located on an affixed identifier are based on the substances of which the identifier is physically comprised rather than the information (preferably alphanumeric) that is intended to be communicated by the identifier. For instance, in the case of a wine bottle, features may be captured from the bottle and from a label affixed to the bottle. If the label includes a standard UPC bar code, the paper of the label and the ink pattern of the bar code may be used to extract a feature vector without reading the alphanumeric information reflected by the bar code. An identifier, such as a UPC bar code print consisting of lines and numbers, has no greater significance in the generation and use of a feature vector than a set of randomly printed lines and numbers.
Although reading identifier information is not necessary for digital fingerprinting, in some embodiments, where a user desires to capture or store identifier information (such as a name, serial number, or a bar code) in an association with an object, the system may allow the user to capture such information and store it in the digital fingerprint. Identifier information may, for example, be read and stored by utilizing techniques such as optical character recognition, and may be used to facilitate digital fingerprint comparisons. In some cases, serial numbers may be used as the primary index into a database that may also contain digital fingerprints. There may be practical reasons for referencing serial numbers in relations to digital fingerprints. In one example, a user is seeking determine whether a bank note is a match with a particular original. In this case, the user may be able to expedite the comparison by referencing the bank note serial number as an index into the digital fingerprinting database rather than iterating through a large quantity of fingerprints. In these types of cases, the index recognition may speed up the comparison process but it is not essential to it.
Once a suitable digital fingerprint of an object is generated the digital fingerprint may be stored or registered in a database. For example, in some embodiments, the digital fingerprint may comprise one or more fingerprint features which are stored as feature vectors. The database should preferably be secure. In some embodiments, a unique identifier, such as a serial number, may also be assigned to an object to serve, for example, as a convenient index. However, assigning a unique identifier is not essential as a digital fingerprint may itself serve as a key for searching a database independent of any addition of a unique identifier. In other words, since a digital fingerprint of an object identifies the object by the unique features and characteristics of the object itself the digital fingerprint renders unnecessary the use of arbitrary identifiers such as serial numbers or other labels and tags, etc.
Next, at block 206, a database query may be conducted to see if a template exists in the system for the object that was scanned at 202. For example, in some cases, the initial image may be processed to extract a serial number or other identifying information. In an embodiment, the database may then be interrogated; decision 206, to see if a template exists for that serial number. If the answer is YES, path 208, the system accesses the template 212 and uses it to select one or more authentication regions 210. The template 212 lists the regions and their respective locations in the image (i.e. on the passport front page in this example). Physical locations may, as an example, be specified relative to a given location, and/ or relative to each other. Location may be important because, for example, a replaced photograph may not be in exactly the same location as the removed original. In short, the template guides the authentication software in analyzing the image data. In that analysis, for each authentication region (called a “Feature” in 212), various features are extracted from the image data, block 222.
The extracted features are used to form a digital fingerprint of the object, block 224. For example, each feature may be described by a feature vector. Location and other data and metadata may be included in the fingerprint. In general, the process for extracting features and describing them in feature vectors may be specified in the template. The template may also specify which regions must be matched to declare the passport a match. In the passport example, all specified regions must match a record in the database for the passport to be determined to be authentic and unaltered. In other cases, a few matches may be sufficient. The digital fingerprint generated at block 224 is then used to query a reference database 230 for a match.
Returning to the decision block 206, there may not be an existing template in the system for the object under inspection—NO branch for “Non-Template Object Class.” The process here may vary with the type of object under inspection and the purpose for the inspection. In some cases, a scanned image of an object may be processed to find locations of interest, block 232, for example, surface areas that are non-homogenous and thus have considerable image data content. In other words, finding locations of interest may be automated or semi-automated. The locations may be used to extract features, block 234 and/or recorded in a template for later use. Preferably, locations should be recorded in, or otherwise associated with, the digital fingerprint of the object.
In other examples, user input may be used to select authentication regions, and then the process proceeds to 234 as before. In some embodiments, an entire object may be scanned and all of the data processed to find and record digital fingerprint data. Whatever the case, the process proceeds to create a digital fingerprint, block 236, which can then be used to query the database 230 for a match. The match result may not be binary (yes/no); rather, in many cases, the result may indicate a confidence level of a match or may be a composite of binary results or confidence levels—such as when an object has been altered in part or in whole and/or has been assembled, or disassembled.
In an embodiment, an object is scanned and an image is generated. The steps that follow depend on the operation to be performed. Several illustrative example cases are discussed below.
Case 1: For authentication of a previously fingerprinted object, the following steps may be followed (see
The extracted data is processed to generate a digital fingerprint, block 312. A database 320 may be queried for a matching fingerprint, block 314. A “match” may be defined by a binary, probability, or similarity metric or be a composite of metrics. Results of the database query may be reported to a user, block 322. Finally, a new digital fingerprint may be added to the database 320, shown at process block 330.
Case 2: For inspection of specific features of a previously fingerprinted object to determine whether they have been altered, the steps are similar to Case 1, but the process is aimed at detection of alterations rather than authentication of the object:
Cases 3 and 4 are elaborated in related patent applications.
In all of the above cases, features may be extracted from images of objects scanned under variable conditions, such as different lighting conditions. Therefore, it is unlikely two different scans will produce completely identical digital fingerprints. In a preferred embodiment, the system is arranged to look up and match objects in the database when there is a “near miss.” For example, two feature vectors [0, 1, 5, 5, 6, 8] and [0, 1, 6, 5, 6, 8] are not identical but by applying an appropriate difference metric the system can determine that they are close enough to say with a degree of certainty that they are from the same object that has been seen before. One example would be to calculate Euclidean distance between the two vectors in multi-dimensional space, and compare the result to a threshold value. This is similar to the analysis of human fingerprints. Each fingerprint taken is slightly different, but the identification of key features allows a statistical match with a high degree of certainty.
One advantage of the feature-based method is that when an object is worn from handling or use (even very worn), a system may still identify the object as original, which may be impossible with the bitmapped approach.
The image of the damaged bill is analyzed by a processor. The processor accesses a database of previously stored fingerprint data. If the dollar bill serial number is legible (by eye or machine), the record for the corresponding bill may be accessed from the datastore using the serial number as an index. Similarly, if any portion of the serial number is legible, the search for a matching record can be narrowed on that basis. Either way, a candidate record, containing a set of stored regions of interest may be compared to the image of the damaged bill.
As explained above, in addition to being able to recognize a worn object, the feature-based approach is able to address other external problems such as rotated images. This is especially important in a system where an unsophisticated user, such as a retail customer, may be scanning an object to be authenticated. In such cases, external factors like lighting and rotation may not be under the system operator's control.
Referring now to
Once an appropriate transformation is found, further matching may be done to increase the level of confidence of the match, if desired. In some embodiments, a number of matches on the order of tens or hundreds of match points may be considered sufficient. The number of non-match points also should be taken into account. That number should preferably be relatively low, but it may be non-zero due to random dirt, system “noise”, and other circumstances. Preferably, the allowed mapping or transformation should be restricted depending on the type of object under inspection. For instance, some objects may be inflexible, which may restrict the possible deformations of the object.
Summarizing the imaging requirements for a typical fingerprinting system, for example for inspecting documents, the system preferably should provide sufficient imaging capability to show invariant features. Particulars will depend on the regions used for authentication. For many applications, 10-fold magnification may be adequate. For ink bleeds on passports, bills, and other high-value authentication, 40-fold magnification may likely be sufficient. In preferred embodiments, the software should implement a flexible response to accommodate misalignment (rotation), misorientation, and scale changes. Color imaging and analysis is generally not required for using the processes described above, but may be used in some cases.
In
After induction, the object 1804 may encounter wear and tear, and otherwise may change, intentionally or not, in ways that may not be known a priori, represented by the question mark 1808. The original object 1804 may even in fact be lost or stolen after induction and a counterfeit may be introduced. Along path 1809, an object 1810 may be presented for authentication, represented by block 1820. Below are described some additional scenarios and use cases for the authentication technology described herein, and what may be done under the broad heading of “authentication”. Under many circumstances, induction, authentication, or both may be done remotely by use of technology such as drones or by other covert means. In one example, an agent may take a photograph of an object with a smartphone, without the knowledge or consent of the possessor of the object, and the resulting image may be utilized for induction and/or authentication as described herein.
More specifically, in some embodiments, some part of the induction/ authentication process may be done remote from a facility intended for that purpose. In addition, some part of the induction/authentication process may be accomplished without the knowledge of the then-current possessor of an object. In particular, the induction and/or authentication are not part of the current possessors' normal processes. These two criteria are not essential for the present disclosure, but are generally representative of some applications.
To begin forming a digital fingerprint of a scanned object, at least one authentication region is selected, block 2130, in the authentication image data. This selection preferably is carried out by the fingerprinting software. The authentication region(s) may be selected according to a predetermined template based on the class of objects. Locations of the authentication regions may be stored in the digital fingerprint record, block 2132.
At block 2134, the process continues by selecting locations of interest within each authentication region. To select locations of interest (areas in an image from which to extract fingerprint features), a software process may automatically select a large number—typically hundreds or even thousands per square mm—of preferred locations of interest for purposes of the digital fingerprint. A location may be of interest because of a relatively high level of content. That “content” in a preferred embodiment may comprise a gradient or vector, including a change in value and a direction. The selected locations of interest may be added to the fingerprint record, block 2136. In one example, such areas may be identified by a location or centroid, and a radius thus defining a circular region. Circular regions are preferred for some applications because they are not affected by rotation of the image.
Next, block 2138, the process calls for extracting features from each location of interest, and forming feature vectors to describe those features in a compact form that facilitates later analysis, for example, calculation of vector distances as a metric of similarity in comparing fingerprints for authentication. Various techniques are known for extracting such features. The resulting feature vectors are added to the fingerprint, block 2140. At block 2142, additional information may be added to the digital fingerprint identifying other fingerprints and related information associated with the same object. In some embodiments, a relationship, such as relative location of the other fingerprints to the current fingerprint may be used. For example, in some objects, multiple regions may be authentic individually, but a change in their relative location may indicate that the object is not authentic. Thus, a fingerprint record may include first and second feature vectors (each describing a corresponding feature extracted from an area of interest) and a relative location of one to the other.
Above, with regard to
Authentication may be conducted in response to a trigger. That is, authentication performed outside the normal steady functioning of a system (in contrast, for example, to inducting parts as they are manufactured and authenticating them as they are installed). In view in this disclosure is any form of event trigger (see the progression below) and any form of authentication using fingerprinting or similar technology. Each of the following are non-limiting examples of events that could serve as triggers. Each of them could be utilized to trigger the kinds of authentication taught above in this document.
Schedule-based triggering. In one example, this disclosure envisions a system where authentication is triggered on a schedule (e.g. as part of quarterly inventory, or two hours past closing time). Triggering on a schedule is close to being “part of the normal . . . functioning of the system” but is included for completeness in the spectrum of “event-driven authentication”. This form would include normal calendaring but also following computer scripts or even periodic, random, or from time-to-time manual interrupts of normal processes.
Event triggering.
Referring to
The ETP may initiate various actions, responsive to a trigger input signal, for example, by sending a message to another entity or system, in particular an authentication system. Hence the title, “Event-Driven Authentication.” The ETP may command the actions, for example, using known network communication protocols. In one example, responsive to the back door of a warehouse being detected as opening (a sensor input), the ETP may send a message to a remote system to have it conduct an inventory of the warehouse, in part or in whole. The remote system may utilize appropriate scanning equipment to capture images for the inventory for fingerprinting. The processes illustrated by
In some embodiments, one or more remote sensors 2210, i.e., sensors that are not at the same physical location as the ETP 2200, may be coupled over a network, such as a LAN, WAN, or the internet 2212, for connection to the ETP 2200 via a suitable network interface 2216. In operation, output signals from the remote sensor(s) may be utilized by the ETP 2200 as triggers to initiate authentication actions, which again may be local or remote.
In some embodiments, other remote processes or systems 2230 may be similarly coupled over a network to communicate with the ETP 2200. As one illustration: a piece of luggage is going down a conveyor (not shown) and is normally to be routed by reading the bag tag. It passes a bag tag reader, but this time the reader does not get a read. The bag tag reader may be a remote process or system 2230 coupled to the ETP 2200. In this case, a tag reader failure message triggers a process or response in the ETP 2200 that initiates a full fingerprint-based authentication of the (previously inducted) luggage item. The authentication process may be performed in various ways, several of which are described in detail above.
In some embodiments, the ETP 2200 may direct a local field imaging system 2232 via a link 2234. The ETP may be coupled directly to the local imaging system 2232 in some applications. In other cases, it may be communicatively coupled over a network. In an embodiment, the local system 2232 may acquire image data of an object 2236 (for example, the aforementioned luggage item). The imaging system 2232 may interact via link 2236 with a fingerprint processing and storage system 2240. In an embodiment, the fingerprint system 2240 may include a digital fingerprint processor 2256, a secure database server 2258, and a fingerprint database 2260 described in more detail above. The fingerprinting system 2240 may be local or remote, for example, in the cloud. It may be coupled via link 2243 to the ETP 2200.
In some embodiments, the triggered authentication process may be done remotely from the ETP 2200. for example, the ETP 2200 may communicate via interface 2216 and internet 2212 with a remote field image acquisition system 2242. This system is configured for image capture for authentication (and optionally other purposes). The image system 2242 may be part of a larger manufacturing, assembly, or other operation. The image system 2242 may be integrated into other machinery, or it may stand alone. The image system 2242 may be operable by a robot 2250 to capture an image of an object 2248 for authentication. The robot 2250 may be mobile, for example, to move about a warehouse capturing images for inventory control. The robot may capture images, for example, following a door ajar or break-in trigger (detected by a sensor as described). The image system 2242 may work in concert with a fingerprint system such as 2240, with which it may communicate over a network. In another example, authentication may be triggered by loading dock receipt of components missing an expected RFID tag or documentation.
Preferably, authentication may be triggered by sensors (as noted), or by rules or logic 2253, which may be realized in the form of computer code or scripts, or by the physical presence of an unexpected item, or the absence of an expected one. The trigger processor may take an action based on a combination of inputs, processed according to the applicable rules and logic.
This disclosure further includes authentication triggered by detection of another event—which event may or may not be directly related to the authentication process. Other events and processes 2222 may communicate with the ETP 2200 as illustrated or otherwise. One example is a conveyor that is carrying bags to their airplanes when a jam occurs. Currently this would mean that all those bags must, once the belt is restarted, be routed past a bag tag reader to reestablish each bag's identity. With a proposed embodiment, the system would immediately authenticate and locate each bag on the affected conveyor(s) so that when the jam is cleared, each bag can continue on its way without the need to reroute past a bag tag reader. Thus, in such a scenario, an image system 2232 or 2242 may be configured to capture images of luggage items, responsive to direction from the ETP 2200, which reacts to a jam sensor signal (from, for example, local sensors 2202) from a luggage conveyor (not shown).
In another embodiment, a particular machined part may be both expensive and critical to system functioning and its arrival at an aircraft manufacturer may trigger a full authentication process (e.g. reading the serial number and manufacturer, fingerprinting the item, comparing the fingerprints with those in the reference database, and confirming/denying the authenticity of the item.)
Security cameras have in recent years become commonplace and widespread in both the public and private sector. Some security cameras are monitored by security personnel but others (such as at baggage or parcel handling facilities, along with most in-store security cameras) are intended for post facto forensics. The present disclosure teaches the triggering of authentication by real-time forensics, generally taken to mean using some form of predictive analytics or artificial intelligence to determine that an unusual event has taken place and what the correct response to that event is. Systems and methods such as those illustrated above may be used to provide these features.
As a further illustration, an AI program detects a person moving near a baggage conveyor in the airport where no persons are supposed to be present. In some embodiments, a camera may be the input for local sensor 2202 that provides image data (still or motion) as its “output signals.” An AI program may be part of the ETP 2200 for analyzing the image data. In response to this recognition “trigger,” the ETP 2200 may enhance or escalate the level of tracking on the bags in the airport luggage handling system, such as looking to find bags that have been added or are now missing from the system or that are now out of place. For example, the system may then acquire fingerprints of bags at a given location—say in the vicinity of the detected unauthorized person—using a system 2232, and query the fingerprint system 2240 database (via link 2242) to confirm that no bags have been added or removed. This feature may be applied for parcels at a sortation house, manufactured items on a conveyor, and many other cases. The proposed system may also include predictive or AI modeling to monitor external data (e.g. on the web) such as related news and sentiment to weight the frequency of authentication as well as communicate awareness/status on any item or group of items related to the area of abnormal concern.
Most of the equipment discussed above comprises hardware and associated software. For example, the typical portable device is likely to include one or more processors and software executable on those processors to carry out the operations described. We use the term software herein in its commonly understood sense to refer to programs or routines (subroutines, objects, plug-ins, etc.), as well as data, usable by a machine or processor. As is well known, computer programs generally comprise instructions that are stored in machine-readable or computer-readable storage media. Some embodiments of the present invention may include executable programs or instructions that are stored in machine-readable or computer-readable storage media, such as a digital memory. We do not imply that a “computer” in the conventional sense is required in any particular embodiment. For example, various processors, embedded or otherwise, may be used in equipment such as the components described herein.
Memory for storing software again is well known. In some embodiments, memory associated with a given processor may be stored in the same physical device as the processor (“on-board” memory); for example, RAM or FLASH memory disposed within an integrated circuit microprocessor or the like. In other examples, the memory comprises an independent device, such as an external disk drive, storage array, or portable FLASH key fob. In such cases, the memory becomes “associated” with the digital processor when the two are operatively coupled together, or in communication with each other, for example by an I/O port, network connection, etc. such that the processor can read a file stored on the memory. Associated memory may be “read only” by design (ROM) or by virtue of permission settings, or not. Other examples include but are not limited to WORM, EPROM, EEPROM, FLASH, etc. Those technologies often are implemented in solid state semiconductor devices. Other memories may comprise moving parts, such as a conventional rotating disk drive. All such memories are “machine readable” or “computer-readable” and may be used to store executable instructions for implementing the functions described herein.
A “software product” refers to a memory device in which a series of executable instructions are stored in a machine-readable form so that a suitable machine or processor, with appropriate access to the software product, can execute the instructions to carry out a process implemented by the instructions. Software products are sometimes used to distribute software. Any type of machine-readable memory, including without limitation those summarized above, may be used to make a software product. That said, it is also known that software can be distributed via electronic transmission (“download”), in which case there typically will be a corresponding software product at the transmitting end of the transmission, or the receiving end, or both.
Having described and illustrated the principles of the invention with reference to illustrated embodiments, it will be recognized that the illustrated embodiments can be modified in arrangement and detail without departing from such principles, and can be combined in any desired manner. And although the foregoing discussion has focused on particular embodiments, other configurations are contemplated. In particular, even though expressions such as “according to an embodiment of the invention” or the like are used herein, these phrases are meant to generally reference embodiment possibilities, and are not intended to limit the invention to particular embodiment configurations. As used herein, these terms can reference the same or different embodiments that are combinable into other embodiments.
Embodiments of the invention may include a non-transitory machine-readable medium comprising instructions executable by one or more processors, the instructions comprising instructions to perform the elements of the embodiments as described herein.
Consequently, in view of the wide variety of permutations to the embodiments described herein, this detailed description and accompanying material is intended to be illustrative only, and should not be taken as limiting the scope of the invention.
Having described and illustrated the principles of the invention in a preferred embodiment thereof, it should be apparent that the invention may be modified in arrangement and detail without departing from such principles. We claim all modifications and variations coming within the spirit and scope of the following claims.
This application is a non-provisional of, and claims priority pursuant to 35 U.S.C. § 119(e) (2012) to U.S. provisional application no. 62/374,162 filed Aug. 12, 2016, hereby incorporated by reference as through fully set forth.
| Number | Date | Country | |
|---|---|---|---|
| 62374162 | Aug 2016 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | 15672182 | Aug 2017 | US |
| Child | 17125437 | US |
