Not applicable.
Not Applicable.
Not applicable.
1. Technical Field of the Invention
This invention relates to three-dimensional (3D) biometric data processing and in particular to systems and methods for processing 3D biometric data for feature detection, classification and recognition.
2. Description of Related Art
Biometrics is the science of measuring and analyzing biological data. In law enforcement and security fields, biometrics is used to measure and analyze human features, such as fingerprints, facial patterns, palm prints, retinas, etc. Currently, binary or two-dimensional fingerprint images are the most relied upon biometric measurement for verifying a person's identity and for linking persons to a criminal history and for background checks. Criminal justice agencies rely on binary fingerprint images for positive identification to latent prints collected as evidence at crime scenes and in processing persons through the criminal justice system.
The National Institute of Science and Technology (NIST) and the American National Standards Institute (ANSI) supports the ANSI/NIST-ITL 1-2000 Data Format for the Interchange of Fingerprint, Facial, & Scar Mark & Tattoo (SMT) Information. This standard defines the content, format, and units of measurement for the exchange of biometric image information, such as fingerprint, palm print, facial/mug shot, and scar, mark, & tattoo (SMT) image information that may be used in the identification process of a subject. The information consists of a variety of mandatory and optional items, including scanning parameters, related descriptive and record data, digitized fingerprint information, and compressed or uncompressed images. This information is intended for interchange among criminal justice administrations or organizations that rely on automated fingerprint and palm print identification systems or use facial/mug shot or SMT data for identification purposes. Other organizations have different standards as well for the content, format or units of measurement for biometric information. The fingerprint and palm-print images meeting specified NIST and ANSI standards allow for matching of print images in large databases of existing fingerprint and palm-print based records. For example, the FBI maintains an Interstate Identification Index System for fingerprints and palm prints. Currently, the biometric image information required by NIST and ANSI standards for fingerprints and palm prints includes only binary biometric images or two-dimensional coordinates of fingerprint features.
In addition, the most common methods of finger print and palm print acquisition collect two-dimensional biometric images. One method is a rolled ink print technique wherein a trained technician manipulates a person's finger or hand on paper to capture an ink print that meets industry standards. Another method is electronic scanning of a finger or palm as it contacts a surface to generate a two-dimensional (2D) binary image.
Systems are currently being developed for non-contact means of acquiring fingerprints using 3D imaging techniques. Recent advancements in the fields of computing hardware and software have made capturing 3D biometric data a more practical solution. For example, one method of capturing 3D biometric image data is described in PCT Application No. WO2007/050776, entitled System and Method for 3D Imaging using Structured Light Illumination, which is incorporated by reference herein.
For backward compatibility with the existing fingerprint databases, acquired 3D biometric image data is converted into a gray scale 2D image. In addition, the 2D gray scale image may be further converted into a 2D binary image. For example, an l(x,y) value is generated to represent the gray-level of a point of the fingerprint in two dimensions. A binarization of the gray scale image is then performed by assigning gray levels close to 0 to surface valleys and dark pixels with high values of gray to ridges. The ridges and valleys of the 2D binary image are then processed to detect features of the fingerprint. The detected features of the fingerprint are described with a location having two-dimensional coordinates. The detected features then are used to perform validation, classification and recognition tasks.
a and 7b illustrate an example of a subset of a 3D surface map;
a and 9b illustrate embodiments of a method of operation of the 3D feature detection module;
When acquired 3D biometric image data is converted into a gray scale 2D image for compatibility with NIST and ANSI standards, potentially useful data is lost. Therefore, a need exists to provide a method and a system for detection, validation, classification and recognition of biometric features using 3D biometric data.
An example of a 3D biometric image sensor 102 and processing module 104 is described in PCT Application No. WO/2007/050776, entitled System and Method for 3D Imaging using Structured Light Illumination, which is incorporated by reference herein.
In operation, the SLI pattern projector 112 projects focused light through an SLI pattern slide 114 onto 3D object 110 in imaging area 128. The SLI pattern is distorted by the surface variations of 3D object 110 as seen with SLI pattern distortion 124. While the SLI pattern is projected onto 3D object 110, a camera 116 captures an image of the 3D object with the SLI pattern distortion 124. The camera 116 generates a frame composed of a matrix of camera pixels 120 wherein each camera pixel 120 captures image data for a corresponding object point 122 on the 3D object 110. The camera 116 captures one or more images of the 3D object 110 with the distortions in the structured light pattern. Additional SLI slide patterns may be projected onto the 3D object 110 while additional images are captured. The one or more 3D biometric images are then stored in a biometric image data file for processing.
Referring again to
Once the 3D surface map is generated, it is further processed by ignoring certain points while connecting other points to reduce the 3D surface map data. In an embodiment, 3D image processing module 104 includes a segmentation module to segment the 3D surface map to eliminate unwanted points or data. The segmentation technique includes background-foreground modeling to eliminate background image data from a region of interest. The background-foreground modeling is performed as part of a training stage by collecting a number of background images and computing the average background model image. The foreground image information is extracted by labeling any image pixel that does not lie within a specified tolerance of the average background model image. The segmented 3D shape is the 3D information for the segmented foreground image. For example, the 3D points on a surface map for a finger are extracted from background points or separated from the 3D points of other fingers.
In known systems, once a 3D surface map is created of a fingerprint, a 2D rolled equivalent print is generated from the 3D surface map. This conversion allows the known systems to capture and store data in 3D while at the same time remaining backward compatible with existing databases of fingerprints that meet NIST/FBI standards, such as ANSI/NIST ITL 1-2000. Feature recognition and matching of features is then performed using the 2D rolled equivalent print or a converted 2D binary image. The feature data generated from the 2D binary image includes coordinates only in two dimensions, such as x, y coordinates.
The conversion of the 3D surface map to 2D rolled equivalent print or 2D binary image results in a loss of data. The use of 3D biometric data from the 3D surface map can improve detection and recognition of the biometric data.
The 3D feature detection module 200 logically divides the 3D surface map into one or more subsets of points for processing in step 214. For each subset, the 3D feature detection module 200 determines whether a 3D feature is present in step 216. When it is determined that a subset includes a 3D feature in step 218, the 3D feature detection module 200 generates 3D feature data for the detected feature in step 220. The 3D feature data includes 3D location data and feature type. The 3D feature detection module 200 then determines in step 222 if additional subsets of points need to be processed. If so, the process continues to determine 3D features for the additional subsets. If not, the 3D feature detection module 200 outputs the feature data of the detected features for the 3D surface map.
In another embodiment, the 3D surface map is also processed to detect one or more 2D features and generate 2D feature data. The 2D feature data is then compared with biometric data sets that include 2D feature data. By comparing both 3D feature data and 2D feature data with the known biometric data sets, recognition is further improved.
The 3D detection module 200 includes Subset Module 250, Template Comparison Module 252, Feature Validation Module 256, Feature Classifier Module 258, and Feature Data Module 260. Though the modules are shown as separate modules, one or more of the functions of the modules may be combined into another module or functions further segmented into additional modules. The 3D detection module 200 and the Subset module 250, Template Comparison module 252, Feature Validation Module 256, Feature Classifier Module 258, and Feature data module 260 may be integrated into one or more devices or may be separate devices. The 3D Feature Detection Module 200 is coupled to a database system 270. The database system 270 stores 3D feature template files 272 and feature data files 274.
The subset module 250 receives the 3D surface map of the biometric object, e.g. fingerprint or palm print. The 3D surface map includes a plurality of points each having 3D coordinates. The 3D coordinates include for example Cartesian (x, y, z) coordinates, spherical (r, θ, φ) coordinates or cylindrical (y, r, θ) coordinates. The 3D coordinates are in reference to an axis point defined in the surface map or other defined reference plane. Each of the points in the surface map can also include texture data. For example, texture data includes color information such as RGB values or a brightness value or a grey level. The subset module divides the 3D surface map into subwindows or subsets of the plurality of points. The subsets of points may be exclusive or overlapping. This step is performed to ease processing of feature detection and may be eliminated depending on the application.
The template comparison module 252 processes the subsets of points to detect one or more predetermined types of features. For example, in the case of fingerprints and palmprints, types of features include various patterns of ridges and furrows as well as minutiae points. Minutiae points are local ridge characteristics that occur at a ridge bifurcation, a ridge termination, ridge lake, independent ridge, point of island, spur or crossover ridge. Basic patterns of ridges and furrows include a whorl, arch and loop. These patterns may be further classified into plain arches or tented arches. Loops may be radial or ulnar depending on the side of the hand the tail points towards. Whorls also have sub-group classifications including plain whorls, accidental whorls, double loop whorls, peacock's eye, composite, and central pocket loop whorls. Cores and deltas and scars on the fingerprint are further types of features. Further features may be defined in other classification systems having different defined patterns and minutiae points than those described herein.
Since the 3D coordinates are available from the 3D surface map, additional types of 3D features can be detected. For example, ridge height, ridge shape, ridge angle and furrow depth can be determined. In addition, gradients of ridge height or furrow depth can be measured to produce a gradient vector field. Finger curvature, finger shape and finger size can also be determined from the 3D surface map.
a illustrates an exemplary subset of a 3D surface map 300. In the example embodiment in
In another embodiment, features relating to pores are extracted from the 3D surface map.
Referring back to
In an embodiment, the 3D feature templates for fingerprints include templates for the following types of features: a ridge shape, a ridge height, a furrow depth, a furrow shape, ridge maxima distribution, furrow minima distribution, a ridge height or furrow depth vector field, ridge height or furrow depth gradient field, finger curvature, finger shape and finger size. In addition, 3D feature templates include at least of the following predetermined classifications: a whorl, an arc, a loop, a delta, a core, a ridge ending, ridge orientation and a type of minutia point. In another embodiment, features related to pores such as number of pores, distance between pores, size of pores, shape of pores and position of the pores on a ridge or furrow are included as 3D feature templates.
The template comparison module 252 compares a subset of the 3D surface map with a feature vector. Again, matched filters, correlation filters, Gabor filters (with Gabor wavelets, log-Gabor wavelets) and Fourier transforms can be used to perform the comparison between the feature vector and subset. Based on the comparison, the template comparison module 252 generates a quality assessment value. In another embodiment, a multi-layered neural network can be implemented to process the subset and determine a type of feature in the data.
The template comparison module 252 performs a subset by subset analysis for feature recognition. In another embodiment, subsets are selected for feature recognition based on a flow direction of a ridge or valley. Ridge or valley flow direction measured with vectors fields are used to select the subsets for feature recognition. The template comparison module 252 compares a subset to one or more 3D feature templates. After a comparison with a feature template, a quality assessment value is assigned based on a probability or correlation that a feature is present in the subset.
The feature validation module 256 analyzes the quality assessment values assigned to subsets and determines a quality assessment. Not all the features detected by the template comparison module 252 are valid. The feature validation module 256 adds another level of robustness to the overall system. The feature validation module 256 detects distinctions between valid and invalid features in the 3D surface map. For example, when a quality assessment value falls below a threshold, the feature validation module 256 employs additional processing to determine whether a valid feature is present in the location. In another embodiment, the feature validation module further defines a type of feature detected by the template comparison module 252. The feature validation module 256 employs larger M×N feature vectors with additional information for a type of feature and additional training vectors to further define a type of feature. The feature validation module 256 processes the subsets using one or more of the following methods: Principal Component Analysis (PCA), Independent component analysis (ICA), Linear discriminant analysis (LDA), Kernel-PCA, Support Vector Machine (SVM) or a Neural Network. For example, the feature validation module 256 processes a feature detected by the template comparison module 252 and generates a PCA vector. The generated PCA vector is then compared with one or more feature vectors. A quality assessment is generated based on the comparison.
The feature classification module 258 receives a quality assessment from the feature validation module 256. The feature classification module 258 is a feature labeling stage. In an embodiment, each valid feature is labeled accordingly as a type of feature, such as minutiae, whorl, delta, ridge ending, bifurcation, etc. One or more of the classified features with high quality assessments are labeled as dominant features. Clusters of features are also identified. In a minutiae-based techniques, minutiae points are classified as a ridge bifurcation, a ridge termination, ridge lake, independent ridge, point of island, spur or crossover ridge. In an embodiment, the relative placement of classified minutiae points are then determined on the 3D surface map. In a correlation-based embodiment, a global pattern of ridges and furrows is classified into a category such as arch, loop, scar, or whorl.
The feature classification module 258 outputs the feature data for the 3D surface map. The feature data includes a list of features with 3D coordinates and type of feature. In an embodiment, the feature data further includes relative placement of the features with respect to other features. For example, it may include a distance or number of ridgelines between features. The feature data may further include an orientation angle of a feature. When a feature includes a vector or gradient vector, the feature data includes 3D coordinates, orientation angle of the vector and value for the vector. The feature data further includes the quality assessment value of the feature. The feature data for a 3D surface map is stored in a feature data file 274 in database system 270.
a illustrates another embodiment of a method of operation 350 of the 3D feature detection module 200. In step 352, a 3D surface map is processed as described herein to identify 3D features. 3D feature data for the identified features is generated. In step 354, the 3D surface map is converted into 2D image data. The conversion is performed using one or more known processes, e.g. as described in PCT Application No. WO2007/050776, entitled System and Method for 3D Imaging using Structured Light Illumination, which is incorporated by reference herein. The 2D image data is then processed to identify features and generate 2D feature data. The 2D feature data includes 2D coordinates, such as x,y coordinates and an angle of orientation within the x, y plane of a feature and type of feature information. Feature data for the fingerprint thus includes 2D feature data and 3D feature data. In an embodiment, the feature data includes relative location between 2D features and 3D features. For example, a core of a fingerprint occurs at a point of maximum ridge line curvature. Information on relative position of a ridge maxima to the core of a loop of a fingerprint image includes 3D coordinate information for the ridge maxima and 2D coordinate information for a location of the core.
b illustrates another embodiment of a method of operation 360 of the 3D feature detection module 200. In step 362, a 3D surface map is processed as described herein to identify 3D features and generate 3D feature data. In step 354, the 3D surface map is processed to generate 2D feature data without flattening or converting the 3D surface map into a 2D binary image. Instead, 2D feature data is mapped directly from the 3D coordinates and angles. In a 3D surface map, the measurement differences between minutia points at a local distance are minimal between a 3D surface map and 2D binary image. The distance and angle measurements of features in the 3D surface map can be directly compared to 2D coordinates. When mapping 3D feature data to 2D feature data, e.g. for compatibility with current ANSI/NIST standardized files, the x, y coordinates are determined from the 3D surface map and mapped to x, y coordinates in the 2D feature data. The distances between features and orientation angle are mapped directly during matching. This method avoids locally concentrated distortions that can occur with flattening methods. In an embodiment, since the errors increase with distance between features, the 3D surface map can be converted into 2D binary image to generate 2D feature data for relatively larger fingerprint patterns, such as whorls, loops and arches while the 3D feature data is mapped directly to generate 2D feature data for clusters of minutia points in relatively close proximity. The feature data for the fingerprint including 2D feature data and 3D feature data is generated in step 364.
The 3D recognition module 202 includes a categorization module 400, a comparison module 402 and matching module 404. Though the modules are shown as separate modules, one or more of the functions of the modules may be combined into a module or functions further segmented into additional modules. The 3D recognition module 202, categorization module 400, comparison module 402 and matching module 404 may be integrated into one or more devices or may be separate devices.
The 3D recognition module 402 interfaces with database system 270. The database system 270 stores a set of files with biometric data 406. The Data Set Files 406 includes feature data for known fingerprints, palm prints or other biometric data to be matched. The data set files 406 include data files with 3D biometric data 408 and data files with 2D biometric data. 3D biometric data set files 408 include 3D feature data with location of features types in 3D coordinates. The 2D biometric data set files 410 include 2D feature data with location of features types in 2D coordinates. In addition, the 3D/2D biometric data set files 412 include both 3D feature data and 2D feature data. In operation, the 3D recognition module 202 receives the feature data for a 3D surface map and matches the feature data with one of the known data set files 406. The feature data can include 3D feature data and 2D feature data generated from the 3D surface map.
The comparison module 402 receives the data set files 406 and the feature data to be matched. The comparison module 402 processes the feature data and compares the feature data with the data set files 406 in step 428. Embodiments of the matching process are described in more detail with respect to
When comparison of the 3D features has been completed in step 508, the comparison module generates matching data in 510. The entries of the compatibility table are linked into similar clusters in step 514 and a confidence value is determined based on a number of linked entries and similarity values in step 516. In an embodiment, dominant features or features with a more favorable matching value in the 3D feature data are given greater weight, e.g. increasing the similarity value for those entries. A match between dominant features in the 3D feature data and a data set file 406 results in a higher confidence value than a match between less dominant or lower match value features. In another embodiment, only entries with a threshold similarity values are processed to determine the confidence value. This matching process is repeated for each of the selected data set files 406.
In another example, a latent print may be imaged on a 3D surface such as a doorknob or steering wheel. The 2D image of the latent print is distorted by the 3D surface. The 3D surface map of a suspect's finger can then distorted similarly, e.g. the 3D surface map of the finger is distorted or wrapped around dimensions of the 3D surface. The 3D surface map can then converted into a 2D binary image and features extracted. The 2D features are then able to be matched to 2D latent prints found on the 3D surface, such as of the doorknob or the steering wheel.
Referring again to
A confidence value is generated in step 588 based on the matching data from the comparisons. In an embodiment, similarities in 3D feature data from the comparison step in 582 are given a greater weight when determining the confidence value. In another embodiment, matching data having greater similarity values is weighted more when determining the confidence value. Alternatively, matching data from just one of the comparisons in steps 582, 584 and 586 is used to determine the confidence value.
In an embodiment, one or more of various techniques, such as Bayesian Networks, Hidden Markov Models, Gaussian Mixture Models and/or Multi-stage filter banks, are implemented to analyze the matching data generated from one or more of the comparisons and generate the confidence value. For example, the similarity values generated in comparison of 3D features is analyzed with similarity values generated in comparison of 2D features using Baye's theorem to generate an overall confidence value.
In an embodiment, the 3D surface map is generated from one or more images captured of a 3D object as described herein. For example, in SLI techniques, an SLI pattern is projected onto a 3D object and images are captured. The distortions in the SLI pattern are analyzed with respect to reference points to determine 3D coordinates of points in the images to generate the 3D surface map. The reference points include the camera position, the projector position. These reference points are determined during calibrations of the SLI system. Because the 3D surface map includes coordinates and texture data for each point, it is a very large file to store. To save space, the captured images utilized to generate the 3D surface map and the calibrations are stored. The images can be compressed using known image compression techniques. If later calibrations of the SLI system find inaccuracies, the stored calibrations can be updated. The 3D surface map can be regenerated from the images and calibrations whenever needed. The feature data extracted from the 3D surface map can also be stored with the image files.
The use of 3D biometric data from the 3D surface map improves extraction of features and recognition of the biometric data. For fingerprints, additional features can be extracted from the 3D surface map, such as ridge heights, furrow depths, gradients in ridge height, etc. Additional features and feature data in 3D coordinates increases reliability of the recognition process.
As may be used herein, the term “operable to” indicates that an item includes one or more of processing modules, data, input(s), output(s), etc., to perform one or more of the described or necessary corresponding functions and may further include inferred coupling to one or more other items to perform the described or necessary corresponding functions.
The present invention has also been described above with the aid of method steps illustrating the performance of specified functions and relationships thereof. The boundaries and sequence of these functional building blocks and method steps have been arbitrarily defined herein for convenience of description. Alternate boundaries and sequences can be defined so long as the specified functions and relationships are appropriately performed. Any such alternate boundaries or sequences are thus within the scope and spirit of the claimed invention.
The present invention has been described above with the aid of functional building blocks illustrating the performance of certain significant functions. The boundaries of these functional building blocks have been arbitrarily defined for convenience of description. Alternate boundaries could be defined as long as the certain significant functions are appropriately performed. Similarly, flow diagram blocks may also have been arbitrarily defined herein to illustrate certain significant functionality. To the extent used, the flow diagram block boundaries and sequence could have been defined otherwise and still perform the certain significant functionality. Such alternate definitions of both functional building blocks and flow diagram blocks and sequences are thus within the scope and spirit of the claimed invention. One of average skill in the art will also recognize that the functional building blocks, and other illustrative blocks, modules and components herein, can be implemented as illustrated or by one or multiple discrete components, networks, systems, databases or processing modules executing appropriate software and the like or any combination thereof.