Patent Grant
11151355
Fingerprint sensors have become ubiquitous in mobile devices as well as other applications for authenticating a user's identity. They provide a fast and convenient way for the user to unlock a device, provide authentication for payments, etc. In general, a user must enroll his or her fingerprint for every device including a fingerprint sensor. Because the size of a fingerprint sensor is in general smaller than the complete fingerprint, the enrollment process for each fingerprint sensor is typically a laborious process, which must be repeated for every device including a fingerprint sensory.
The accompanying drawings, which are incorporated in and form a part of the Description of Embodiments, illustrate various embodiments of the subject matter and, together with the Description of Embodiments, serve to explain principles of the subject matter discussed below. Unless specifically noted, the drawings referred to in this Brief Description of Drawings should be understood as not being drawn to scale. Herein, like items are labeled with like item numbers.
The following Description of Embodiments is merely provided by way of example and not of limitation. Furthermore, there is no intention to be bound by any expressed or implied theory presented in the preceding background or in the following Description of Embodiments.
Reference will now be made in detail to various embodiments of the subject matter, examples of which are illustrated in the accompanying drawings. While various embodiments are discussed herein, it will be understood that they are not intended to limit to these embodiments. On the contrary, the presented embodiments are intended to cover alternatives, modifications and equivalents, which may be included within the spirit and scope the various embodiments as defined by the appended claims. Furthermore, in this Description of Embodiments, numerous specific details are set forth in order to provide a thorough understanding of embodiments of the present subject matter. However, embodiments may be practiced without these specific details. In other instances, well known methods, procedures, components, and circuits have not been described in detail as not to unnecessarily obscure aspects of the described embodiments.
Some portions of the detailed descriptions which follow are presented in terms of procedures, logic blocks, processing and other symbolic representations of operations on data within an electrical device. These descriptions and representations are the means used by those skilled in the data processing arts to most effectively convey the substance of their work to others skilled in the art. In the present application, a procedure, logic block, process, or the like, is conceived to be one or more self-consistent procedures or instructions leading to a desired result. The procedures are those requiring physical manipulations of physical quantities. Usually, although not necessarily, these quantities take the form of acoustic (e.g., ultrasonic) signals capable of being transmitted and received by an electronic device and/or electrical or magnetic signals capable of being stored, transferred, combined, compared, and otherwise manipulated in an electrical device.
It should be borne in mind, however, that all of these and similar terms are to be associated with the appropriate physical quantities and are merely convenient labels applied to these quantities. Unless specifically stated otherwise as apparent from the following discussions, it is appreciated that throughout the description of embodiments, discussions utilizing terms such as “receiving,” “extracting,” “generating,” “acquiring,” “updating,” “determining,” “comparing,” “communicating,” “converting,” “authenticating” or the like, refer to the actions and processes of an electronic device such as an electrical device.
Embodiments described herein may be discussed in the general context of processor-executable instructions residing on some form of non-transitory processor-readable medium, such as program modules, executed by one or more computers or other devices. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform particular tasks or implement particular abstract data types. The functionality of the program modules may be combined or distributed as desired in various embodiments.
In the figures, a single block may be described as performing a function or functions; however, in actual practice, the function or functions performed by that block may be performed in a single component or across multiple components, and/or may be performed using hardware, using software, or using a combination of hardware and software. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, logic, circuits, and steps have been described generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the overall system. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present disclosure. Also, the example fingerprint sensing system and/or mobile electronic device described herein may include components other than those shown, including well-known components.
Various techniques described herein may be implemented in hardware, software, firmware, or any combination thereof, unless specifically described as being implemented in a specific manner. Any features described as modules or components may also be implemented together in an integrated logic device or separately as discrete but interoperable logic devices. If implemented in software, the techniques may be realized at least in part by a non-transitory processor-readable storage medium comprising instructions that, when executed, perform one or more of the methods described herein. The non-transitory processor-readable data storage medium may form part of a computer program product, which may include packaging materials.
The non-transitory processor-readable storage medium may comprise random access memory (RAM) such as synchronous dynamic random access memory (SDRAM), read only memory (ROM), non-volatile random access memory (NVRAM), electrically erasable programmable read-only memory (EEPROM), FLASH memory, other known storage media, and the like. The techniques additionally, or alternatively, may be realized at least in part by a processor-readable communication medium that carries or communicates code in the form of instructions or data structures and that can be accessed, read, and/or executed by a computer or other processor.
Various embodiments described herein may be executed by one or more processors, such as one or more motion processing units (MPUs), sensor processing units (SPUs), host processor(s) or core(s) thereof, digital signal processors (DSPs), general purpose microprocessors, application specific integrated circuits (ASICs), application specific instruction set processors (ASIPs), field programmable gate arrays (FPGAs), a programmable logic controller (PLC), a complex programmable logic device (CPLD), a discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein, or other equivalent integrated or discrete logic circuitry. The term “processor,” as used herein may refer to any of the foregoing structures or any other structure suitable for implementation of the techniques described herein. As it employed in the subject specification, the term “processor” can refer to substantially any computing processing unit or device comprising, but not limited to comprising, single-core processors; single-processors with software multithread execution capability; multi-core processors; multi-core processors with software multithread execution capability; multi-core processors with hardware multithread technology; parallel platforms; and parallel platforms with distributed shared memory. Moreover, processors can exploit nano-scale architectures such as, but not limited to, molecular and quantum-dot based transistors, switches and gates, in order to optimize space usage or enhance performance of user equipment. A processor may also be implemented as a combination of computing processing units.
In addition, in some aspects, the functionality described herein may be provided within dedicated software modules or hardware modules configured as described herein. Also, the techniques could be fully implemented in one or more circuits or logic elements. A general purpose processor may be a microprocessor, but in the alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. A processor may also be implemented as a combination of computing devices, e.g., a combination of an SPU/MPU and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with an SPU core, MPU core, or any other such configuration.
Fingerprint (fingerprint) sensors are used in devices for user authentication. Typical fingerprint sensors have a surface area of that is smaller than the complete fingerprint of a user. Because such a fingerprint sensor does not capture the complete fingerprint, and during use the user does not necessarily place their finger at exactly the same position on the fingerprint sensor, an enrollment process may be used to capture to complete fingerprint of the user. The enrollment process generally includes capturing a plurality of fingerprint images with the fingerprint sensor at different positions, in an attempt to capture the entire (or close to entire) fingerprint of the user. For example, a user interface may guide a user through the enrollment process, asking the user to move the finger between each image capture to cover the entire fingerprint. The plurality of fingerprint images thus captured, are then gathered, aggregated, combined and/or stitched together to form a complete fingerprint image or set of images that will be used in the user authentication process. This images or set of images may be referred to as the authentication reference image.
The enrollment process may require the capture of many images (e.g., 10-30 or more) per finger, where the user moves the finger between each image capture. This can be a laborious and time-consuming process, especially if more than one finger is to be used in the authentication process, since then the enrollment process is repeated for each finger. In addition, a user may have different devices with a fingerprint sensor, and the enrollment process may be required for each device or when the user replaces an old device with one sensor by a new device with another sensor. For example, each individual fingerprint sensor may have its own characteristics that defines how the real or true fingerprint (profile) is converted to the fingerprint image as captured by the sensor, requiring a separate enrollment process for each fingerprint sensor. In devices that have a display, e.g., a smartphone, the user can be guided through the enrollment process by displaying instructions, messages and/or results. However, for devices or objects that have a fingerprint sensor but no display, e.g., a digital door knob or door handle with a fingerprint sensor, performing an enrollment process requiring multiple different images, would be more difficult.
Embodiments described herein provide of methods and techniques for a simplified enrollment process for a fingerprint sensor by generating an estimated fingerprint that can be used for fingerprint enrollment and authentication. In some embodiments, a ridge/valley pattern (e.g., a two-dimensional pattern) of a fingerprint of a finger is received. A sensor image including a partial fingerprint of the finger is received, where the sensor image is captured by a fingerprint sensor. Ridge/valley characteristics (e.g., the shape or profile of the ridge/valley combination) of the fingerprint are extracted from the sensor image including the partial fingerprint. An estimated fingerprint image is generated based on the ridge/valley pattern of the fingerprint and the ridge/valley characteristics of the fingerprint.
Other embodiments described herein provide methods and techniques for cross-sensor enrollment for a fingerprint, simplifying the enrollment process across multiple devices. A ridge/valley pattern for a fingerprint is received at a second electronic device from a first electronic device, where the ridge/valley pattern may be sensor-independent. A sensor image including a partial fingerprint of the finger is captured at a fingerprint sensor of the second device. The ridge/valley characteristics of the fingerprint are extracted from the sensor image including the partial fingerprint. An estimated fingerprint image is generated based on the ridge/valley pattern of the fingerprint received from the first electronic device and the ridge/valley characteristics of the fingerprint extracted at the second electronic device.
In a first example, embodiments for generating an estimated fingerprint using a combination of a camera and a fingerprint sensor of an electronic device is described. It should be appreciated that the fingerprint sensor may be any type of fingerprint sensor using different techniques, e.g., optical, capacitive or ultrasound fingerprint sensors. In the enrollment process of the described embodiment, an image (or images) of a fingerprint acquired with a camera, e.g., a camera of a smartphone, is combined with a sensor image (or images) of a partial fingerprint acquired by the fingerprint sensor.
The described embodiments combine the ridge/valley pattern as determined by the camera image with the fingerprint characteristics as determined by the fingerprint sensor. The ridge/valley pattern is defined as the two-dimensional pattern of the fingerprint, and the ridge/valley characteristics may be defined as a cross-section of a ridge/valley combination, for example, the variation of the pixel intensity over a ridge/valley. Examples of fingerprint characteristics include the shape or profile of a ridge/valley combination, the width variation of ridges and/or valleys, the intensity variations of the ridges and/or valleys, the location of the ridge on the finger, etc. The ridge/valley characteristics of a fingerprint image captured by a camera may be different than a ridge/valley characteristics as captured by the fingerprint sensor due to the use of different capturing technology, and thus a correction is made by combining the ridge/valley characteristics based on the sensor image with the ridge/valley pattern captured by a camera.
At procedure 130, based on the camera image of the fingerprint, a ridge/valley pattern of the fingerprint is determined.
The camera image (e.g., camera image 200) may allow the capture of the complete, or at least a large part of, the user's fingerprint and its ridge/valley pattern (e.g., ridge/valley pattern 210). In some embodiments, the camera image or ridge/valley pattern may be post-processed to enable an efficient matching against images of fingerprint sensors, e.g., unrolled to compensate for the effect of the finger curvature when captured with the camera. Another effect that may be compensated for is that when the fingerprint is captured with the fingerprint sensor, the user presses the finger against a surface, while this is not the case for the fingerprint camera image. Thus, a correction for the compressing of the ridges and/or finger surface may be performed.
Different techniques may be applied to enhance the quality of the camera image of the fingerprint acquired by the camera. In one embodiment, light sources close to the camera may be controlled, e.g., to enhance the contrast of the fingerprint in the camera image. An example light source is a light-emitting diode (LED) placed close to the camera. For example, an image may be taken with the light source on and another image with the light source off. Comparison of the images may help improve the quality of the fingerprint in the camera image. In another embodiment, more than one light source may be used, and the light sources may be placed on different sides of the camera lens. By capturing images with different light sources from different sides, different sides of the ridges may be exposed to the emitted light. As such, by combining the different light sources, three-dimensional information concerning the ridge/valley characteristics or profiles may be determined.
At procedure 140, a sensor image including a partial fingerprint of the finger is captured at a fingerprint sensor. Since the fingerprint sensor is smaller than a fingerprint in most situations, this image includes a partial fingerprint image.
Embodiments described herein generate an estimated fingerprint image that can be used for authentication of the user using the fingerprint sensor just as a full fingerprint image acquired at the fingerprint sensor. For this to work, the estimated fingerprint image should include the actual ridge/valley characteristics of the fingerprint as captured by the sensor. Currently, this image would be constructed from the plurality of images captured by the fingerprint sensor, requiring the lengthy, and undesired, enrollment process. As shown at procedure 150, embodiments herein extract the ridge/valley characteristics from the partial fingerprint image captured at the fingerprint sensor, which is then combined with the complete ridge/valley pattern, as based on the fingerprint camera image, to generate an estimated fingerprint sensor image of the complete fingerprint. The estimated fingerprint sensor image may also be referred to as a synthetic fingerprint sensor image or a predicted fingerprint sensor image. The estimated fingerprint sensor image should closely resemble a sensor image taken by the fingerprint sensor of the complete fingerprint. In one embodiment, the fingerprint sensor image as captured with the fingerprint sensor is analyzed to determine the ridge/valley characteristics, e.g., the ridge/valley shape or profile, by taking a cross-section profile perpendicular to the ridge at various points in the partial fingerprint image. As an example,
At procedure 160, the ridge/valley characteristics are combined with the ridge/valley pattern to generate the full estimated fingerprint image. This means that the ridge/valley characteristics are applied to all locations of the ridge/valley pattern. In one example embodiment, the ridge/valley profile, which may be dependent on the ridge/valley spacing, is applied to a vector graph, or similar graph, representing the ridge/valley pattern. In another example embodiment, the ridge/valley characteristic as estimated from the sensor image are used to modify the ridge/valley characteristics on the camera image. This may be performed using any existing image treatment method. For example, the grey level distribution may be adapted to transform the ridge/valley characteristics from the camera image like to sensor image which may, for example, make the valley deeper, less deep, wider, or narrower. An example estimated fingerprint image 240 is shown in
In some embodiments, as shown at procedure 170, the portion of the estimated fingerprint sensor image that was also captured by the fingerprint sensor can be compared to the partial fingerprint sensor image to determine the accuracy of the estimation. In other words, the portion of estimated fingerprint image 240 within box 242 can be compared with fingerprint sensor image 220. Based on this comparison, as shown at procedure 180, a confidence can be determined how accurate the estimated fingerprint image is and with what confidence it can be applied for user authentication. Any type of known image processing technique may be used to analyze and quantify the difference between the images. For example, the two images may be subtracted, and the amplitude or variance of the remaining features may be compared to the noise level to quantify the estimation quality. A confidence threshold may be used, and if the determined confidence is below this threshold it may be that the estimation did not work accurately enough to be used in authentication. If this is the case, the user may be prompted to capture another image with the fingerprint sensor, e.g., at a slightly different position, and then the process may be repeated. By acquiring multiple images, the ridge/valley characteristics may be determined more accurately, which may lead to a better estimation of the complete fingerprint sensor image. This process may be repeated until a complete fingerprint is obtained. In some embodiments, a second threshold may be used, lower than the threshold, and if the confidence is below the second threshold, the user may be prompted to perform a conventional enrollment. In another embodiment, the prediction does not work well, but in some situations the user is prompted for enough images that can be combined to use as an authentication reference image, which then basically resembles the basic enrollment process. Alternatively, or in addition, the confidence factor may also be used to adapt the parameters of the process, e.g., how to determine the ridge/valley characteristics. For example, initially a first process may be used which is faster but less accurate, and if this does not work, a second more elaborate process may be used. For example, the first process may run locally on the device, and the second process may run in the cloud. If the first process works, the second process is not required since sharing fingerprint data in the cloud may be more prone to privacy issues.
Once the estimated fingerprint image has been generated, it may be used in user authentication as the authentication reference image. The authentication reference image may be dynamically updated with real fingerprint image captured by the fingerprint sensor during the normal usage of the device. For example, if the recently captured image with the fingerprint image is similar to the authentication reference image, the authentication reference image may be updated with the recently captured image and a certain weight may be applied to the new image. The threshold used to compare the images and perform the update may be less stringent when only the estimated fingerprint image is available. As portions of the estimated fingerprint image are replaced with the actual fingerprint image, the stringency of the threshold can increase over time. The newly acquired fingerprint sensor images may also be compared to the predicted image and as such used to improve the prediction and estimation of the estimated fingerprint image for which no fingerprint sensor image has been captured up to that point. This is similar to the embodiment described above when the confidence is too low, but now dynamically updated during actual use of the device and fingerprint sensor. In accordance with some embodiments, over time, the authentication reference image, which initially was based on the estimated finger image, may consist of actual captured images.
In some embodiments, the fingerprint camera image can also be used to assist the enrollment on the fingerprint sensor in a different manner. In one embodiment, the fingerprint camera image may be used as a guide for algorithms to enroll using multiple fingerprint sensor images that capture only a small part of the fingerprint. Combining or stitching multiple fingerprint images of the same finger is a difficult task because the fingerprint images represent only a small part of the fingerprint, so enrollment algorithms traditionally fail to indicate which portion of the finger is missing and which regions are already covered by a finger enrollment process. Therefore, many images are required so that sufficient overlap is available for the generation of the compete authentication reference image. Using the complete ridge/valley pattern as obtained using the fingerprint camera image would allow the system to estimate the location of the fingerprint images, thus reducing the images required for the enrollment. This information may improve finger coverage estimation and user guidance during enrollment because it is easier to estimate how the user should move the finger to go to the missing parts.
In another embodiment, after the generation of the full fingerprint camera image, a partial fingerprint sensor image may be acquired and compared direct to the camera image. The quality of the fingerprint camera image may be analyzed, and if the quality is good enough and differences between the fingerprint camera image and the fingerprint sensor image is below a threshold, the full fingerprint camera image may be used directly as an authentication reference, without the need for the intermediary steps of estimation a full sensor image as described in flow diagram 100. As the different parts of the fingerprint are captured by the fingerprint sensor, the authentication reference based on the camera images may be gradually and dynamically replaced by the sensor images. If the quality check of the camera image determines the quality is below a threshold and/or the difference between the fingerprint camera image and fingerprint sensor image is above a threshold, the system may decide that a process as discussed in process 100 of
The embodiments described above explain a simplified fingerprint enrollment process using different sensors (e.g., a camera and a fingerprint sensor) of an electronic device. Fingerprint data is sensitive data and security is a concern. When applying these methods within a single device, it is easier to perform it securely than if implementing across multiple devices. The following embodiments describe cross-sensor enrollment using multiple devices, and any of the techniques discussed above may be applied below also, with or without modifications.
As describe above, enrolling fingerprints is a laborious process. Currently, each time a user acquires a new electronic device including a fingerprint sensor, the user must enroll each finger being used with the fingerprint sensor. Furthermore, for some devices it may be more difficult to perform an enrollment, either because of the position, location, form factor of the device, or because the device may not have a display or other means to guide the user. Each enrollment process for each electronic device is a distinct process, and currently there is no sharing or reusing of fingerprint information available to assist or speed up the enrollment process. Embodiments described herein allow for the use or reuse of at least a portion of the fingerprint data for a first fingerprint sensor when enrolling the fingerprint at a second fingerprint sensor. It should be appreciated that the first fingerprint sensor and the second fingerprint sensor may be similar type sensors, e.g., both capacitive sensors or both ultrasonic sensors, or the first sensor and the second sensor may be different type sensors, e.g., one capacitive sensor and one ultrasonic sensor.
The same methods and techniques discussed above in relation to the camera assisted enrollment may also be applied for cross-sensor enrollment and cross-device enrollment. The term cross-sensor enrollment is defined as meaning to perform the fingerprint enrollment using a first fingerprint sensor, and then using at least a portion of the enrollment data to perform user authentication using a second fingerprint sensor. The example of the camera-assisted enrollment discussed above, is an example of cross-sensor enrollment. The term cross-device enrollment is defined as meaning to perform the fingerprint enrollment using a first fingerprint sensor on a first device, and then using the enrollment data to perform user authentication on a second device using a second fingerprint sensor.
At procedure 330, based on the authentication reference image of the fingerprint corresponding the to the first sensor, a ridge/valley pattern of the fingerprint is determined.
At procedure 350, a sensor image including a partial fingerprint of the finger is captured at a fingerprint sensor, e.g., the second sensor in the second device. Since the fingerprint sensor is smaller than a fingerprint in most situations, this image includes a partial fingerprint image.
At procedure 370, the ridge/valley characteristics are combined with the ridge/valley pattern to generate the full estimated fingerprint image. An example estimated fingerprint image 440 is shown in
In some embodiments, as shown at procedure 380, the portion of the estimated fingerprint sensor image that was also captured by the fingerprint sensor can be compared to the partial fingerprint sensor image to determine the accuracy of the estimation. In other words, the portion of estimated fingerprint image 440 within box 442 with fingerprint sensor image 420 can be compared. Based on this comparison, as shown at procedure 390, a confidence can be determined how accurate the estimated fingerprint image is and with what confidence it can be applied for user authentication. Details of procedures 380 and 390 are similarly above in procedures 170 and 180, and can used here.
As discussed in relation to the camera-assisted enrollment, the quality of the authentication reference from the first sensor may be analyzed, and the reference from the first sensor may be compared to the acquired partial fingerprint sensor image of the second sensor, and based on this analysis and comparison the best strategy is selected. If the quality of the reference from the first sensor is good, and the different between the reference image and the captured image from the second sensor is small, the reference from the first sensor may be directly used as an authentication reference for the second sensor, and this reference may be gradually updated as partial fingerprint sensor images with the second sensor are acquired. Other embodiments, as discussed in relation to the camera-assisted enrollment may also be applied here, e.g., modifying the authentication reference image from the first sensor using the ridge/valley characteristics of the second sensor using image processing techniques.
At procedure 530, the authentication reference image is processed and converted to a sensor-independent reference image or sensor-independent fingerprint reference image. In one embodiment, the sensor-independent image may be a vector image or a set of vectors describing the ridge, valleys, and/or other features of the fingerprint. In another embodiment, the sensor dependent image is converted to a sensor independent image by removing any sensor transformation. The sensor transformation is defined as the transformation of the real/actual fingerprint of the user to the finger image as captured by a particular sensor. Images captured by different sensors may be different due to the characteristics that are unique to a type of sensor or to a particular sensor, e.g., caused by non-uniformities in the sensor stack. For example, the transformation data defines how the actual ridge/valley profile is converted into the image ridge/valley profile for the capturing sensor. For example, ridges in the captured images may be narrower or wider than they actually are, or the valleys may be deeper or less deep than they actually are. In a perfect sensor, this is a unity transformation, meaning that the image ridge/valley profile is identical to the real ridge/valley profile. However, differences may occur, for example, due to any non-linear effect during image capture at the particular sensor. The transformation function may be determined during a calibration process, and may be stored on the sensor. The transformation function may be any linear or non-linear function applied to the sensor signal or converted pixel values to correct or compensate for the sensor transformation.
Once the sensor-independent reference image has been generated, as shown at procedure 540, it may be transferred to the second device. This communication may be wired, wireless, and may comprise any type of encryption to improve security. In some embodiments, to initiate the transfer, a successful authentication of the user must first be accomplished. This authentication may comprise an anti-spoof or anti-fake procedure/check to improve security. Once the sensor-independent reference image has been received on the second device, as shown at procedure 550, a fingerprint image is acquired using the second sensor of the second device. The second device may prompt the user to do so, or if the second device has no means to prompt the user (e.g., no display in case of door knob), the communication may pass through the first device, e.g., using the display or speaker of the first device.
In one embodiment, as shown at procedure 560, the acquired fingerprint image may first be verified to determine it is not a fake or spoof, for example by analyzing the ridge/valley characteristics. If it is determined the acquired fingerprint image corresponds to a real finger, as shown at procedure 570, the fingerprint image may be processed and converted to a sensor-independent image. It should be appreciated that the determination whether the fingerprint is from a real finger may also be performed at a different time in flow diagram 500, e.g., after the authentication. The conversion to a sensor-independent image may be performed as discussed above using a sensor transformation function. If the first sensor and second sensor are similar type sensors, the transformation function may be identical. If the first sensor and second sensor are different types of sensors, each sensor will have its own transformation function.
At procedure 580, the user may be authenticated by comparing the sensor-independent fingerprint image as captured using the second sensor in the second device to the sensor-independent reference image captured using the first sensor on the first device. In one embodiment, as shown at procedure 590, the successful authentication on the second device may be reported to the first device. The confirmation data communicated back to the first device may also comprise additional data concerning the fingerprint estimation process, e.g., a quality factor or confidence factor of the fingerprint estimation.
In some embodiments, timing limits may be imposed for the procedure for added security. For example, in one embodiment, after sending the sensor-independent reference image from the first device, a successful authentication of the user on the second device, and/or a confirmation back to the first device, must take place within a certain time to validate the process. If this is not the case, the reference image may be refused/deleted from the second device. An additional, successful authentication may be required on the first device after the confirmation.
Additional user verification data may also be required for the second device before the second device allows an enrollment or registration of a user on the second device. This additional validation step may also pass through the first device, or may be assisted by the first device. In some embodiments, this may be a verification code or password. In some embodiments, e.g., where multiple user should be able to use the second device and be authenticated by the second sensor, a successful authentication of another user, e.g., a master user, may be required at one or more instances during the procedure for the process to be authorized.
When communicating sensor-independent reference images from a first device to a second device, additional data may be communicated and stored on the second device with the sensor-independent reference image. For example, data and information about the user, the first device, and/or the first sensor may be stored. Knowledge about the first sensor may help correction for sensor transformations and improved thereby the accuracy of the estimated fingerprint image. In addition, or alternatively, the first device may also store data and information about the second device and/or the second sensor, to which the sensor-independent reference image was communicated. The second device may also communicate data related to future user authentications back to the first device. This data may relate to the authentication efficiency of the second sensor, such as the quality of the authentication, the confidence of the authentication, the rejection rate, etc. The data may give an indication of the successfulness of the transfer and user of the sensor-independent reference image transfer, and may be used for improvement of the process, e.g., the processing required to determine the sensor-independent reference image.
Once the second device comprises the sensor-independent reference image, it may be used for authentication of the user. The second device may then be involved in identical procedures to transmit the fingerprint enrollment and authentication data to a third device with a third fingerprint sensor. Any transfer from one device to another device may be reported to the different device, or back to the first/original/master device.
The term sensor-independent fingerprint image may refer to different types of fingerprint images and may be implemented in many different ways. In general, a sensor-independent fingerprint image refers to a fingerprint image that is processed, modified, and/or normalized so that it may be shared and used for a different fingerprint sensor. The sharing may be limited to a certain type of sensor. For example, the sensor-independency may be limited to an ultrasonic fingerprint sensor, meaning the sharing may works for different ultrasonic fingerprint sensor but not necessary on other types of sensors, such as capacitive sensors. The sensor-independency may also cover all types of different fingerprint sensors, including ultrasonic, capacitive, and/or optical sensor. When initiating the communication from the first device to the second device, or vice-versa, a verification step may be performed to make sure the process may be started. For example, a check may be performed if there is enough compatibility between the sensors, if they are the same type of sensor (if this is required), or if sharing the sensor-independent authentication data is allowed and/or possible (between the devices or groups of devices). In some embodiments and extra authorization step may be performed here, requesting permission from another device and/or user. Any of the embodiments, concepts, or principles discussed in relation to
In some embodiments, the sensor-independent fingerprint image is generated by normalizing the estimated fingerprint image.
In one embodiment, the conversion to sensor-independent fingerprint images includes converting the ridge/valley characteristics and profiles to approximate as closely as possible to actual or true ridge/valley characteristics and profiles of the actual finger and fingerprint, which may include the curvature of the finger. In many devices with fingerprint sensors, the user pushes the sensor against a hard surface, thereby deforming the finger and possible the ridge/valley profile, depending on the amount of applied pressure. However, this deformation is gradual, and by following the fingerprint image from the time of a first contact (in theory a small point) to a full fingerprint image, information about the shape of the finger and ridge/valley profile can be deduced. Therefore, in one embodiment, by analyzing the changes in the fingerprint image as the user presses his or her finger on the hard sensor surface, data can be deduced on the actual shape of the finger and ridge/valley pattern. This information may then be used to convert the finally acquired fingerprint image to the actual or true ridge/valley characteristics and profiles of the actual finger and fingerprint. In other embodiments, empirical or experimental techniques may be used to determine a conversion algorithm, matrix or method, between an acquired fingerprint image and the true fingerprint characteristics. For example, based on optical non-contact methods the fingerprints and their profiles of one or more users (crowd-sourcing) may be determined, and then the correct conversion may be determined. This conversion method may then be used as a standard in all types of sensors for which this validation may be performed. The conversion may be dependent on other characteristics of the user, if this improves accuracy, such as age, gender, size, ethnicity. This means the conversion is predetermined, and then stored in the fingerprint sensor or communicated to the fingerprint sensor when required.
In some embodiments, the authentication of the user may be performed by using the estimated fingerprint image that estimates the fingerprint authentication reference image as if captured by the sensor itself, as discussed in relation to
At procedure 630, the authentication reference image is processed and converted to a sensor-independent reference image or sensor-independent fingerprint reference image. In one embodiment, the sensor-independent image may be a vector image or a set of vectors describing the ridge, valleys, and/or other features of the fingerprint. In another embodiment, the sensor dependent image is converted to a sensor independent image by removing any sensor transformation. The sensor transformation is defined as the transformation of the real/actual fingerprint of the user to the finger image as captured by a particular sensor. Images captured by different sensors may be different due to the characteristics that are unique to a type of sensor or to a particular sensor, e.g., caused by non-uniformities in the sensor stack. For example, the transformation data defines how the actual ridge/valley profile is converted into the image ridge/valley profile for the capturing sensor.
Once the sensor-independent reference image has been generated, as shown at procedure 640, it may be transferred to the second device. This communication may be wired, wireless, and may comprise any type of encryption to improve security. In some embodiments, to initiate the transfer, a successful authentication of the user must first be accomplished. This authentication may comprise an anti-spoof or anti-fake procedure/check to improve security.
Once the sensor-independent reference image has been received on the second device, as shown at procedure 650, a fingerprint image is acquired using the second sensor of the second device. Since the fingerprint sensor is smaller than a fingerprint in most situations, this image includes a partial fingerprint image.
At procedure 670, the ridge/valley characteristics are combined with the ridge/valley pattern to generate the full estimated fingerprint image. An example estimated fingerprint image 440 is shown in
In some embodiments, as shown at procedure 680, the portion of the estimated fingerprint sensor image that was also captured by the fingerprint sensor can be compared to the partial fingerprint sensor image to determine the accuracy of the estimation. In other words, the portion of estimated fingerprint image 440 within box 442 with fingerprint sensor image 420 can be compared. Based on this comparison, as shown at procedure 690, a confidence can be determined how accurate the estimated fingerprint image is and with what confidence it can be applied for user authentication. Details of procedures 680 and 690 are similarly above in procedures 170 and 180, and can used here.
At procedure 695, the user may be authenticated by comparing the true fingerprint image as captured using the second sensor in the second device to the true fingerprint reference.
The processing to generate the sensor-independent fingerprint images may be performed using many different methods. Several embodiments are listed below, and they may be used by itself or in combination with each other. The list of embodiments is not considered to be exhaustive. In one embodiment, the processing may comprise detecting the ridges and valleys and then correction or normalizing all ridges to a certain ridge reference value and all the valleys to a valley reference value. In one example, the valley reference value may be the minimum (e.g., 0) and the ridge reference value may be the maximum (e.g., 1 or a maximum amount of gray levels). This would also remove any unwanted background signal or darkfield signals. The signal between the ridges and valleys may be scaled accordingly. In one embodiment, the ridge/valley shape may be unaltered as measured in the acquired fingerprint image.
With reference to procedure 710, a ridge/valley pattern of a fingerprint of a finger is received. In one embodiment, a camera image comprising the fingerprint of the finger is received. The ridge/valley pattern of the fingerprint is extracted from the camera image comprising the fingerprint. In one embodiment, the camera image comprising the fingerprint of the finger is acquired at a camera of an electronic device. In one embodiment, the ridge/valley pattern is received from a second device. In one embodiment, the ridge/valley pattern comprises a sensor-independent fingerprint pattern. In one embodiment, the ridge/valley pattern is comprised within a sensor-independent fingerprint reference image received from another electronic device.
In one embodiment, a camera image comprising a plurality of fingerprints of a plurality of fingers is received. The ridge/valley pattern of the fingerprint of the finger is extracted from the camera image comprising the plurality of fingerprints. In one embodiment, a second ridge/valley pattern of a second fingerprint of a second finger is extracted from the camera image comprising the plurality of fingerprints. In one embodiment, a second estimated fingerprint image is generated based at least in part on the second ridge/valley pattern of the second fingerprint and the ridge/valley characteristics of the fingerprint.
At procedure 720, a sensor image comprising a partial fingerprint of the finger is received. In one embodiment, the sensor image comprising the partial fingerprint of the finger is acquired at a fingerprint sensor of an electronic device. At procedure 730, ridge/valley characteristics of the fingerprint from the sensor image comprising the partial fingerprint are extracted. At procedure 740, an estimated fingerprint image is generated based at least on the ridge/valley pattern of the fingerprint and the ridge/valley characteristics of the fingerprint.
In one embodiment, as shown at procedure 750, a second sensor image comprising a second partial fingerprint of the finger is received. At procedure 760, the estimated fingerprint image is updated based on the second partial fingerprint.
In one embodiment, as shown at procedure 770, a portion of the estimated fingerprint image comprising the partial fingerprint is compared to the sensor image comprising the partial fingerprint. A confidence in accuracy of the estimated fingerprint image is determined based at least in part on the comparing. In one embodiment, provided the confidence in accuracy of the estimated fingerprint image does not satisfy a confidence threshold, it is determined that the estimated fingerprint image is not satisfactory for authentication.
In one embodiment, as shown at procedure 780, the user authentication is performed using the estimated fingerprint image.
As depicted in
Host processor 810 can be one or more microprocessors, central processing units (CPUs), DSPs, general purpose microprocessors, ASICs, ASIPs, FPGAs or other processors which run software programs or applications, which may be stored in host memory 830, associated with the functions and capabilities of mobile electronic device 800.
Host bus 820 may be any suitable bus or interface to include, without limitation, a peripheral component interconnect express (PCIe) bus, a universal serial bus (USB), a universal asynchronous receiver/transmitter (DART) serial bus, a suitable advanced microcontroller bus architecture (AMBA) interface; an Inter-integrated Circuit (I2C) bus, a serial digital input output (SDIO) bus, a serial peripheral interface (SPI) or other equivalent. In the embodiment shown, host processor 810, host memory 830, display 840, interface 850, transceiver 860, sensor processing unit 870, and other components of mobile electronic device 800 may be coupled communicatively through host bus 820 in order to exchange commands and data. Depending on the architecture, different bus configurations may be employed as desired. For example, additional buses may be used to couple the various components of mobile electronic device 800, such as by using a dedicated bus between host processor 810 and memory 830.
Host memory 830 can be any suitable type of memory, including but not limited to electronic memory (e.g., read only memory (ROM), random access memory, or other electronic memory), hard disk, optical disk, or some combination thereof. Multiple layers of software can be stored in host memory 830 for use with/operation upon host processor 810. For example, an operating system layer can be provided for mobile electronic device 800 to control and manage system resources in real time, enable functions of application software and other layers, and interface application programs with other software and functions of mobile electronic device 800. Similarly, a user experience system layer may operate upon or be facilitated by the operating system. The user experience system may comprise one or more software application programs such as menu navigation software; games, device function control, gesture recognition, image processing or adjusting, voice recognition, navigation software, communications software (such as telephony or wireless local area network (WLAN) software), and/or any of a wide variety of other software and functional interfaces for interaction with the user can be provided. In some embodiments, multiple different applications can be provided on a single mobile electronic device 800, and in some of those embodiments, multiple applications can run simultaneously as part of the user experience system. In some embodiments, the user experience system, operating system, and/or the host processor 810 may operate in a low-power mode (e.g., a sleep mode) where very few instructions are processed. Such a low-power mode may utilize only a small fraction of the processing power of a full-power mode (e.g., an awake mode) of the host processor 810.
Camera 835 may comprise, without limitation: a camera, and infrared camera, or other type of optical sensor for capturing an image of a person, an object or a scene. It should be appreciated that mobile electronic device 800 may include more than one camera 835. In one example, external sensor 838 is a front-side optical sensor (e.g., front-side camera) and camera 835 is a back-side optical sensor (e.g., back-side camera).
Display 840, when included, may be a liquid crystal device, (organic) light emitting diode device, or other display device suitable for creating and visibly depicting graphic images and/or alphanumeric characters recognizable to a user. Display 840 may be configured to output images viewable by the user and may additionally or alternatively function as a viewfinder for camera.
Interface 850, when included, can be any of a variety of different devices providing input and/or output to a user, such as audio speakers, touch screen, real or virtual buttons, joystick, slider, knob, printer, scanner, computer network I/O device, other connected peripherals and the like.
Transceiver 860, when included, may be one or more of a wired or wireless transceiver which facilitates receipt of data at mobile electronic device 800 from an external transmission source and transmission of data from mobile electronic device 800 to an external recipient. By way of example, and not of limitation, in various embodiments, transceiver 860 comprises one or more of: a cellular transceiver, a wireless local area network transceiver (e.g., a transceiver compliant with one or more Institute of Electrical and Electronics Engineers (IEEE) 802.11 specifications for wireless local area network communication), a wireless personal area network transceiver (e.g., a transceiver compliant with one or more IEEE 802.15 specifications for wireless personal area network communication), and a wired a serial transceiver (e.g., a universal serial bus for wired communication).
Mobile electronic device 800 also includes a general purpose sensor assembly in the form of integrated sensor processing unit (SPU) 870 which includes sensor processor 872, memory 876, fingerprint sensor 878, and at least one sensor optional sensor 880, and a bus 874 for facilitating communication between these and other components of sensor processing unit 870. In some embodiments, all of the components illustrated in sensor processing unit 870 may be embodied on a single integrated circuit. It should be appreciated that sensor processing unit 870 may be manufactured as a stand-alone unit (e.g., an integrated circuit), that may exist separately from a larger electronic device. Although depicted as a portion of mobile electronic device 800, in some embodiments, sensor processing unit 870 may be incorporated in an electronic device that is not mobile.
Sensor processor 872 can be one or more microprocessors, CPUs, DSPs, general purpose microprocessors, ASICs, ASIPs, FPGAs or other processors which run software programs, which may be stored in memory 876, associated with the functions of sensor processing unit 870.
Bus 874 may be any suitable bus or interface to include, without limitation, a peripheral component interconnect express (PCIe) bus, a universal serial bus (USB), a universal asynchronous receiver/transmitter (DART) serial bus, a suitable advanced microcontroller bus architecture (AMBA) interface, an Inter-Integrated Circuit (I2C) bus, a serial digital input output (SDIO) bus, a serial peripheral interface (SPI) or other equivalent. Depending on the architecture, different bus configurations may be employed as desired. In the embodiment shown, sensor processor 872, memory 876, fingerprint sensor 878, and other components of sensor processing unit 870 may be communicatively coupled through bus 874 in order to exchange data.
Memory 876 can be any suitable type of memory, including but not limited to electronic memory (e.g., read only memory (ROM), random access memory, or other electronic memory). Memory 876 may store algorithms or routines or other instructions for processing data received from fingerprint sensor 878 and one or more sensors 880, as well as the received data either in its raw form or after some processing. Such algorithms and routines may be implemented by sensor processor 872 and/or by logic or processing capabilities included in sensor 878.
A sensor 880 may comprise, without limitation: a temperature sensor, an atmospheric pressure sensor, an infrared sensor, an ultrasonic sensor, a radio frequency sensor, a navigation satellite system sensor (such as a global positioning system receiver), an acoustic sensor (e.g., a microphone), an inertial or motion sensor (e.g., a gyroscope, accelerometer, or magnetometer) for measuring the orientation or motion of the sensor in space, or other type of sensor for measuring other physical or environmental quantities. In some embodiments, one or more sensors 880 may be implemented using a micro-electro-mechanical system (MEMS) that is integrated with sensor processor 872 and one or more other components of SPU 870 in a single chip or package.
In some embodiments, the architecture may be simplified and the device will consist of an SPU, as illustrated SPU 970 of
What has been described above includes examples of the subject disclosure. It is, of course, not possible to describe every conceivable combination of components or methodologies for purposes of describing the subject matter, but it is to be appreciated that many further combinations and permutations of the subject disclosure are possible. Accordingly, the claimed subject matter is intended to embrace all such alterations, modifications, and variations that fall within the spirit and scope of the appended claims.
In particular and in regard to the various functions performed by the above described components, devices, circuits, systems and the like, the terms (including a reference to a “means”) used to describe such components are intended to correspond, unless otherwise indicated, to any component which performs the specified function of the described component (e.g., a functional equivalent), even though not structurally equivalent to the disclosed structure, which performs the function in the herein illustrated exemplary aspects of the claimed subject matter.
The aforementioned systems and components have been described with respect to interaction between several components. It can be appreciated that such systems and components can include those components or specified sub-components, some of the specified components or sub-components, and/or additional components, and according to various permutations and combinations of the foregoing. Sub-components can also be implemented as components communicatively coupled to other components rather than included within parent components (hierarchical). Additionally, it should be noted that one or more components may be combined into a single component providing aggregate functionality or divided into several separate sub-components. Any components described herein may also interact with one or more other components not specifically described herein.
In addition, while a particular feature of the subject innovation may have been disclosed with respect to only one of several implementations, such feature may be combined with one or more other features of the other implementations as may be desired and advantageous for any given or particular application. Furthermore, to the extent that the terms “includes,” “including,” “has,” “contains,” variants thereof, and other similar words are used in either the detailed description or the claims, these terms are intended to be inclusive in a manner similar to the term “comprising” as an open transition word without precluding any additional or other elements.
Thus, the embodiments and examples set forth herein were presented in order to best explain various selected embodiments of the present invention and its particular application and to thereby enable those skilled in the art to make and use embodiments of the invention. However, those skilled in the art will recognize that the foregoing description and examples have been presented for the purposes of illustration and example only. The description as set forth is not intended to be exhaustive or to limit the embodiments of the invention to the precise form disclosed.
This application claims priority to and the benefit of U.S. Patent Provisional Patent Application 62/621,511, filed on Jan. 24, 2018, entitled “CROSS-SENSOR FINGERPRINT ENROLLMENT AND AUTHENTICATION,” by Jonathan Baudot and Etienne DeForas, and assigned to the assignee of the present application, which is incorporated herein by reference in its entirety.
| Number | Name | Date | Kind |
|---|---|---|---|
| 4880012 | Sato | Nov 1989 | A |
| 5575286 | Weng et al. | Nov 1996 | A |
| 5684243 | Gururaja et al. | Nov 1997 | A |
| 5808967 | Yu et al. | Sep 1998 | A |
| 5867302 | Fleming | Feb 1999 | A |
| 5911692 | Hussain et al. | Jun 1999 | A |
| 6071239 | Cribbs et al. | Jun 2000 | A |
| 6104673 | Cole et al. | Aug 2000 | A |
| 6289112 | Jain et al. | Sep 2001 | B1 |
| 6292576 | Brownlee | Sep 2001 | B1 |
| 6350652 | Libera et al. | Feb 2002 | B1 |
| 6428477 | Mason | Aug 2002 | B1 |
| 6483932 | Martinez et al. | Nov 2002 | B1 |
| 6500120 | Anthony | Dec 2002 | B1 |
| 6676602 | Barnes et al. | Jan 2004 | B1 |
| 6736779 | Sano et al. | May 2004 | B1 |
| 7067962 | Scott | Jun 2006 | B2 |
| 7109642 | Scott | Sep 2006 | B2 |
| 7243547 | Cobianu et al. | Jul 2007 | B2 |
| 7257241 | Lo | Aug 2007 | B2 |
| 7400750 | Nam | Jul 2008 | B2 |
| 7433034 | Huang | Oct 2008 | B1 |
| 7459836 | Scott | Dec 2008 | B2 |
| 7471034 | Schlote-Holubek et al. | Dec 2008 | B2 |
| 7489066 | Scott et al. | Feb 2009 | B2 |
| 7634117 | Cho | Dec 2009 | B2 |
| 7739912 | Schneider et al. | Jun 2010 | B2 |
| 8018010 | Tigli et al. | Sep 2011 | B2 |
| 8139827 | Schneider et al. | Mar 2012 | B2 |
| 8255698 | Li et al. | Aug 2012 | B2 |
| 8311514 | Bandyopadhyay et al. | Nov 2012 | B2 |
| 8335356 | Schmitt | Dec 2012 | B2 |
| 8433110 | Kropp et al. | Apr 2013 | B2 |
| 8508103 | Schmitt et al. | Aug 2013 | B2 |
| 8515135 | Clarke et al. | Aug 2013 | B2 |
| 8666126 | Lee et al. | Mar 2014 | B2 |
| 8703040 | Liufu et al. | Apr 2014 | B2 |
| 8723399 | Sammoura et al. | May 2014 | B2 |
| 8805031 | Schmitt | Aug 2014 | B2 |
| 9056082 | Liautaud et al. | Jun 2015 | B2 |
| 9070861 | Bibl et al. | Jun 2015 | B2 |
| 9224030 | Du et al. | Dec 2015 | B2 |
| 9245165 | Slaby et al. | Jan 2016 | B2 |
| 9424456 | Kamath Koteshwara et al. | Aug 2016 | B1 |
| 9572549 | Belevich et al. | Feb 2017 | B2 |
| 9582102 | Setlak | Feb 2017 | B2 |
| 9582705 | Du et al. | Feb 2017 | B2 |
| 9607203 | Yazdandoost et al. | Mar 2017 | B1 |
| 9607206 | Schmitt et al. | Mar 2017 | B2 |
| 9613246 | Gozzini et al. | Apr 2017 | B1 |
| 9665763 | Du et al. | May 2017 | B2 |
| 9747488 | Yazdandoost et al. | Aug 2017 | B2 |
| 9785819 | Oreifej | Oct 2017 | B1 |
| 9815087 | Ganti et al. | Nov 2017 | B2 |
| 9817108 | Kuo et al. | Nov 2017 | B2 |
| 9818020 | Schuckers et al. | Nov 2017 | B2 |
| 9881195 | Lee et al. | Jan 2018 | B2 |
| 9881198 | Lee et al. | Jan 2018 | B2 |
| 9898640 | Ghavanini | Feb 2018 | B2 |
| 9904836 | Yeke Yazdandoost et al. | Feb 2018 | B2 |
| 9909225 | Lee et al. | Mar 2018 | B2 |
| 9922235 | Cho et al. | Mar 2018 | B2 |
| 9934371 | Hong et al. | Apr 2018 | B2 |
| 9939972 | Shepelev et al. | Apr 2018 | B2 |
| 9953205 | Rasmussen et al. | Apr 2018 | B1 |
| 9959444 | Young et al. | May 2018 | B2 |
| 9967100 | Hong et al. | May 2018 | B2 |
| 9983656 | Merrell et al. | May 2018 | B2 |
| 9984271 | King et al. | May 2018 | B1 |
| 10275638 | Yousefpor et al. | Apr 2019 | B1 |
| 10315222 | Salvia et al. | Jun 2019 | B2 |
| 10322929 | Soundara Pandian et al. | Jun 2019 | B2 |
| 10387704 | Dagan et al. | Aug 2019 | B2 |
| 10461124 | Berger et al. | Oct 2019 | B2 |
| 10478858 | Lasiter et al. | Nov 2019 | B2 |
| 10497747 | Tsai et al. | Dec 2019 | B2 |
| 10515255 | Strohmann et al. | Dec 2019 | B2 |
| 10539539 | Garlepp et al. | Jan 2020 | B2 |
| 10600403 | Garlepp et al. | Mar 2020 | B2 |
| 10656255 | Ng et al. | May 2020 | B2 |
| 10670716 | Apte et al. | Jun 2020 | B2 |
| 10706835 | Garlepp et al. | Jul 2020 | B2 |
| 10755067 | De Foras et al. | Aug 2020 | B2 |
| 20020135273 | Mauchamp et al. | Sep 2002 | A1 |
| 20030013955 | Poland | Jan 2003 | A1 |
| 20040085858 | Khuri-Yakub et al. | May 2004 | A1 |
| 20040122316 | Satoh et al. | Jun 2004 | A1 |
| 20040174773 | Thomenius et al. | Sep 2004 | A1 |
| 20050023937 | Sashida et al. | Feb 2005 | A1 |
| 20050057284 | Wodnicki | Mar 2005 | A1 |
| 20050100200 | Abiko | May 2005 | A1 |
| 20050110071 | Ema et al. | May 2005 | A1 |
| 20050146240 | Smith et al. | Jul 2005 | A1 |
| 20050148132 | Wodnicki et al. | Jul 2005 | A1 |
| 20050162040 | Robert | Jul 2005 | A1 |
| 20060052697 | Hossack et al. | Mar 2006 | A1 |
| 20060079777 | Karasawa | Apr 2006 | A1 |
| 20060230605 | Schlote-Holubek et al. | Oct 2006 | A1 |
| 20060280346 | Machida | Dec 2006 | A1 |
| 20070046396 | Huang | Mar 2007 | A1 |
| 20070047785 | Jang | Mar 2007 | A1 |
| 20070073135 | Lee et al. | Mar 2007 | A1 |
| 20070202252 | Sasaki | Aug 2007 | A1 |
| 20070215964 | Khuri-Yakub et al. | Sep 2007 | A1 |
| 20070223791 | Shinzaki | Sep 2007 | A1 |
| 20070230754 | Jain et al. | Oct 2007 | A1 |
| 20080125660 | Yao et al. | May 2008 | A1 |
| 20080150032 | Tanaka | Jun 2008 | A1 |
| 20080194053 | Huang | Aug 2008 | A1 |
| 20090005684 | Kristoffersen et al. | Jan 2009 | A1 |
| 20090182237 | Angelsen et al. | Jul 2009 | A1 |
| 20090232367 | Shinzaki | Sep 2009 | A1 |
| 20090274343 | Clarke | Nov 2009 | A1 |
| 20090303838 | Svet | Dec 2009 | A1 |
| 20100030076 | Vortman et al. | Feb 2010 | A1 |
| 20100046810 | Yamada | Feb 2010 | A1 |
| 20100113952 | Raguin et al. | May 2010 | A1 |
| 20100168583 | Dausch et al. | Jul 2010 | A1 |
| 20100195851 | Buccafusca | Aug 2010 | A1 |
| 20100201222 | Adachi et al. | Aug 2010 | A1 |
| 20100202254 | Roest et al. | Aug 2010 | A1 |
| 20100239751 | Regniere | Sep 2010 | A1 |
| 20100251824 | Schneider et al. | Oct 2010 | A1 |
| 20100256498 | Tanaka | Oct 2010 | A1 |
| 20100278008 | Ammar | Nov 2010 | A1 |
| 20110285244 | Lewis et al. | Nov 2011 | A1 |
| 20110291207 | Martin et al. | Dec 2011 | A1 |
| 20120016604 | Irving et al. | Jan 2012 | A1 |
| 20120092026 | Liautaud et al. | Apr 2012 | A1 |
| 20120095347 | Adam et al. | Apr 2012 | A1 |
| 20120147698 | Wong et al. | Jun 2012 | A1 |
| 20120224041 | Monden | Sep 2012 | A1 |
| 20120232396 | Tanabe | Sep 2012 | A1 |
| 20120238876 | Tanabe et al. | Sep 2012 | A1 |
| 20120263355 | Monden | Oct 2012 | A1 |
| 20120279865 | Regniere et al. | Nov 2012 | A1 |
| 20120288641 | Diatezua et al. | Nov 2012 | A1 |
| 20120300988 | Ivanov | Nov 2012 | A1 |
| 20130051179 | Hong | Feb 2013 | A1 |
| 20130064043 | Degertekin et al. | Mar 2013 | A1 |
| 20130127592 | Fyke et al. | May 2013 | A1 |
| 20130133428 | Lee et al. | May 2013 | A1 |
| 20130201134 | Schneider et al. | Aug 2013 | A1 |
| 20130271628 | Ku et al. | Oct 2013 | A1 |
| 20130294201 | Hajati | Nov 2013 | A1 |
| 20130294202 | Hajati | Nov 2013 | A1 |
| 20140060196 | Falter et al. | Mar 2014 | A1 |
| 20140117812 | Hajati | May 2014 | A1 |
| 20140176332 | Alameh et al. | Jun 2014 | A1 |
| 20140208853 | Onishi et al. | Jul 2014 | A1 |
| 20140219521 | Schmitt et al. | Aug 2014 | A1 |
| 20140232241 | Hajati | Aug 2014 | A1 |
| 20140265721 | Robinson et al. | Sep 2014 | A1 |
| 20140294262 | Schuckers et al. | Oct 2014 | A1 |
| 20140313007 | Harding | Oct 2014 | A1 |
| 20140355387 | Kitchens et al. | Dec 2014 | A1 |
| 20150036065 | Yousefpor et al. | Feb 2015 | A1 |
| 20150049590 | Rowe et al. | Feb 2015 | A1 |
| 20150087991 | Chen et al. | Mar 2015 | A1 |
| 20150097468 | Hajati et al. | Apr 2015 | A1 |
| 20150145374 | Xu et al. | May 2015 | A1 |
| 20150164473 | Kim et al. | Jun 2015 | A1 |
| 20150165479 | Lasiter et al. | Jun 2015 | A1 |
| 20150169136 | Ganti et al. | Jun 2015 | A1 |
| 20150189136 | Chung et al. | Jul 2015 | A1 |
| 20150198699 | Kuo et al. | Jul 2015 | A1 |
| 20150206738 | Rastegar | Jul 2015 | A1 |
| 20150213180 | Herberholz | Jul 2015 | A1 |
| 20150220767 | Yoon et al. | Aug 2015 | A1 |
| 20150241393 | Ganti et al. | Aug 2015 | A1 |
| 20150261261 | Bhagavatula et al. | Sep 2015 | A1 |
| 20150286312 | Kang et al. | Oct 2015 | A1 |
| 20150301653 | Urushi | Oct 2015 | A1 |
| 20150345987 | Hajati | Dec 2015 | A1 |
| 20150371398 | Qiao et al. | Dec 2015 | A1 |
| 20160051225 | Kim et al. | Feb 2016 | A1 |
| 20160063294 | Du et al. | Mar 2016 | A1 |
| 20160063300 | Du et al. | Mar 2016 | A1 |
| 20160070967 | Du et al. | Mar 2016 | A1 |
| 20160070968 | Gu et al. | Mar 2016 | A1 |
| 20160086010 | Merrell et al. | Mar 2016 | A1 |
| 20160092715 | Yazdandoost et al. | Mar 2016 | A1 |
| 20160092716 | Yazdandoost et al. | Mar 2016 | A1 |
| 20160100822 | Kim et al. | Apr 2016 | A1 |
| 20160107194 | Panchawagh et al. | Apr 2016 | A1 |
| 20160180142 | Riddle | Jun 2016 | A1 |
| 20160326477 | Fernandez-Alcon et al. | Nov 2016 | A1 |
| 20160350573 | Kitchens et al. | Dec 2016 | A1 |
| 20160358003 | Shen et al. | Dec 2016 | A1 |
| 20170004352 | Jonsson et al. | Jan 2017 | A1 |
| 20170330552 | Garlepp et al. | Jan 2017 | A1 |
| 20170032485 | Vemury | Feb 2017 | A1 |
| 20170075700 | Abudi et al. | Mar 2017 | A1 |
| 20170100091 | Eigil et al. | Apr 2017 | A1 |
| 20170110504 | Panchawagh et al. | Apr 2017 | A1 |
| 20170119343 | Pintoffl | May 2017 | A1 |
| 20170124374 | Rowe et al. | May 2017 | A1 |
| 20170168543 | Dai et al. | Jun 2017 | A1 |
| 20170185821 | Chen | Jun 2017 | A1 |
| 20170194934 | Shelton et al. | Jul 2017 | A1 |
| 20170200054 | Du et al. | Jul 2017 | A1 |
| 20170219536 | Koch et al. | Aug 2017 | A1 |
| 20170231534 | Agassy et al. | Aug 2017 | A1 |
| 20170255338 | Medina et al. | Sep 2017 | A1 |
| 20170293791 | Mainguet et al. | Oct 2017 | A1 |
| 20170316243 | Ghavanini | Nov 2017 | A1 |
| 20170316248 | He et al. | Nov 2017 | A1 |
| 20170322290 | Ng | Nov 2017 | A1 |
| 20170322291 | Salvia et al. | Nov 2017 | A1 |
| 20170322292 | Salvia et al. | Nov 2017 | A1 |
| 20170322305 | Apte et al. | Nov 2017 | A1 |
| 20170323133 | Tsai | Nov 2017 | A1 |
| 20170325081 | Chrisikos et al. | Nov 2017 | A1 |
| 20170326590 | Daneman | Nov 2017 | A1 |
| 20170326591 | Apte et al. | Nov 2017 | A1 |
| 20170326593 | Garlepp et al. | Nov 2017 | A1 |
| 20170326594 | Berger et al. | Nov 2017 | A1 |
| 20170328866 | Apte et al. | Nov 2017 | A1 |
| 20170328870 | Garlepp et al. | Nov 2017 | A1 |
| 20170330012 | Salvia et al. | Nov 2017 | A1 |
| 20170330553 | Garlepp et al. | Nov 2017 | A1 |
| 20170357839 | Yazdandoost et al. | Dec 2017 | A1 |
| 20180025202 | Ryshtun et al. | Jan 2018 | A1 |
| 20180032788 | Krenzer et al. | Feb 2018 | A1 |
| 20180101711 | D'Souza et al. | Apr 2018 | A1 |
| 20180107852 | Fenrich et al. | Apr 2018 | A1 |
| 20180107854 | Tsai et al. | Apr 2018 | A1 |
| 20180129849 | Strohmann et al. | May 2018 | A1 |
| 20180129857 | Bonev | May 2018 | A1 |
| 20180178251 | Foncellino et al. | Jun 2018 | A1 |
| 20180206820 | Anand et al. | Jul 2018 | A1 |
| 20180225495 | Jonsson et al. | Aug 2018 | A1 |
| 20180229267 | Ono et al. | Aug 2018 | A1 |
| 20180276443 | Strohmann et al. | Sep 2018 | A1 |
| 20180329560 | Kim et al. | Nov 2018 | A1 |
| 20180349663 | Garlepp et al. | Dec 2018 | A1 |
| 20180357457 | Rasmussen et al. | Dec 2018 | A1 |
| 20180369866 | Sammoura et al. | Dec 2018 | A1 |
| 20180373913 | Panchawagh et al. | Dec 2018 | A1 |
| 20190005300 | Garlepp et al. | Jan 2019 | A1 |
| 20190012673 | Chakraborty et al. | Jan 2019 | A1 |
| 20190018123 | Narasimha-Iyer et al. | Jan 2019 | A1 |
| 20190057267 | Kitchens et al. | Feb 2019 | A1 |
| 20190073507 | D'Souza et al. | Mar 2019 | A1 |
| 20190087632 | Raguin et al. | Mar 2019 | A1 |
| 20190102046 | Miranto et al. | Apr 2019 | A1 |
| 20190130083 | Agassy et al. | May 2019 | A1 |
| 20190171858 | Ataya et al. | Jun 2019 | A1 |
| 20190188441 | Hall et al. | Jun 2019 | A1 |
| 20190188442 | Flament et al. | Jun 2019 | A1 |
| 20190325185 | Tang | Oct 2019 | A1 |
| 20190340455 | Jung et al. | Nov 2019 | A1 |
| 20190370518 | Maor et al. | Dec 2019 | A1 |
| 20200030850 | Apte et al. | Jan 2020 | A1 |
| 20200050820 | Iatsun et al. | Feb 2020 | A1 |
| 20200050828 | Li et al. | Feb 2020 | A1 |
| 20200074135 | Garlepp et al. | Mar 2020 | A1 |
| 20200125710 | Andersson et al. | Apr 2020 | A1 |
| 20200147644 | Chang | May 2020 | A1 |
| 20200175143 | Lee et al. | Jun 2020 | A1 |
| 20200210666 | Flament | Jul 2020 | A1 |
| 20200285882 | Skovgaard Christensen et al. | Sep 2020 | A1 |
| 20200302140 | Lu et al. | Sep 2020 | A1 |
| Number | Date | Country |
|---|---|---|
| 1826631 | Aug 2006 | CN |
| 102159334 | Aug 2011 | CN |
| 105264542 | Jan 2016 | CN |
| 105378756 | Mar 2016 | CN |
| 109255323 | Jan 2019 | CN |
| 1214909 | Jun 2002 | EP |
| 2884301 | Jun 2015 | EP |
| 3086261 | Oct 2016 | EP |
| 2011040467 | Feb 2011 | JP |
| 201531701 | Aug 2015 | TW |
| 2009096576 | Aug 2009 | WO |
| 2009137106 | Nov 2009 | WO |
| 2014035564 | Mar 2014 | WO |
| 2015009635 | Jan 2015 | WO |
| 2015112453 | Jul 2015 | WO |
| 2015120132 | Aug 2015 | WO |
| 2015131083 | Sep 2015 | WO |
| 2015134816 | Sep 2015 | WO |
| 2015183945 | Dec 2015 | WO |
| 2016007250 | Jan 2016 | WO |
| 2016011172 | Jan 2016 | WO |
| 2016040333 | Mar 2016 | WO |
| 2016061406 | Apr 2016 | WO |
| 2016061410 | Apr 2016 | WO |
| 2017003848 | Jan 2017 | WO |
| 2017053877 | Mar 2017 | WO |
| 2017192895 | Nov 2017 | WO |
| 2017196678 | Nov 2017 | WO |
| 2017196682 | Nov 2017 | WO |
| 2017192903 | Dec 2017 | WO |
| 2019164721 | Aug 2019 | WO |
| Entry |
|---|
| Tang, et al., “Pulse-Echo Ultrasonic Fingerprint Sensor on a Chip”, IEEE Transducers, Anchorage, Alaska, USA, Jun. 21-25, 2015, pp. 674-677. |
| ISA/EP, International Search Report and Written Opinion for International Application No. PCT/US2017/031120, 12 pages, dated Aug. 29, 2017 (Aug. 29, 2017). |
| ISA/EP, International Search Report and Written Opinion for International Application No. PCT/US2017/031120, 13 pages, dated Sep. 1, 2017 (Sep. 1, 2017). |
| ISA/EP, International Search Report and Written Opinion for International Application No. PCT/US2017/031134, 12 pages, dated Aug. 30, 2017 (Aug. 30, 2017). |
| ISA/EP, International Search Report and Written Opinion for International Application No. PCT/US2017/031140, 18 pages, dated Nov. 2, 2017 (Nov. 2, 2017). |
| ISA/EP, International Search Report and Written Opinion for International Application No. PCT/US2017/031421 13 pages, dated Jun. 21, 2017 (Jun. 21, 2017). |
| ISA/EP, International Search Report and Written Opinion for International Application No. PCT/US2017/031426 13 pages, dated Jun. 22, 2017 (Jun. 22, 2017). |
| ISA/EP, International Search Report and Written Opinion for International Application No. PCT/US2017/031431, 14 pages, dated Aug. 1, 2017 (Aug. 1, 2017). |
| ISA/EP, International Search Report and Written Opinion for International Application No. PCT/US2017/031434, 13 pages, dated Jun. 26, 2017 (Jun. 26, 2017). |
| ISA/EP, International Search Report and Written Opinion for International Application No. PCT/US2017/031439, 10 pages, dated Jun. 20, 2017 (Jun. 20, 2017). |
| ISA/EP, International Search Report and Written Opinion for International Application No. PCT/US2017/031824, 18 pages, dated Sep. 22, 2017 (Sep. 22, 2017). |
| ISA/EP, International Search Report and Written Opinion for International Application No. PCT/US2017/031827, 16 pages, dated Aug. 1, 2017 (Aug. 1, 2017). |
| ISA/EP, International Search Report and Written Opinion for International Application No. PCT/US2017/031831, 12 pages, dated Jul. 21, 2017 (Jul. 21, 2017). |
| ISA/EP, International Search Report for International Application No. PCT/US2017/031826, 16 pages, dated Feb. 27, 2018 (Feb. 27, 2018). |
| ISA/EP, Partial International Search Report for International Application No. PCT/US2017/031140, 13 pages, dated Aug. 29, 2017 (Aug. 29, 2017). |
| ISA/EP, Partial International Search Report for International Application No. PCT/US2017/031823, 12 pages, dated Nov. 30, 2017 (Nov. 30, 2017). |
| Dausch, et al., “Theory and Operation of 2-D Array Piezoelectric Micromachined Ultrasound Transducers”, IEEE Transactions on Ultrasonics, and Frequency Control, vol. 55, No. 11;, Nov. 2008, 2484-2492. |
| Hopcroft, et al., “Temperature Compensation of a MEMS Resonator Using Quality Factor as a Thermometer”, Retrieved from Internet: http://micromachine.stanford.edu/˜amanu/linked/MAH_MEMS2006.pdf, 2006, 222-225. |
| Hopcroft, et al., “Using the temperature dependence of resonator quality factor as a thermometer”, Applied Physics Letters 91. Retrieved from Internet: http://micromachine.stanford.edu/˜hopcroft/Publications/Hopcroft_QT_ApplPhysLett_91_013505.pdf, 2007, 013505-1-031505-3. |
| Lee, et al., “Low jitter and temperature stable MEMS oscillators”, Frequency Control Symposium (FCS), 2012 IEEE International, May 2012, 1-5. |
| Li, et al., “Capacitive micromachined ultrasonic transducer for ultra-low pressure measurement: Theoretical study”, AIP Advances 5.12. Retrieved from Internet: http://scitation.aip.org/content/aip/journal/adva/5/12/10.1063/1.4939217, 2015, 127231. |
| Qiu, et al., “Piezoelectric Micromachined Ultrasound Transducer (PMUT) Arrays for Integrated Sensing, Actuation and Imaging”, Sensors 15, doi:10.3390/s150408020, Apr. 3, 2015, 8020-8041. |
| Rozen, et al., “Air-Coupled Aluminum Nitride Piezoelectric Micromachined Ultrasonic Transducers at 0.3 MHz to 0.9 MHz”, 2015 28th IEEE International Conference on Micro Electro Mechanical Systems (MEMS), IEEE, Jan. 18, 2015, 921-924. |
| Savoia, et al., “Design and Fabrication of a cMUT Probe for Ultrasound Imaging of Fingerprints”, 2010 IEEE International Ultrasonics Symposium Proceedings, Oct. 2010, 1877-1880. |
| Shen, et al., “Anisotropic Complementary Acoustic Metamaterial for Canceling out Aberrating Layers”, American Physical Society, Physical Review X 4.4: 041033., Nov. 19, 2014, 041033-1-041033-7. |
| Thakar, et al., “Multi-resonator approach to eliminating the temperature dependence of silicon-based timing references”, Hilton Head'14. Retrieved from the Internet: http://blog.narotama.ac.id/wp-content/uploads/2014/12/Multi-resonator-approach-to-eliminating-the-temperature-dependance-of-silicon-based-timing-references.pdf, 2014, 415-418. |
| ISA/EP, Partial International Search Report for International Application No. PCT/US2019/034032, 8 pages, dated Sep. 12, 2019, 8. |
| ISA/EP, International Search Report and Written Opinion for International Application # PCT/US2018/063431, pp. 1-15, dated Feb. 5, 2019. |
| ISA/EP, International Search Report and Written Opinion for International Application # PCT/US2019/015020, pp. 1-23, dated Jul. 1, 2019. |
| ISA/EP, International Search Report and Written Opinion for International Application # PCT/US2019/023440, pp. 1-10, dated Jun. 4, 2019. |
| ISA/EP, International Search Report and Written Opinion for International Application No. PCT/US2018/037364, 10 pages, dated Sep. 3, 2018. |
| “Moving Average Filters”, Waybackmachine XP05547422, Retrieved from the Internet: URL:https://web.archive.org/web/20170809081353/https//www.analog.com/media/en/technical-documentation/dsp-book/dsp_book_Ch15.pdf—[retrieved on Jan. 24, 2019], Aug. 9, 2017, 1-8. |
| Office Action for CN App No. 201780029016.7 dated Mar. 24, 2020, 7 pages. |
| “Receiver Thermal Noise Threshold”, Fisher Telecommunication Services, Satellite Communications. Retrieved from the Internet: URL:https://web.archive.org/web/20171027075705/http//www.fishercom.xyz:80/satellite-communications/receiver-thermal-noise-threshold.html, Oct. 27, 2017, 3. |
| “Sleep Mode”, Wikipedia, Retrieved from the Internet: URL:https://web.archive.org/web/20170908153323/https://en.wikipedia.org/wiki/Sleep_mode [retrieved on Jan. 25, 2019], Sep. 8, 2017, 1-3. |
| “TMS320C5515 Fingerprint Development Kit (FDK) Hardware Guide”, Texas Instruments, Literature No. SPRUFX3, XP055547651, Apr. 2010, 1-26. |
| “ZTE V7 Max. 5,5” smartphone on MediaTeck Helio P10 cpu; Published on Apr. 20, 2016; https://www.youtube.com/watch?v=ncNCbpkGQzU (Year 2016). |
| Cappelli, et al., “Fingerprint Image Reconstruction from Standard Templates”, IEEE Transactions on Pattern Analysis and Machine Intelligence, IEEE Computer Society, vol. 29, No. 9, Sep. 2007, 1489-1503. |
| Feng, et al., “Fingerprint Reconstruction: From Minutiae to Phase”, IEEE Transactions on Pattern Analysis and Machine Intelligence, IEEE Computer Society, vol. 33, No. 2, Feb. 2011, 209-223. |
| Jiang, et al., “Ultrasonic Fingerprint Sensor with Transmit Beamforming Based on a PMUT Array Bonded to CMOS Circuitry”, IEEE Transactions on Ultrasonics Ferroelectrics and Frequency Control, Jan. 1, 2017, 1-9. |
| Kumar, et al., “Towards Contactless, Low-Cost and Accurate 3D Fingerprint Identification”, IEEE Transactions on Pattern Analysis and Machine Intelligence, IEEE Computer Society, vol. 37, No. 3, Mar. 2015, 681-696. |
| Pang, et al., “Extracting Valley-Ridge Lines from Point-Cloud-Based 3D Fingerprint Models”, IEEE Computer Graphics and Applications, IEEE Service Center, New York, vol. 33, No. 4, Jul./Aug. 2013, 73-81. |
| Papageorgiou, et al., “Self-Calibration of Ultrasonic Transducers in an Intelligent Data Acquisition System”, International Scientific Journal of Computing, 2003, vol. 2, Issue 2 Retrieved Online: URL: https://scholar.google.com/scholar?q=self-calibration+of+ultrasonic+transducers+in+an+intelligent+data+acquisition+system&hl=en&as_sdt=0&as_vis=1&oi=scholart, 2003, 9-15. |
| Ross, et al., “From Template to Image: Reconstructing Fingerprints from Minutiae Points”, IEEE Transactions on Pattern Analysis and Machine Intelligence, IEEE Computer Society, vol. 29, No. 4, Apr. 2007, 544-560. |
| Zhou, et al., “Partial Fingerprint Reconstruction with Improved Smooth Extension”, Network and System Security, Springer Berlin Heidelberg, Jun. 3, 2013, 756-762. |
| ISA/EP, International Search Report and Written Opinion for International Application No. PCT/US2020/039452, 11 pages, dated Sep. 9, 2020. |
| ISA/EP, Invition to Pay Additional Fees for International Application No. PCT/US2020/033854, 10 pages, dated Sep. 8, 2020. |
| ISA/EP, International Search Report and Written Opinion for International Application No. PCT/US2020/033854, 16 pages, dated Nov. 3, 2020. |
| ISA/EP, International Search Report and Written Opinion for International Application No. PCT/US2020/039208, 10 pages, dated Oct. 9, 2020. |
| ISA/EP, International Search Report and Written Opinion for International Application No. PCT/US2020/042427, 18 pages, dated Dec. 14, 2020. |
| ISA/EP, Partial Search Report and Provisional Opinion for International Application No. PCT/US2020/042427, 13 pages, dated Oct. 26, 2020. |
| Office Action for CN App No. 201780029016.7 dated Sep. 25, 2020, 7 pages. |
| Tang, et al., “11.2 3D Ultrasonic Fingerprint Sensor-on-a-Chip”, 2016 IEEE International Solid-State Circuits Conference, IEEE, Jan. 31, 2016, 202-203. |
| ISA/EP, International Search Report and Written Opinion for International Application No. PCT/US2019061516, 14 pages, dated Mar. 12, 2020. |
| ISA/EP, International Search Report and Written Opinion for International Application No. PCT/US2021/021412, 12 pages, dated Jun. 9, 2021. |
| Taiwan Application No. 106114623, 1st Office Action, dated Aug. 5, 2021, pp. 1-8. |
| Number | Date | Country | |
|---|---|---|---|
| 20190228206 A1 | Jul 2019 | US |
| Number | Date | Country | |
|---|---|---|---|
| 62621511 | Jan 2018 | US |
