Patent Application
20240249548
The present disclosure relates to fingerprint image generation, and in particular to a method and device for fingerprint image generation for saving memory, reducing the amount of bandwidth, and improving efficiency of a fingerprint recognition process.
Biometric data processing or biometrics authentication is utilized for identifying individuals. Each person's biometrics, such as fingerprints, are unique to the person.
In order to get better images, a fingerprint sensor array takes fingerprint images. For example, the fingerprint sensor array has 270×270 pixels and each picture's data size is 270×270×12 bits. As a result, in order to capture the six images, the static random access memory (SRAM) size needed to store the six images is a minimum storage size for 270×270×12-bit×6 or 6 pieces of 270×270×12 bits of data.
Unfortunately, this amount of data results in slow fingerprint acquisitions, excessive bandwidth requirements, and increased data transfer sizes.
Therefore, it is desirable to have improvements in fingerprint recognition systems that increase the efficiency of the fingerprint image generation process.
An objective of the present disclosure is to provide a method and device of fingerprint image generation for saving memory.
In an embodiment of the present invention the method of fingerprint image generation for saving memory comprises receiving a plurality of touch sensing values from a touch display panel and determining which touch sensing electrodes detect a touch event; taking an area corresponding to a number of touch sensing electrodes, which are a part of a plurality of touch sensing electrodes determined as detecting the touch event, as an effective fingerprint area, wherein the effective fingerprint area is determined by comparing a preset value and touch sensing values sensed by the plurality of touch sensing electrodes those are determined as detecting the touch event. The method further comprises performing fingerprint sensing in a fingerprint readout area and storing a first fingerprint image corresponding to the effective fingerprint area.
In an embodiment of the present invention the device for fingerprint image generation for saving memory comprises a touch controller and a fingerprint readout circuit. The touch controller is configured to receive a plurality of touch sensing values from a touch display panel and determining which touch sensing electrodes detect a touch event, and take an area corresponding to a number of touch sensing electrodes, which are a part of a plurality of touch sensing electrodes determined as detecting the touch event, as an effective fingerprint area. The effective fingerprint area is determined by comparing a preset value and touch sensing values sensed by the plurality of touch sensing electrodes those are determined as detecting the touch event. The fingerprint readout circuit is configured to perform fingerprint sensing in a fingerprint readout area and storing a first fingerprint image corresponding to the effective fingerprint area.
As a result of utilizing the present invention, the memory is saved, transmission time is reduced, and user experience is improved.
To achieve at least the above objectives, the present disclosure provides a method and device for fingerprint image generation for fingerprint image generation for saving memory, reducing the amount of bandwidth, and improving efficiency of the fingerprint recognition process.
To facilitate understanding of the object, characteristics and effects of this present disclosure, embodiments together with the attached drawings for the detailed description of the present disclosure are provided.
Refer to
A fingerprint readout circuit 100 can be embodied as a microcontroller or can be integrated with a display driving circuit and a touch control circuit into one single chip, i.e. a fingerprint, touch, and display integration (FTDI) 220, for driving the touch screen of an electronic device, such as a cell phone. A fingerprint sensor is a fingerprint sensing pixel array 210, which can be embedded in a touch screen or provided on the touch screen. The fingerprint readout circuit 100 acquires multiple images from the fingerprint sensor to output multiple pages of fingerprint image data, which are transmitted to a back-end processor 240, such as a system application processor of a cell phone. The back-end processor 240 performs image processing based on the multiple pages of images to obtain clearer fingerprint images for fingerprint verification, i.e. to verify whether the input fingerprint is a user's registered fingerprint.
Supposing that one page of fingerprint image data has a data volume of 270×270×12 bits (based on the active fingerprint sensor array has 270×270 pixels) and an image is acquired six times, for example. Then, the fingerprint readout circuit would need a space in SRAM 140 as large as 270×270×12 bits×6 for storing these six pages of images, forming a considerable usage of SRAM 140.
In cases when the fingerprint readout circuit 100 is a microcontroller, the SRAM 140 is located in the microcontroller; or, in cases when the fingerprint readout circuit 100 is integrated with a display driving circuit and a touch control circuit 230 into one single chip (FTDI) 220, the SRAM 140 is located in the integrated chip.
The fingerprint readout integrated circuit (ROIC) 100 further comprises analog front end (AFE) channels 110, an analog to digital converter (ADC) 120, digital circuitry 130, and an interface 150 connecting the fingerprint readout IC 100 to the back-end processor 240.
Refer to
In the embodiment illustrated in
Although the fingerprint data is obtained at different exposure times, they should have very closely related data values. Therefore, for other images obtained after the first page of image data, only the difference value between them and their respective preceding pages of image data is stored to save SRAM space and reduce the data volume transmitted by the fingerprint sensing module 210 to the back-end processor 240, such as the application processor (AP) of a cell phone, or in other words, to save the transmission bandwidth.
For example, when the stored data volume of the first page of image data is 270×270×12 bits and the stored data volume of each of the second to the sixth pages of images is 270×270×10 bits (i.e. only the difference value between each image and its preceding page of image data is stored, and ¾ of the space of SRAM 140 can be saved.
When the fingerprint readout circuit 100 acquires the first page of image data, it will store the full data of 270×270×12 bits of the first image in SRAM 140.
When the fingerprint readout circuit 100 acquires the second page of image data, it will calculate line by line to obtain the difference value between the second and the first page of image data (i.e. to obtain the difference in data at the same finger pixel position of the two pages of images) and stores only the difference value in SRAM 140. Since multiple pages of images are acquired in a continuous exposure time period, and the second page of image data has an exposure time longer than that of the first page of image data, a positive code will be obtained from the above subtraction of first page of image data from the second page of image data. Referring to the tables illustrated in
In this embodiment, the code generated by the ADC is higher when the exposure time is longer (this is due to the analog voltage converted by the ADC becomes larger because of the longer exposure time). On the other hand, in embodiments where the code is lower when the exposure time is longer, the difference value is obtained by Image(N)−Image(N−1).
The following is given as an example in application. The method for fingerprint image generation for saving memory in this example comprises capturing six fingerprint images of six different exposures.
The fingerprint first image is captured at a complete 270×270×12 bits of data.
The second image is then captured. The first image is subtracted from the second image (the image data at the same position is subtracted) to obtain the difference between the first and second image. Since the second image has a longer exposure than the first image, the difference between first image and the second image will be a positive number when the first image is subtracted from the second image.
The method then obtains the difference in data at the same finger pixel position of the two pages of images. As shown in
Refer to
When the fingerprint readout circuit 100 acquires the third page of image data, it calculates line by line to obtain the difference value between the third and the second page of image data (i.e. to obtain the difference of data at the same finger pixel position of the two pages of images) and stores only the difference value in SRAM 140. Since multiple pages of images are acquired in a continuous exposure time period, and the third page of image data has an exposure time longer than that of the second page of image data, a positive code will be obtained from the above subtraction of the second image data from the third image data. Referring to the tables illustrated in
The image is acquired from the fingerprint sensing pixel array 210 row by row; and each time one page of image data is acquired, one line of data is stored in a buffer 141. Since the first page of image data does not have a preceding image, it is completely stored in the SRAM 140 at the original data size.
The following is given as an example in application. The method for fingerprint image generation for saving memory in this example comprises capturing six fingerprint images of six different exposures.
In the example, the third image is captured. The second image is subtracted from the third image (the same position is subtracted) to obtain the difference between the second and the third image. Since the third image has a longer exposure than the second image, the difference between the second image and the third image will be a positive number when the second image is subtracted from the third image.
This process is performed for all six pictures (image one, image two, image three, image four, image five, and image six).
As shown in
Since the fingerprint pictures are exposed at different times, the RAW data should be close, so only the differences are stored, which can save SRAM space and bandwidth (reduce the amount of data sent by SPI to AP).
For example, the first image is 270×270×12 bits. The second image is 270 10×270×10 bits (save the difference, save ¾ of the original SRAM space). The third image is 270×270×10 bits (save the difference, save ¾ of the original SRAM space). The fourth image is 270×270×10 bits (save the difference, save ¾ of the original SRAM space). The fifth image is 270×270×10 bits (save the difference, save ¾ of the original SRAM space). The sixth image is 270×270×10 bits (save the difference, save ¾ of the original SRAM space).
The RAW data of the fingerprint images exposed at different times will be close. The SRAM 140 only stores the original data size of the first image, and all others images have the difference stored. Assuming it is incremental data, a subtractive method is utilized.
As described above in regard to
Similarly, the first image's complete second fingerprint image data (897) is stored at original data size, the second image minus the first image's second fingerprint image data (246) is stored, and the third image minus the second image's second fingerprint image data (290) is stored.
Refer to
When acquiring the second page of image data, since the sensed voltage (
Starting from the second page of image data, only the difference value between the codes of any two corresponding lines in two consecutive pages of images is stored in SRAM 140.
The fingerprint readout circuit 100 transmits the complete data of the first page of image data and the difference values obtained for the subsequent pages of images to the back-end processor 240 (such as the AP of a cell phone) via the SPI (serial peripheral interface) 150.
As illustrated in
The selection signals Sel at the lines in the fingerprint sensing pixel array 210 are also the G1-Gn in the sequence diagram shown in
Refer to
The FPR GOA circuit 300 receives the STV signal and the clock signals Ck1/Ck1b and Ck2/Ck2b output by the fingerprint readout circuit 100. The STV signal is shifted according to the clock signals Ck1/Ck1b, and the selection signals Sel at the lines of the fingerprint sensing array 210 are then generated according to the clock signals Ck2/Ck2b, i.e. the G1˜Gn in the sequence diagrams illustrated in
Refer to
Since all the pages of images are very close in their background values, the minimum code can be subtracted from the data of each page of images before storing the image data in SRAM 140. However, it is necessary to correspondingly record the minimum code of each page of images.
In this way, for each fingerprint pixel, the 12-bit raw data of each page of image data can be represented in 10-bits and plus the minimum code of the page of image data to thereby save SRAM space to be used. In the tables illustrated in
First, the current Image N is stored in a buffer 141, which can be an area in the SRAM 140 for temporarily storing data. Before a next page of image data is stored in the buffer 141, the minimum value of the pixel data (i.e. the minimum code or minimal pixel data) of the current image is obtained. Then, the minimal pixel value is subtracted from each of the pixel data of the current image before storing the data of the current image in SRAM 140, so as to reduce the required SRAM space.
The fingerprint readout circuit 100 transmits the minimal pixel data of all pages of images and the data of difference values to the AP via the SPI interface 150, and the AP can recover the image data according to the stored minimal pixel data and difference values.
The following is given as an example in application. The method for fingerprint image generation for saving memory in this example comprises capturing plurality of fingerprint images of different exposures.
In this example, the method comprises the following. Since the background value of each image is close, before saving into SRAM 140, the minimum RAW data value of each image is recorded and the minimum RAW Data of each image is cut.
For example, 2 to the 12th power of raw data, can be expressed as 2 to the 10th power+minimum raw data, which can reduce the required storage space of SRAM, and the data can be restored after the serial peripheral interface (SPI) 150 sends it to the application program (AP). The lower tables in bold in
For example, the first page of image data is captured. Among the first page of image data it is determined that the minimal pixel value is 716. Then, 716 is subtracted from all first page image data and stored. Since 716 is the minimal pixel value, the result for this image data is 0.
Then the second page of image data is captured. Among the second page of image data it is determined that the minimal pixel value is 986. Then, 986 is subtracted from all second page image data and stored. For example, the original image data was 1035, the result for this image data is 49.
Then the third page of image data is captured. Among the third page of image data it is determined that the minimal pixel value is 1231. Then, 1231 is subtracted from all third page image data and stored. For example, the original image data was 1403, the result for this image data is 172.
This process is repeated for all of the plurality of captured images.
Refer to
In this embodiment before storing the pages of images (including the first page of images) in SRAM 140, the method first subtracts the minimum pixel data of the first page of image data from the data of each of the pages of images.
In this way, for each fingerprint pixel, the 12-bit raw data of each page of images can be represented in 10-bits and plus the minimal pixel value of the first page of image data to thereby save SRAM space to be used. In the tables illustrated in
First, the complete image of the first page of image data is stored in a buffer 141, which can be an area in SRAM 140 for temporarily storing data. Before the second page of image data is stored in the buffer 141, the minimum value of the pixel data (i.e. the minimal pixel data) of the first page of image data is obtained and stored in the minimum buffer 142, and the minimal pixel data of the first page of image data in the minimum buffer 142 is stored in SRAM 140. Then, the minimal pixel data of the first page of image data stored in the minimum buffer 142 is subtracted from each of the pixel data of the second page of image data and the obtained difference values are stored in SRAM 140, so as to reduce the required SRAM space.
The fingerprint readout circuit 100 transmits the minimal pixel data of the first page of image data and the data of the difference value between the minimal pixel data of the first page of image data and each of the other pages of image data to the AP via the SPI interface 150, and the AP can recover the image data according to the stored minimal pixel data and the difference value data.
Refer to
The area that can be sensed by the fingerprint readout circuit 100 (i.e. the area framed by the solid line in
In this embodiment, the fingerprint has an effective area determined according to a touch sensing value, as the area framed by the dash lines shown in
When sensing the fingerprint, only the data of the fingerprint image that is in the effective fingerprint area will be stored in the SRAM 140 by the fingerprint readout circuit, and data of the fingerprint image outside of the effective fingerprint area, such as the area in the above picture located between the solid line and the dashed lines, is not stored in SRAM 140, in order to save SRAM space. Data of the ineffective fingerprint area is also not transmitted to the back-end processor 240, in order to save SRAM in the fingerprint readout circuit and the time for transmitting the data to the AP, which in turn shortens the unlocking time and enhances the user's experience.
The touch electrodes highlighted in
A larger touch sensing value indicates the fingerprint image sensed in the area is quite clear. For the back-end processor 240, the fingerprint verification can be done according to a partial but relatively clear fingerprint image, and not necessarily according to a complete fingerprint image.
After obtaining the touch sensing values of the touch electrodes from the touch sensing circuit, the touch controller 235 can determine, according to a critical value, which of the touch electrodes have detected the occurrence of a touch. For example, the touch electrodes highlighted in
Further, the touch controller 235 determines which of the touch electrodes have a touch sensing value larger than a preset value, and the area corresponding to the touch electrodes having touch sensing values larger than the preset value is used as the effective fingerprint area. The preset value would influence the area size of the effective fingerprint area. For example, when the preset value is 150, the area corresponding to the 9 touch electrodes enclosed in the outer dashed lines of
Refer to
An effective numerical range of fingerprint data is defined between a code higher than a middle code of the fingerprint data range (e.g. code 2000) by a predetermined range and a code lower than the middle code by a predetermined range, as the area with the solid frame shown in the ADC code distribution diagram illustrated in
The fingerprint readout circuit 100 calculates according to the received sensing voltages to obtain corresponding ADC codes. In the event that the obtained ADC codes fall in the negligible numerical range, they are not stored in SRAM 140 to save SRAM space. In this way, only the fingerprint image data within the effective numerical range is transmitted by the fingerprint readout circuit to the back-end processor 240, while the negligible fingerprint image data is not transmitted to further save the time needed for data transmission and to enhance the user's experience.
Taking the fingerprint image acquired under a strong light as an example, the effective numerical range of the fingerprint data is approximately between codes 1100 and 2400. This is because the effective numerical range of fingerprint data is in connection with the light strength during the fingerprint sensing array exposure. Actually, the effective numerical range of the fingerprint data won't distribute with the middle value of the ADC code full range (e.g. code 0˜4095 in 12-bit ADC) as a center.
In an embodiment of the present invention, the method comprises eliminating the two most significant bits (MSB) from the original data size of 270×270×12 bits to generate a compressed image data size of 270×270×10 bits.
In embodiments of the present invention, a combination of embodiments is used to implement the method of fingerprint image generation for saving memory of the present invention.
While the present disclosure has been described by means of specific embodiments, numerous modifications and variations could be made thereto by those skilled in the art without departing from the scope and spirit of the present disclosure.
This application is a divisional application of and claims the priority benefit of U.S. application Ser. No. 17/548,602, filed on Dec. 13, 2021, now allowed. The U.S. application Ser. No. 17/548,602 claims the priority benefit of U.S. provisional patent application Ser. No. 63/124,103, filed on Dec. 11, 2020. The entirety of the above-mentioned patent application is hereby incorporated by reference and made a part of this specification.
| Number | Date | Country | |
|---|---|---|---|
| 63124103 | Dec 2020 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | 17548602 | Dec 2021 | US |
| Child | 18625159 | US |
