Patent Application
20190019000
This application claims priority from Korean Patent Application No. 10-2017-0089143, filed on Jul. 13, 2017 in the Korean Intellectual Property Office, Korean Patent Application No. 10-2018-0001861, filed on Jan. 5, 2018 in the Korean Intellectual Property Office, and Korean Patent Application No. 10-2018-0008187, filed on Jan. 23, 2018, in the Korean intellectual property Office, the disclosures of which are incorporated herein by reference in their entireties.
Apparatuses and methods consistent with example embodiments relate to an electronic device, and more particularly, relate to an optics-based fingerprint sensor, an electronic device including the optics-based fingerprint sensor, and an operation method of the electronic device.
Electronic devices provide various functions, based on operations of various electronic circuits/modules/chips included therein. Electronic devices such as a computer, a smartphone, a tablet, and the like include electronic circuits, modules, or chips for the purpose of providing various functions.
As the security of an electronic device becomes more important, the electronic device may perform various user authentication functions to provide a service only to an authenticated user. For example, fingerprint detection and recognition is one of the widely used methods of user authentication. The electronic device may provide user authentication by detecting the user's fingerprint and determining whether the detected user fingerprint matches an authentication fingerprint (i.e., reference fingerprint data) stored in advance.
Conventionally, fingerprint recognition methods have included a capacitive detection method and an optics-based detection method. In the case of the optics-based fingerprint recognition method, the level of security may be relatively low because the user fingerprint can be more easily replicated (or duplicated).
Embodiments of the present disclosure provide an optics-based fingerprint sensor with improved security, an electronic device including the optics-based fingerprint sensor, and an operation method of the electronic device, by determining whether a recognized fingerprint is a fingerprint coming from a finger of a real person or a fake fingerprint, in recognition and detection of a user fingerprint.
According to an aspect of an example embodiment, an optics-based fingerprint sensor includes a first photodiode, a first metal shield structure that is formed on the first photodiode and includes a first opening, a first color filter structure that is formed on the first metal shield structure and includes a second opening exposing the first opening, a second photodiode that is spaced apart from the first photodiode by a predetermined distance, a first nano-optical filter structure that is formed on the second photodiode, and a first band limit filter structure that is formed on the nano-optical filter structure.
According to an aspect of an example embodiment, an optics-based fingerprint sensor may include a first fingerprint pixel that outputs a first signal, based on a first intensity of light reflected from a user fingerprint; a first spectrum pixel that outputs a second signal, based on a second intensity of first narrow band light, which has a first wavelength, of the reflected light; and a sensor driver. The sensor driver may be configured to control the first fingerprint pixel and the first spectrum pixel, to generate fingerprint image information about the user fingerprint, based on the first signal, and generate fingerprint spectrum information about the user fingerprint, based on the second signal.
According to an aspect of an example embodiment, an electronic device may include a display panel including a plurality of display pixels, and an optics-based fingerprint sensor configured to generate fingerprint image information and fingerprint spectrum information, based on reflected light. The reflected light may be light emitted from at least one display pixel of the plurality of display pixels and reflected from a user fingerprint. The optics-based fingerprint sensor may include a plurality of fingerprint pixels each configured to generate a first electrical signal, based on the reflected light; a plurality of spectrum pixels each configured to generate a second electrical signal, based on different narrow band light of the reflected light; and a sensor driver configured to generate the fingerprint image information based on the first electrical signal from each of the plurality of fingerprint pixels, and generate the fingerprint spectrum information based on the second electrical signal from each of the plurality of spectrum pixels.
According to an aspect of an example embodiment, a method of operating an electronic device including an optics-based fingerprint sensor may include sensing a touch from a user; allowing at least a part of a plurality of display pixels included in a display panel to emit light, in response to the sensed touch; obtaining fingerprint image information based on light reflected from a user fingerprint after being emitted from the at least a part of the plurality of display pixels; obtaining fingerprint spectrum information based on a plurality of narrow band light of the reflected light; performing a fingerprint matching operation of comparing the obtained fingerprint image information and preset fingerprint image information; performing a liveness detection operation of comparing the obtained fingerprint spectrum information and preset fingerprint spectrum information; determining whether a user authentication is successful, based on results of the fingerprint matching operation and the liveness detection operation, and performing a corresponding operation depending on a result of the determining.
The above and other objects and features of the present disclosure will become apparent by describing in detail example embodiments thereof with reference to the accompanying drawings In which:
Reference will now be made in detail to example embodiments, with reference to the accompanying drawings. In the drawings, parts irrelevant to the description are omitted to clearly describe the example embodiments, and like reference numerals refer to like elements throughout the specification. In this regard, the present example embodiments may have different forms and should not be construed as being limited to the descriptions set forth herein.
The fingerprint sensor 110 may be configured to sense or detect a fingerprint of a user. That is, the fingerprint sensor 110 may be a fingerprint detection sensor or a fingerprint recognition sensor that detects a user fingerprint for the purpose of authenticating a specific user or providing a specific authority to the specific user. In an example embodiment, the fingerprint sensor 110 may be an optics-based fingerprint detecting sensor. However, the scope and spirit of the present disclosure is not limited thereto.
The fingerprint sensor 110 may include a fingerprint sensing pixel array 111 and a sensor driver 112. The fingerprint sensing pixel array 111 may include a plurality of optical pixels. Each of the plurality of optical pixels may be configured to detect the intensity of light from the outside. In an example embodiment, the plurality of optical pixels of the fingerprint sensing pixel array 111 may be divided into a plurality of fingerprint pixels FPX and a plurality of spectrum pixels SPX. That is, the fingerprint sensing pixel array 111 may include the plurality of fingerprint pixels FPX and the plurality of spectrum pixels SPX.
Each of the plurality of fingerprint pixels FPX may be configured to sense light reflected from a user fingerprint. That is, the plurality of fingerprint pixels FPX may be configured to capture an image (i.e., fingerprint image information) of the user fingerprint.
Each of the plurality of spectrum pixels SPX may be configured to sense the intensity of light, which is associated with a specific wavelength or a narrow band wavelength (hereinafter referred to as “narrow band light”), of the light reflected from the user fingerprint. For example, a first spectrum pixel of the plurality of spectrum pixels SPX may detect the intensity of first narrow band light of a first wavelength, and a second spectrum pixel thereof may detect the intensity of second narrow band light of a second wavelength different from the first wavelength.
In an example embodiment, each of the plurality of fingerprint pixels FPX may be configured to detect light of a specific band (e.g., a first band). In contrast, each of the plurality of spectrum pixels SPX may detect narrow band light of bands that are narrower than the first band. In detail, a first fingerprint pixel of the plurality of fingerprint pixels FPX may be configured to detect light of a visible ray range (i.e., visible spectrum). In contrast, the first spectrum pixel of the plurality of spectrum pixels SPX may be configured to detect narrow band light of a wavelength (e.g., a wavelength of about 550 nm) of a narrow band. The second spectrum pixel of the plurality of spectrum pixels SPX may be configured to detect light of any other wavelength (e.g., a wavelength of about 560 nm). That is, fingerprint spectrum information about a specific wavelength band (e.g., 500 nm to 650 nm) may be obtained by the plurality of spectrum pixels SPX.
In other words, information detected by the plurality of fingerprint pixels FPX may indicate the intensity of light in the entire specific wavelength band, while fingerprint spectrum information detected by the plurality of spectrum pixels SPX may indicate the intensity of light for each wavelength.
In an example embodiment, the plurality of spectrum pixels SPX may respectively include nano-optical filters having different optical characteristics, for the purpose of detecting light of different narrow band wavelengths. A fingerprint pixels FPX and a spectrum pixels SPX will be more fully described with reference to the following drawings.
In an example embodiment, the information (i.e., the fingerprint image information) detected by the plurality of fingerprint pixels FPX may be used in a fingerprint matching operation for user authentication, and the information (i.e., the fingerprint spectrum information) detected by the plurality of spectrum pixels SPX may be used in a liveness detection operation for user authentication (in particular, fake fingerprint detection).
As described above, the fingerprint sensor 110 may determine whether a detected fingerprint is a fingerprint obtained from a finger of a real person, through the liveness detection operation, in addition to the fingerprint matching operation for a user fingerprint. In other words, whether a user fingerprint is forged may be determined through the liveness detection operation.
The sensor driver 112 may be configured to control the fingerprint sensing pixel array 111 under the control of the AP 101. For example, the sensor driver 112 may control the fingerprint sensing pixel array 111 such that fingerprint image information is obtained from the plurality of fingerprint pixels FPX of the fingerprint sensing pixel array 111 and fingerprint spectrum information FSI is obtained from the plurality of spectrum pixels SPX.
The touch panel 121 may include a plurality of sensing lines. The plurality of sensing lines may be arranged in a row direction and a column direction. Each of the plurality of sensing lines of the touch panel 121 may sense a user touch under the control of the TDI 122 and may provide the sensed signal to the TDI 122.
The TDI 122 may control the touch panel 121 and sense a touch of a user, based on a signal from the touch panel 121. For example, the TDI 122 may provide a sensing signal to a plurality of sensing lines and sense a touch of the user or a touch location or a touch area on the touch panel 121, based on a change of a signal received from the touch panel 121.
The touch panel 121 and the TDI 122 described above are an example, and the scope and spirit of the present disclosure is not limited thereto. The touch panel 121 and the TDI 122 may be implemented depending on various manners such as a mutual capacitance manner, a self-capacitance manner, and the like.
The display panel 131 may include a plurality of pixels for displaying image information to the outside. Each of the plurality of pixels may be an organic light-emitting diode (OLED) pixel, but the scope and spirit of the present disclosure is not limited thereto. Under the control of the DDI 132, the display panel 131 may display image information or allow pixels of a specific area to emit light.
According to an example embodiment illustrated in
According to an example embodiment illustrated in
The AP 101 may control overall operations of the electronic device 100. The AP 101 may control the sensor driver 112, the TDI 122, and the DDI 132 depending on an operation mode of the electronic device 100, an input of the user, or various maintenance operations.
In an example embodiment, the touch panel 121 and the display panel 131 may include fingerprint sensing areas SA and SA′, respectively. The fingerprint sensing areas SA and SA′ may correspond to an area where a user fingerprint is touched, for the purpose of sensing or detecting the user fingerprint.
The sensing areas SA and SA′ of the panels 121 and 131 and the fingerprint sensor 110 may be placed to overlap each other. For example, the sensing area SA′ of the display panel 131 may be placed under the sensing area SA of the touch panel 121, and the fingerprint sensor 110 (or the fingerprint sensing pixel array 111) may be placed under the sensing area SA′ of the display panel 131.
When a touch of the user is detected, the AP 101 may control the DDI 132 to allow the sensing area SA′ of the display panel 131 (or some pixels of the sensing area SA′) to emit light. The DDI 132 may allow some pixels of the sensing area SA′ of the display panel 131 to emit light under the control of the AP 101.
Afterwards, when the user fingerprint approaches or contacts the sensing area SA of the touch panel 121, the AP 101 may control the DDI 132 to allow all or some pixels of the sensing area SA′ of the display panel 131 to emit light. In this case, the amount of light emitted by the pixels may be greater than before a fingerprint touch. Alternatively, the amount of light emitted by the pixels may be greater than the amount of light set in advance by the user. Alternatively, the amount of light emitted by the pixels may be a maximum value.
The light emitted from all or some pixels of the sensing area SA′ of the display panel 131 may be reflected by the user fingerprint, and the reflected light may be provided to the fingerprint sensor 110. The fingerprint sensor 110 may capture light reflected from the user fingerprint to obtain user fingerprint image information and fingerprint spectrum information. In an example embodiment, the user fingerprint image information may be obtained by the plurality of fingerprint pixels FPX, and the fingerprint spectrum information may be obtained through the plurality of spectrum pixels SPX. That is, display pixels emitting light may be used as a light source for detecting the fingerprint image information and the fingerprint spectrum information.
The AP 101 may perform a fingerprint matching operation, based on the obtained fingerprint image information and may determine whether the obtained fingerprint image information is an authenticated fingerprint, based on the result of the fingerprint matching operation; the AP 101 may perform the liveness detection operation, based on the fingerprint spectrum information and may determine whether a fingerprint being in contact with the sensing area SA of the touch panel 121 is a fingerprint of a real person based on the result of the liveness detection operation.
Referring to
In detail, the touch panel 121 and the display panel 131 may be disposed in such a way that the sensing area SA of the touch panel 121 is aligned with the sensing area SA′ of the display panel 131. The fingerprint sensing pixel array 111 and the display panel 131 may be disposed in such a way that the sensing area SA′ of the display panel 131 and the fingerprint sensing pixel array 111 are aligned.
Since the fingerprint sensing pixel array 111, the touch panel 121, and the display panel 131 are disposed as shown in
Referring to
The display panel 131 may include a plurality of display pixels DP. The plurality of display pixels DP may emit light under the control of the DDI 132. For example, the plurality of display pixels DP may be OLED pixels, but the scope and spirit of the present disclosure is not limited thereto.
First display pixels DP1 of the plurality of display pixels may indicate all or some of display pixels included in the sensing area SA. For example, the first display pixels DP1 may correspond to display pixels included in the sensing area SA. Alternatively, the first display pixels DP1 may be display pixels placed on/over the fingerprint sensing pixel array 111. Alternatively, the first display pixels DP1 may be display pixels included in an area of a touch panel, with which a fingerprint FP is in contact.
Second display pixels DP2 of the plurality of display pixels may not emit light. For example, the second display pixels DP2 may indicate those display pixels that are included in a non-sensing area NSA (e.g., areas of the touch panel 121 and the display panel 131 that are not included in the sensing area SA). Alternatively, the second display pixels DP2 may be display pixels included in an area of the touch panel 121 where a fingerprint FP is not in contact.
For example, when the user fingerprint FP is in contact with or approaches the sensing area SA of the touch panel 121, some display pixels (e.g., the first display pixels DP1) of the display pixels of the display panel 131 may emit light under the control of the DDI 132. For example, the DDI 132 may allow the first display pixels DP1 to emit light such that light emitted from the first display pixels DP1 is reflected by the fingerprint FP and the reflected light is provided to the fingerprint sensing pixel array 111 of the fingerprint sensor 110. In this case, the first display pixels DP1 may emit light, the amount of which is greater than a reference value set in advance by the user. Alternatively, the first display pixels DP1 may emit light maximally. Light emitted from the first display pixels DP1 may be reflected by the fingerprint FP, and the reflected light may be provided to the fingerprint sensing pixel array 111.
The fingerprint sensing pixel array 111 may include the plurality of fingerprint pixels FPX and the plurality of spectrum pixels SPX. The plurality of fingerprint pixels FPX and the plurality of spectrum pixels SPX may obtain fingerprint image information and fingerprint spectrum information, respectively, based on the light that is emitted from the first display pixels DP1 and reflected from the fingerprint FP. For example, the plurality of fingerprint pixels FPX may obtain the fingerprint image information based on the light reflected from the fingerprint FP, and the plurality of spectrum pixels SPX may obtain the fingerprint spectrum information based on the light reflected from the fingerprint FP.
In an example embodiment, the obtained fingerprint image information may be used for the fingerprint matching operation, and the obtained fingerprint spectrum information may be used for the liveness detection operation.
In an example embodiment, some pixels of the first display pixels DP1 may emit light more brightly than the remaining pixels. In this case, the location of these pixels may correspond to the location of the spectrum pixels SPX. That is, display pixels, which correspond to the spectrum pixels SPX, from among the first display pixels DP1 may emit light more brightly than the remaining display pixels.
Below, for convenience of description, the terms “wide band light” and “narrow band light” are used. The wide band light indicates light having a wavelength of a specific band which is reflected from a user fingerprint. The narrow band light indicates light having a specific wavelength. For example, the wide band light may indicate light having a wavelength included in a band from 500 nm to 650 nm and may be light filtered by a band limit filter. The narrow band light may be light having a wavelength of 550 nm±α (where α is a non-negative integer and a disclosed numerical value is an example) and may be light filtered by a nano-optical filter.
In an example embodiment, the wavelength of the narrow band light may be determined by an optical characteristic of the nano-optical filter. However, the terms are to describe the technical idea of the present disclosure easily, and the scope and spirit of the present disclosure is not limited thereto.
Referring to
Below, for convenience of description, an example embodiment of the present disclosure is described in reference to the first fingerprint pixel FPX1, but, the scope and spirit of the present disclosure is not limited thereto. The second fingerprint sensing pixel FPX2 may be the same with the first fingerprint sensing pixel FPX1.
The metal shield structure MTS may prevent the reflected light from being provided to the photodiode PD1. That is, the metal shield structure MTS may block the reflected light in the remaining portion other than a first opening L1 to allow only a vertical component of the reflected light to arrive at the photodiode PD1. The reason is that there is a need to block surrounding non-vertical light for the purpose of clearly recognizing a target of a small size, such as ridge and valley of a fingerprint, in object sensing such as fingerprint sensing.
In an example embodiment, a diameter (or length) of the first opening L1 may be shorter than a distance between the first opening L1 and another opening adjacent to the first opening L1. For example, an opening L1′ of the second fingerprint sensing pixel FPX2 may be adjacent to the first opening L1, and the diameter (or length) of the first opening L1 may be shorter than the distance between the first opening L1 and the opening L1′. In an example embodiment, metal material forming the metal shield structure MTS may be filled between the first opening L1 and the opening L1′. Thus, the diameter of the first opening L1 may be shorter than a length La of the metal material formed between the first opening L1 and the adjacent opening L1′.
In an example embodiment, the metal shield structure MTS may be partitioned into a plurality of sub shield structures. Each of the plurality of sub shield structures may be apart from each other by a length of the first opening L1. Each of the plurality of sub structures may be formed on the first and second finger print sensing pixels FPX1 and FPX2 (or, photodiodes PD1 and PD2 thereof). Alternatively, each of the plurality of sub structures may be partially overlapped with the first and second finger print sensing pixels FPX1 and FPX2 (or, photodiodes PD1 and PD2 thereof). For example, a first sub shield structure may be metal material filled between the first opening L1 and the opening L1′. The first sub shield structure may be overlapped partially with two photodiodes PD1 and PD2 as illustrated in
In an example embodiment, the metal shield structure MTS may include metal. The metal shield structure MTS may include, but is not limited to, at least one of tungsten (W), aluminum (Al), and copper (Cu).
The color filter structure CFS may include a plurality of color filters (e.g., red (R), green (G), and blue (B)) stacked on the metal shield structure MTS. For example, a blue filter “B” of the color filter structure CFS may be stacked on the metal shield structure MTS, a green filter “G” may be stacked on the blue filter “B”, and a red filter “R” may be stacked on the green filter “G”.
The filters “R,” “G,” and “B” of the color filter structure CFS may include second to fourth openings L2 to L4, respectively. Each of the second to fourth openings L2 to L4 may be configured to pass the light focused by the optical lens OL. In an example embodiment, diameters of the second to fourth openings L2 to L4 may be different from each other. In other words, as illustrated in
The color filters “R,” “G,” and “B” of the color filter structure CFS may be formed in the remaining area other than an area where the openings L2 to L4 are formed. That is, the color filter structure CFS may be formed to be connected in the fingerprint pixels FPX.
The color filter structure CFS may implement the black exterior of the fingerprint sensing pixel array 111. For example, the metal shield structure MTS may block light reflected from a fingerprint and may simultaneously reflect a portion of the reflected light. When the reflected light is exposed or reflected to the outside by the metal shield structure MTS, a location of an image sensor may be identified by the user. That is, an image sensor, the color of which varies to various colors depending on an angle of reflection on the outside, may be identified with the naked eye by the user.
To prevent this phenomenon, the black exterior of the color filter structure CFS may be implemented as the color filter structure CFS absorbs various wavelength bands of the reflected light. For example, the red filter “R” of the color filter structure CFS may pass only light, which has a wavelength corresponding to a red component, from among the reflected light and may block or absorb light of the remaining wavelengths. In contrast, the green filter “G” and the blue filter “B” of the color filter structure CFS may pass only light, which has wavelengths respectively corresponding to green and blue components, from among the reflected light and may block or absorb light of the remaining wavelengths. That is, since most of the wavelength range is absorbed by the color filter structure CFS where the red filter “R,” the green filter “G,” and the blue filter “B” are stacked, the fingerprint sensing pixel array 111 may appear black from the outside.
The color filter structure CFS may include, but is not limited to, an organic component such as photo resist (PR). In an example embodiment, a stack method, a stack order, or a configuration of color filters of the color filter structure CFS may be variously changed or modified.
The boundary isolation layer BR may be formed under the metal shield structure MTS. The boundary isolation layer BR may be configured to individually separate the plurality of fingerprint pixels FPX and the plurality of spectrum pixels SPX of the fingerprint sensing pixel array 111. That is, the plurality of fingerprint pixels FPX and the plurality of spectrum pixels SPX may be individually separated by the boundary isolation layer BR. In an example embodiment, the boundary isolation layer BR may be formed in a substrate or on the substrate.
The photodiode PD1 may be formed between the boundary isolation layer BR. The photodiode PD1 may be configured to output an electrical signal depending on the intensity of incident light. In an example embodiment, the structure of the boundary isolation layer BR and the photodiode PD1 is not limited to
In an example embodiment, an insulation structure ISS may be provided under the photodiode PD. The insulation structure ISS may include various interconnections or various elements connected with the fingerprint pixels FPX and the spectrum pixels SPX.
The spectrum pixels SPX may be formed on the same substrate as the fingerprint pixels FPX. The spectrum pixels SPX may be formed to be spaced apart from the fingerprint pixels FPX by a predetermined distance. In other words, a dummy area may exist between the spectrum pixels SPX and the fingerprint pixels FPX. The dummy area may include a structure that is partially similar to the fingerprint pixels FPX or may be filled in a specific structure.
In an example embodiment, a length of the dummy area may be longer than a distance between the first fingerprint sensing pixel FPX1 and the second fingerprint sensing pixel FPX2. For example, the first and second fingerprint sensing pixels FPX1 and FPX2 may be adjacent to each other. The first fingerprint sensing pixel FPX1 and the first spectrum pixel SPX1 may be apart from each other by a predetermined distance (i.e., a length of the dummy area). In other words, a distance between the first fingerprint sensing pixel FPX1 and the second finger print pixel FPX2 adjacent to the first fingerprint sensing pixel FPX1 may be shorter than a distance between the first fingerprint pixel FPX1 and the first spectrum pixel SPX1 adjacent to the first fingerprint pixel FPX1.
In an example embodiment, a distance between photodiodes of the fingerprint sensing pixels FPX may be shorter than a distance between a photodiode of the fingerprint sensing pixel FPX and a photodiode of the spectrum pixel adjacent to the fingerprint pixel SPX. For example, a first photodiode PD1 of the first fingerprint sensing pixel FPX1 may be apart from a second photodiode PD2 of the second fingerprint sensing pixel FPX2 adjacent to the first fingerprint sensing pixel FPX1 by a length of the boundary isolation layers BR, but, the first photodiode PD1 of the first fingerprint sensing pixel FPX1 may be apart from a third photodiode PD3 of the first spectrum pixel SPX1 adjacent to the first fingerprint sensing pixel FPX1 by a length of the dummy area. Here, the length of the boundary isolation layers BR may be shorter than the length of the dummy area.
The spectrum pixels SPX may include band limit filters BLF1 and BLF2, nano-optical filters NOF1 and NOF2, and photodiodes PD3 and PD4, respectively.
The band limit filters BLF1 and BLF2 may be formed in the same layer as the color filter structure CFS. For example, each of the band limit filters BLF1 and BLF2 may be an optical filter configured to pass a specific band of wide band light. In an example embodiment, each of the band limit filters BLF1 and BLF2 may be a color filter (e.g., a green filter) configured to pass wide band light of about 500 nm to 650 nm. However, the scope and spirit of the present disclosure is not limited thereto. Also, each of the band limit filters BLF1 and BLF2 of the spectrum pixels SPX may be an optical filter configured to pass wide band light of different wavelength bands.
Although not illustrated clearly in
The nano-optical filters NOF1 and NOF2 may be formed in the same layer as the metal shield structure MTS. A height of the nano-optical filters NOF1 and NOF2 may be the same with that of the metal shield structure MTS. The nano-optical filters NOF1 and NOF2 may have an optical characteristic of passing narrow band light having a specific wavelength. For example, the nano-optical filters NOF1 and NOF2 may be implemented in the form of a hole or a disc. For example, in the case where the nano-optical filters NOF1 and NOF2 are formed in a hole shape, the nano-optical filters NOF1 and NOF2 may pass only first narrow band light having a first wavelength. In the case where the nano-optical filters NOF1 and NOF2 are formed in a disc shape, the nano-optical filter NOF1 and NOF2 may pass the remaining light other than the first narrow band light having the first wavelength.
For example, the nano-optical filters NOF1 and NOF2 may be formed to have different wavelength characteristics. For example, in the case where the first nano-optical filter NOF1 includes a hole having a first diameter, the first nano-optical filter NOF1 may pass the first narrow band light of the first wavelength. In contrast, in the case where the second nano-optical filter NOF2 includes a hole having a second diameter, the second nano-optical filter NOF2 may pass second narrow band light of a second wavelength. Each of the photodiodes PD of the spectrum pixels SPX may generate an electrical signal corresponding to the intensity of narrow band light filtered by a nano-optical filter NOF.
In an example embodiment, the photodiodes PD3 and PD4 of the spectrum pixels SPX may be have the same structure with the photodiodes PD1 and PD2 of the fingerprint sensing pixels FPX. For example, a height of the photodiodes PD3 and PD4 of the spectrum pixels SPX may be the same with that of the photodiodes PD1 and PD2 of the fingerprint sensing pixels FPX. Alternatively, a length of the photodiodes PD3 and PD4 of the spectrum pixels SPX may be the same with that of the photodiodes PD1 and PD2 of the fingerprint sensing pixels FPX.
In an example embodiment, unlike the fingerprint pixels FPX, the spectrum pixels SPX may not include the optical lens OL. Also, unlike the fingerprint pixels FPX, the spectrum pixels SPX may not include a separate opening. The reason is that there is a need to receive a lot of light since the spectrum pixels SPX detect only narrow band light of a specific wavelength, compared with the fingerprint pixels FPX. That is, the spectrum pixels SPX may receive the whole incident light through the area of the spectrum pixels SPX formed on a substrate, by excluding a separate light focusing element or an element for blocking light of a specific range.
As described above, the intensity of narrow band light of a specific wavelength or the remaining light other than the narrow band light of the specific wavelength may be detected by using the nano-optical filter NOF. According to the above description, information about the intensity of the narrow band light may be obtained by the spectrum pixels SPX.
Referring to
A first layer LAY1 may be formed on/over the substrate SUB. For example, the first layer LAY1 may include the metal shield structure MTS and a nano-optical filter structure NOFS. As described with reference to
A second layer LAY2 may be formed on/over the first layer LAY1. For example, the second layer LAY2 may include the color filter structure CFS and a band limit filter structure BLFS. The color filter structure CFS may be formed over the fingerprint pixels FPX, and the band limit filter structure BLFS may be formed over the spectrum pixels SPX. The band limit filter structure BLFS may include the band limit filters BLF1 and BLF2 described with reference to
As described above, the fingerprint pixels FPX and the spectrum pixels SPX may be formed on the same substrate, and the metal shield structure MTS of the fingerprint pixels FPX and the nano-optical filter structure NOFS of the spectrum pixels SPX may be in the same layer (i.e., the first layer LAY1). Also, the color filter structure CFS of the fingerprint pixels FPX and the band limit filter structure BLFS of the spectrum pixels SPX may be formed in the same layer (i.e., the second layer LAY2). However, the scope and spirit of the present disclosure may not be limited thereto. For example, the arrangement or locations of respective structures may be variously changed or modified.
Referring to
For example, the first and second nano-optical filters NOF1 and NOF2 may be implemented in the form of a hole. The first nano-optical filter NOF1 may include a plurality of holes, each of which has a first diameter D1. The second nano-optical filter NOF2 may include a plurality of holes, each of which has a second diameter D2 greater than the first diameter D1.
The first and second nano-optical filters NOF1 and NOF2 may have different optical characteristics depending on hole diameters. For example, as illustrated in
As described above, the plurality of spectrum pixels SPX may include the nano-optical filters NOF including a plurality of holes having different diameters and may be configured to detect light of different wavelengths depending on the optical characteristic of the nano-optical filters NOF.
In detail, a part of the plurality of spectrum pixels SPX may respectively include first nano-optical filters NOF1 each including a plurality of holes having a first diameter, another part of the plurality of spectrum pixels SPX may respectively include second nano-optical filters NOF2 each including a plurality of holes having a second diameter, and another part of the plurality of spectrum pixels SPX may respectively include third nano-optical filters NOF3 each including a plurality of holes having a third diameter.
The plurality of nano-optical filters NOF1 to NOFn may respectively have different optical characteristics depending on hole diameters. For example, as illustrated in
As described above, since the spectrum pixels SPX include nano-optical filters NOF1 to NOFn, respectively, spectrum information (i.e., fingerprint spectrum information) of the 0-th wavelength band W0 may be obtained based on information detected from the plurality of spectrum pixels SPX.
Referring to
As illustrated in
In other words, whether incident light is reflected from a user's finger, is reflected from a glove, is reflected from a printed photo may be determined by analyzing a spectrum characteristic of light in the 0-th wavelength band W0 described above.
For example, as described above, fingerprint spectrum information of the 0-th wavelength band W0 of light reflected from a user fingerprint may be obtained by the spectrum pixels SPX according to the present disclosure. Whether a touched fingerprint FP comes from a finger of a real person or is forged may be determined by determining whether the obtained fingerprint spectrum information has the spectrum characteristic of the first curve L01 illustrated in
For example, the 0-th wavelength band W0 may be a wavelength band of wide band light passing through the band limit filters BLF1 and BLF2.
The first and second nano-optical filters NOF1 and NOF2 may have different optical characteristics depending on disc diameters. For example, as illustrated in
As described above, the plurality of spectrum pixels SPX may include the nano-optical filters NOF including a plurality of discs having different diameters and may be configured to detect light of different wavelengths depending on the optical characteristics of the nano-optical filters NOF.
For example, a description is given with reference to
The nano-optical filters NOF1′ to NOFn′ are similar to nano-optical filters of a hole form described above except that the nano-optical filters NOF1′ to NOFn′ block only narrow band light of a specific wavelength, and thus, a detailed description thereof will not be repeated here.
Referring to
Referring to
Referring to
Referring to
The fingerprint pixels FPX and the spectrum pixels SPX described above are exemplary, and the scope and spirit of the present disclosure is not limited thereto. The structures and forms of the fingerprint pixels FPX and the spectrum pixels SPX may be variously changed or modified. Also, the order of stacking or manufacturing various components included in the fingerprint pixels FPX and the spectrum pixels SPX may be variously changed or modified without departing from the technical idea of the present disclosure.
Also, for brevity of illustration, example arrangements of the fingerprint pixels FPX and the spectrum pixels SPX are illustrated in
Also, each component illustrated in
Also, a dummy area (or dummy pixels) may exist between the fingerprint pixels FPX and the spectrum pixels SPX. The arrangements of pixels of the fingerprint sensing pixel array 111 illustrated in
Referring to
Referring to
Referring to
Referring to
Referring to
Referring to
As described above, according to an embodiment of the present disclosure, the fingerprint sensor 110 placed on a lower side of the display panel 131 may include a plurality of fingerprint pixels FPX and a plurality of spectrum pixels SPX. The plurality of fingerprint pixels FPX may obtain fingerprint image information with respect to a fingerprint touched on the sensing areas SA and SA′ of the touch panel 121 and the display panel 131, and the plurality of spectrum pixels SPX may obtain fingerprint spectrum information with respect to the touched fingerprint. The fingerprint image information may be used for the fingerprint matching operation, and the fingerprint spectrum information may be used for the liveness detection operation. Accordingly, by detecting fingerprint forgery through the liveness detection operation, in addition to user authentication through the fingerprint matching operation, the security and reliability of an electronic device or a fingerprint sensor may be improved.
Referring to
The AP 201 may be configured to control the TDI, the DDI, and the sensor driver 212. For example, the AP 201 may perform the corresponding operation (e.g., device unlock, financial payment, or the like) based on an authentication result AR from the sensor driver 212.
The fingerprint sensing pixel array 211 may include the plurality of fingerprint pixels FPX and the plurality of spectrum pixels SPX. The plurality of fingerprint pixels FPX and the plurality of spectrum pixels SPX are described with reference to
As described above, the plurality of fingerprint pixels FPX may obtain fingerprint image information FII and may provide the obtained fingerprint image information FII to the sensor driver 212. As described above, the plurality of spectrum pixels SPX may obtain fingerprint spectrum information FSI and may provide the obtained fingerprint spectrum information FSI to the sensor driver 212.
The sensor driver 212 may include a fingerprint authentication unit FAU and a liveness authentication unit LAU. The fingerprint authentication unit FAU may perform the fingerprint matching operation, based on the fingerprint image information FII from the plurality of fingerprint pixels FPX. For example, the fingerprint authentication unit FAU may perform the fingerprint matching operation, by comparing the fingerprint image information FII from the plurality of fingerprint pixels FPX and preset (e.g., preregistered) fingerprint image information. In an example embodiment, the preset fingerprint image information may be information that is registered in advance by a user through a separate registration process.
The liveness authentication unit LAU may perform a liveness detection operation, based on the fingerprint spectrum information FSI from the plurality of spectrum pixels SPX. For example, as described above, the fingerprint spectrum information FSI from the plurality of spectrum pixels SPX may include the fingerprint spectrum information FSI of a specific band. The liveness authentication unit LAU may perform the liveness detection operation, by determining whether the fingerprint spectrum information FSI of the specific band included in the fingerprint spectrum information FSI is matched with a specific pattern.
In an example embodiment, whether a fingerprint is a fingerprint of a real person or a fake fingerprint (e.g., an image of a fingerprint on a printed photo, an imprint of a fingerprint on moldable clay) may be determined through the liveness detection operation. A configuration associated with the determination operation is described with reference to
As described above, the sensor driver 212 may transmit the authentication result AR to the AP 201 depending on the operation results of the fingerprint authentication unit FAU and the liveness authentication unit LAU. That is, the sensor driver 212 may automatically perform the fingerprint matching operation and the liveness detection operation. In the case where authentication is successfully made in both the fingerprint authentication unit FAU and the liveness authentication unit LAU, the authentication result AR may include information about authentication success. In contrast, in the case where authentication fails in any one or all of the fingerprint authentication unit FAU and the liveness authentication unit LAU, the authentication result AR may include information about authentication fail.
Unlike the example embodiment of
Unlike the example embodiments of
The authentication module 440 may receive the fingerprint image information FII and the fingerprint spectrum information FSI from the sensor driver 412 and may perform the fingerprint matching operation and the liveness detection operation, based on the received information FII and FSI. For example, the authentication module 440 may include the fingerprint authentication unit FAU and the liveness authentication unit LAU, which perform the fingerprint matching operation and the liveness detection operation based on the fingerprint image information FII and the fingerprint spectrum information FSI, respectively.
As described above, a sensor driver, an AP, or a separate authentication module may determine whether a fingerprint is faked and whether a fingerprint is matched, by performing a fingerprint matching operation based on the fingerprint image information FII and performing the liveness detection operation based on the fingerprint spectrum information FSI. Accordingly, an electronic device with improved security and reliability may be provided.
Referring to
However, the scope and spirit of the present disclosure may not be limited thereto. For example, the touch panel 121 may sense a touch of another area in addition to the sensing area SA. In this case, the touch panel 121 may perform a touch scan operation on the whole area in a power saving mode (e.g., an operation mode where a touch scan frequency is slow).
In operation S110, the electronic device 100 may determine whether the area (or size) of the sensed first touch is greater than a first threshold value TH1. If the area of the sensed first touch is not greater than the first threshold value TH1, it may indicate that a touch has been sensed regardless of the intention of the user. In the case where the area of the sensed first touch is less than the first threshold value TH1, the electronic device may return to operation S105.
In contrast, if the area of the sensed first touch is greater than the first threshold value TH1, the electronic device 100 may be controlled according to the intention of the user. For example, the user may touch the electronic device 100 for the purpose of unlocking the electronic device 100 of a lock state.
In the case where the area of the sensed first touch is greater than the first threshold value TH1, in operation S115, the electronic device may display a sensing area. For example, the AP 101 of the electronic device 100 may control the DDI 132 such that the sensing area SA or SA′ is displayed (e.g., indicated by relevant pixels). The DDI 132 may control the display panel 131 such that display pixels included in the sensing area SA′ display the sensing area SA′.
For example, operation S105 and operation S110 may be omitted, and operation S115 may be performed by a user's request of another manner.
In operation S120, the electronic device 100 may detect a second touch of the user. For example, for fingerprint recognition, the user may touch the displayed sensing area SA′ with his/her finger. In this case, the touch panel 121 may sense the second touch of the user being in contact with the sensing area SA′. For example, the first touch of operation S105 and the second touch of operation S120 may be touches made by separate operations or may be touches made by the same operation.
In operation S125, the electronic device 100 may determine whether the sensed second touch is greater than a second threshold value TH2. If the sensed second touch is not greater than the second threshold value TH2, it may indicate that the contact area is insufficient to obtain the user fingerprint image information FII and the fingerprint spectrum information FSI. In the case where the sensed second touch is not greater than the second threshold value TH2, the electronic device may return to operation S120.
If the sensed second touch is greater than the second threshold value TH2, it may indicate that the contact area is sufficient to obtain the user fingerprint image information FII and the fingerprint spectrum information FSI. In this case, in operation S130, the electronic device 100 may allow light to be emitted in the sensing area SA′. For example, as described with reference to
For example, as described above, the first display pixels DP1 emitting light may be display pixels included in the sensing area SA′ or may be display pixels included in an area corresponding to a contact surface of the user touch.
In operation S135, the electronic device 100 may detect the fingerprint image information FII and the fingerprint spectrum information FSI. For example, the fingerprint sensor 110 of the electronic device 100 may detect the fingerprint image information FII and the fingerprint spectrum information FSI, based on light reflected from a fingerprint. A way to detect the fingerprint image information FII and the fingerprint spectrum information FSI is described above, and thus, a detailed description thereof will not be repeated here.
In operation S140, the electronic device 100 may perform an authentication operation, based on the fingerprint image information FII and the fingerprint spectrum information FSI. For example, the electronic device 100 may perform the fingerprint image information based on the fingerprint image information FII and may perform the liveness detection operation based on the fingerprint spectrum information FSI. The fingerprint matching operation and the liveness detection operation are described above, and thus, a detailed description thereof will not be repeated here.
In the case where both the fingerprint matching operation and the liveness detection operation are successful, in operation S145, the electronic device 100 may be unlocked or may perform a relevant operation.
In the case where either or both of the fingerprint matching operation and the liveness detection operation fail, in operation S150, the electronic device 100 may alert authentication failure. For example, the electronic device 100 may display information about the authentication failure through the display panel 131. Alternatively, the electronic device 100 may notify the user of the authentication failure through another separate notification manner (e.g., sound, vibration, or the like). For example, after the authentication failure, the electronic device 100 may perform a re-authentication operation depending on a request of the user. The re-authentication operation may be performed through operation S115 to operation S150.
Referring to
In operation S210, the AP 101 may transmit a request for displaying the sensing area SA to the DDI 132 in response to the information about the sensed first touch. For example, the AP 101 may perform an operation of comparing the area of the first touch (operation S110 of
In operation S215, the DDI 132 may display the sensing area SA′ in response to the request of the AP 101. For example, the DDI 132 may control the display panel 131 to allow at least a part of display pixels included in the sensing area SA′ to emit light.
Afterwards, in operation S220, the TDI 122 may sense a second touch of the user and may provide information about the sensed second touch to the AP 101.
In operation S225, the AP 101 may transmit a request for light-emitting to the DDI 132 in response to the information about the sensed second touch. For example, the request for light-emitting may be a request that allows the first display pixels DP1 included in the sensing area SA′ emit light of predetermined intensity or greater.
For example, as in the above description, the AP 101 may perform an operation of comparing the area of the second touch (operation S125 of
In operation S230, the DDI 132 may control the display panel 131 depending on the request of the AP 101, to allow the display pixels DP included in the sensing area SA′ to emit light.
In operation S235, the sensor driver 112 may detect the fingerprint image information FII and the fingerprint spectrum information FSI of a fingerprint of the user touched on the touch panel 121.
In operation S240, the sensor driver 112 may transmit the fingerprint image information FII and the fingerprint spectrum information FSI to the AP 101.
In operation S245, the AP 101 may perform an authentication operation (i.e., the fingerprint matching operation and the liveness detection operation), based on the fingerprint image information FII and the fingerprint spectrum information FSI.
The operation method described with reference to
Unlike the example embodiment of
For example, the TDI 522 may sense a touch of the user and may provide information about the sensed touch to the DDI 532 without passing through the AP 501. The DDI 532 may allow display pixels included in the sensing area SA′ to emit light in response to the received information. The DDI 532 may provide the sensor driver 512 with information indicating that the display pixels emit light, without passing through the AP 501. In response to the received information, the sensor driver 512 may obtain the fingerprint image information FII and the liveness detection operation FSI and may perform the fingerprint matching operation and the liveness detection operation, based on the fingerprint image information FII and the fingerprint spectrum information FSI.
As described above, the sensor driver 512, the TDI 522, and the DDI 532 may communicate with each other without intervention of the AP 501. In this case, since the fingerprint matching operation and the liveness detection operation are performed without intervention of the AP 501, even though the AP 501 does not wake from a sleep mode, a normal authentication operation may be performed. Accordingly, a low-power authentication operation may be performed.
In operation S320, the DDI 532 may display the sensing area SA′ in response to the received information. For example, under the control of the DDI 532, some pixels of the sensing area SA′ may emit light such that the user perceives an area where his/her fingerprint will be touched. That is, when some pixels emit light, an area where the user's fingerprint will be touched may be displayed.
In operation S330, the TDI 320 may sense a second touch of the user and may transmit information about the sensed second touch to the DDI 532.
In operation S340, the DDI 532 may allow some pixels of the sensing area SA′ to emit light. In this case, some pixels of the sensing area SA′ may emit light of preset intensity or greater. Alternatively, some pixels of the sensing area SA′ may emit light with maximum intensity.
In operation S350, the DDI 532 may provide the sensor driver 512 with information indicating that some pixels emit light.
In operation S360, the sensor driver 512 may obtain the fingerprint image information FII and the fingerprint spectrum information FSI in response to the received information. In operation S370, the sensor driver 512 may perform an authentication operation (i.e., the fingerprint matching operation and the liveness detection operation). Operation S360 and operation S370 are described above, and thus, a detailed description thereof will not be repeated here.
In operation S380, the sensor driver 512 may transmit the authentication result to the AP 501. For example, the AP 501 may perform an operation (e.g., unlock, sign-in, authentication failure notification, or the like) corresponding to the received authentication result.
As described above, components of the electronic device 500 according to an example embodiment may perform the fingerprint matching operation and the liveness detection operation without intervention of the AP 501. Since an operation of the AP 501 is not required for the authentication operation, a fingerprint sensor may be driven with relatively low power.
Referring to
Afterwards, an electronic device 1000_2 may allow light to be emitted from a sensing area SA_2 or a part of the sensing area SA_2. This may be an indication for inducing a touch of the user.
Afterwards, an electronic device 1000_3 may sense a user touch on a sensing area SA_3.
Afterwards, an electronic device 1000_4 may allow light to be emitted from a sensing area SA_4, for the purpose of obtaining the fingerprint image information FII and the fingerprint spectrum information FSI from the user touch. In this case, the area from which light is emitted may be the whole sensing area SA_4 or may be an area (i.e., a user touch area), in which the user touch is sensed, of the sensing area SA_4.
Afterwards, in the case where a fingerprint matching operation and a liveness detection operation based on the obtained fingerprint image information FII and the obtained fingerprint spectrum information FSI are successful, an electronic device 1000_5 may be unlocked or may display information indicating that the user authentication is completed.
Alternatively, in the case where any one or all of the fingerprint matching operation and the liveness detection operation based on the obtained fingerprint image information FII and the obtained fingerprint spectrum information FSI fail, an electronic device 1000_6 may display information about authentication failure. For example, the above-described authentication manner may be used in various different operations requiring device unlock, financial payment, or authentication.
In an example embodiment, an AP included in an electronic device may be configured to control an overall flow of the above-described authentication operation. Alternatively, components included in the electronic device may be configured to perform the above-described authentication operation without intervention of the AP.
The touch sensor panel 2100, the touch processor 2102, the display panel 2200, the display driver 2202, the image sensor 2300, and the main processor 2900 may respectively correspond to a touch panel, a TDI, a display panel, a DDI, a fingerprint sensor, and an AP described with reference to
The image sensor 2300 may be disposed under the touch display panel 2100 and/or the display panel 2200 to spatially correspond to a specific area on the touch display panel 2100 and/or the display panel 2200. The electronic device 2000 may provide an optics-based fingerprint detection function by using the image sensor 2300. For example, the image sensor 2300 may include the plurality of fingerprint pixels FPX and the plurality of spectrum pixels SPX and may obtain the fingerprint image information FII and the fingerprint spectrum information FSI.
The image sensor 2300 may share an area on the electronic device 2000 with the touch display panel 2100 and/or the display panel 2200. The image sensor 2300 may not need an additional area on the electronic device 2000. Accordingly, it may be possible to reduce the size of the electronic device 2000, or a spare area may be used for any other purpose. In addition, a configuration and an operation for performing a fingerprint detection function may be simplified.
The buffer memory 2400 may store data to be used for an operation of the electronic device 2000. For example, the buffer memory 2400 may temporarily store data that are processed or to be processed by the main processor 2900. For example, the buffer memory 2400 may include a volatile memory such as a static random access memory (SRAM), a dynamic RAM (DRAM), or a synchronous DRAM (SDRAM), and/or a nonvolatile memory such as a phase-change RAM (PRAM), a magneto-resistive RAM (MRAM), a resistive RAM (ReRAM), or a ferroelectric RAM (FRAM).
The nonvolatile memory 2500 may store data regardless of power supply. For example, the nonvolatile memory 2500 may include at least one of various nonvolatile memories such as a flash memory, a PRAM, a MRAM, a ReRAM, and a FRAM. For example, the nonvolatile memory 2500 may include an embedded memory and/or a removable memory of the electronic device 2000.
The image processor 2600 may receive light through a lens 2610. An image sensor 2620 and an image signal processor 2630 included in the image processor 2600 may generate image information about an external object, based on the received light.
The communication block 2700 may exchange signals with an external device/system through an antenna 2710. A transceiver 2720 and a modulator/demodulator (modem) 2730 of the communication block 2700 may process signals exchanged with the external device/system, based on at least one of various wireless communication protocols: long-term evolution (LTE), worldwide interoperability for microwave access (WiMax), Global System for Mobile Communications (GSM), code-division multiple access (CDMA), Bluetooth, near-field communication (NFC), Wi-Fi, and radio-frequency identification (RFID).
The audio processor 2800 may process an audio signal by using an audio signal processor 2810. The audio processor 2800 may receive an audio input through a microphone 2820 or may provide an audio output through a speaker 2830.
The main processor 2900 may control overall operations of the electronic device 2000. The main processor 2900 may control/manage operations of elements of the electronic device 2000. The main processor 2900 may process various operations for operating the electronic device 2000.
As described above, an electronic device according to an example embodiment of the present disclosure may obtain fingerprint image information from a user fingerprint to perform a fingerprint matching operation and may obtain fingerprint spectrum information from the user fingerprint to perform a liveness detection operation. By determining whether a user fingerprint is matched and whether the fingerprint is faked are determined, security and reliability of the electronic device may be improved.
According to the present disclosure, an electronic device and a fingerprint recognition device may perform a user fingerprint matching operation and may determine whether a user fingerprint is faked. Accordingly, an optics-based fingerprint sensor having improved reliability and security, an electronic device including the optics-based fingerprint sensor, and an operation method of the electronic device are provided.
While the present disclosure has been described with reference to example embodiments thereof, it will be apparent to those of ordinary skill in the art that various changes and modifications may be made thereto without departing from the spirit and scope of the present disclosure as set forth in the following claims.
| Number | Date | Country | Kind |
|---|---|---|---|
| 10-2017-0089143 | Jul 2017 | KR | national |
| 10-2018-0001861 | Jan 2018 | KR | national |
| 10-2018-0008187 | Jan 2018 | KR | national |
