Patent Application
20250134462
What is disclosed herein relates to a detection device and a wearable device.
United States Patent Application Publication No. 2018/0012069 describes an optical sensor in which a plurality of photoelectric conversion elements such as photodiodes are arranged on a semiconductor substrate. With the optical sensor, signals output from the photoelectric conversion elements change with an amount of received light, thereby enabling detection of biometric information.
Japanese Patent Application Laid-open Publication No. 2019-180861 describes a configuration to acquire an oxygen saturation level in blood (hereinafter, called “blood oxygen saturation level” (SpO2)) using pulse waves acquired using infrared light and pulse waves acquired using red light. The blood oxygen saturation level (SpO2) refers to a ratio of an amount of oxygen actually bound to hemoglobin to the total amount of oxygen under the assumption that the oxygen is bound to all the hemoglobin in the blood.
For example, when acquiring subcutaneous information such as the pulse waves and blood flow, appropriate biometric information may not be obtained due to superimposition of body motion noise caused by human movement or the like, biosignals not intended to be detected, or noise components at an alternating-current frequency of a commercial power supply (for example, 50 Hz or 60 Hz).
For the foregoing reasons, there is a need for a detection device and a wearable device that are capable of acquiring desired biometric information.
According to an aspect, a detection device includes: a plurality of optical sensors arranged in a detection area; a light source configured to emit light to the optical sensors; an analog front-end (AFE) circuit configured to acquire a detection value of each of the optical sensors; and a signal processing circuit configured to acquire predefined biometric information based on first time-domain data obtained by acquiring the detection values in chronological order. The signal processing circuit is configured to: convert the first time-domain data into a time-domain matrix and perform singular value decomposition on the time-domain matrix, and inversely calculate second time-domain data based on a predetermined singular value among a plurality of singular values obtained as a result of the singular value decomposition; and acquire the biometric information that changes in chronological order as image information using the second time-domain data.
According to an aspect, a wearable device includes the detection device, and the wearable device has a ring shape wearable on a human body.
The following describes a mode (embodiment) for carrying out the present disclosure in detail with reference to the drawings. The present disclosure is not limited to the description of the embodiment given below. Components described below include those easily conceivable by those skilled in the art or those substantially identical thereto. In addition, the components described below can be combined as appropriate. What is disclosed herein is merely an example, and the present disclosure naturally encompasses appropriate modifications easily conceivable by those skilled in the art while maintaining the gist of the disclosure. To further clarify the description, the drawings may schematically illustrate, for example, widths, thicknesses, and shapes of various parts as compared with actual aspects thereof. However, they are merely examples, and interpretation of the present disclosure is not limited thereto. The same component as that described with reference to an already mentioned drawing is denoted by the same reference numeral through the present specification and the drawings, and detailed description thereof may not be repeated where appropriate.
The detection device 1 is electrically coupled to a host. The host is, for example, a higher-level control device for an apparatus (not illustrated) to which the detection device 1 is applied. The detection device 1 according to the first embodiment transmits acquired biometric information to the host via an output circuit 126.
The sensor base member 21 is electrically coupled to a control substrate 121 via a flexible printed circuit board 71. The flexible printed circuit board 71 is provided with the AFE circuit 48. The control substrate 121 is provided with the control circuit 122, the power supply circuit 123, and the output circuit 126. As illustrated in
The control circuit 122 is, for example, a control integrated circuit (IC) that outputs logic control signals. The control circuit 122 may be, for example, a programmable logic device (PLD) such as a field-programmable gate array (FPGA).
The control circuit 122 supplies control signals to the sensor area 10, the gate line drive circuit 15, and the signal line selection circuit 16 to control detection operations in the sensor area 10. The control circuit 122 also supplies control signals to the first and the second light sources 61 and 62 to control lighting and non-lighting of the first and the second light sources 61 and 62.
The power supply circuit 123 supplies voltage signals such as a sensor power supply potential VDDSNS (refer to
The output circuit 126 is, for example, a Universal Serial Bus (USB) controller IC, and controls communication between the control circuit 122 and the host.
The sensor base member 21 has a detection area AA and a peripheral area GA. The detection area AA is an area where a plurality of optical sensors PD (refer to
The gate line drive circuit 15 and the signal line selection circuit 16 are provided in the peripheral area GA. Specifically, the gate line drive circuit 15 is provided in an area extending along the second direction Dy in the peripheral area GA. The signal line selection circuit 16 is provided in an area extending along a first direction Dx in the peripheral area GA, and is provided between the sensor area 10 and the AFE circuit 48.
The first direction Dx is one direction in a plane parallel to the sensor base member 21. The second direction Dy is one direction in the plane parallel to the sensor base member 21 and is a direction orthogonal to the first direction Dx. The second direction Dy may non-orthogonally intersect the first direction Dx. A third direction Dz is a direction orthogonal to the first direction Dx and the second direction Dy and is a direction normal to the sensor base member 21.
The first light sources 61 are provided on the first light source base member 51 and arranged along the second direction Dy. The second light sources 62 are provided on the second light source base member 52, and arranged along the second direction Dy. The first light source base member 51 and the second light source base member 52 are electrically coupled to the control circuit 122 and the power supply circuit 123, through terminals 124 and 125, respectively, provided on the control substrate 121.
For example, inorganic light-emitting diodes (LEDs) or organic electroluminescent (EL) diodes (organic light-emitting diodes (OLEDs)) are used as the first and the second light sources 61 and 62. The first and the second light sources 61 and 62 emit first and second light, respectively, having different wavelengths.
The first light emitted from the first light sources 61 is reflected, for example, on a surface of an object to be detected, such as a finger or a wrist of a subject of examination, and is incident on the sensor area 10. As a result, the sensor area 10 can detect a fingerprint by detecting a shape of asperities on the surface of a finger Fg or the like. The second light emitted from the second light sources 62 is, for example, reflected in the finger Fg or the like, or transmitted through the finger Fg or the like, and is incident on the sensor area 10. As a result, the sensor area 10 can detect information on a living body in the finger, the wrist, or the like of the subject of examination. Examples of the information on the living body include, but are not limited to, pulse waves, pulsation, and a vascular image of the subject of examination. That is, the detection device 1 may be configured as a fingerprint detection device that detects a fingerprint or a vein detection device that detects a vascular pattern of, for example, veins.
The first light may have a wavelength of 420 nm to 600 nm, for example, approximately 500 nm, and the second light may have a wavelength of 780 nm to 950 nm, for example, approximately 850 nm. In this case, the first light is blue or green visible light (blue light or green light), and the second light is infrared light. The sensor area 10 can detect a fingerprint based on the first light emitted from the first light sources 61. The second light emitted from the second light sources 62 is reflected in, or transmitted through or absorbed by the object to be detected, and is incident on the sensor area 10. As a result, the sensor area 10 can detect biometric data such as the pulse waves and the vascular image (vascular pattern) as the information on the living body in the finger, the wrist, or the like of the subject of examination.
Alternatively, the first light may have a wavelength of 600 nm to 700 nm, for example, approximately 660 nm, and the second light may have a wavelength of 780 nm to 950 nm, for example, approximately 850 nm. In this case, the sensor area 10 can detect a blood oxygen level in addition to the pulse waves, the pulsation, and the vascular image as the information on the living body based on the first light emitted from the first light sources 61 and the second light emitted from the second light sources 62. In this way, the detection device 1 includes the first and the second light sources 61 and 62, and performs the detection based on the first light and the detection based on the second light, and thereby can detect the various types of information on the living body. The emission colors of the first and the second light sources 61 and 62 described above are examples, and the present disclosure is not limited by the emission colors of the first and the second light sources 61 and 62.
The sensor area 10 includes the optical sensors PD. Each of the optical sensors PD included in the sensor area 10 is an organic photodiode (OPD), and outputs an electrical signal corresponding to light emitted thereto as a detection signal Vdet to the signal line selection circuit 16. The sensor area 10 performs detection in response to a gate drive signal Vgcl supplied from the gate line drive circuit 15.
The detection control circuit 11 is a circuit that supplies respective control signals to the gate line drive circuit 15, the signal line selection circuit 16, and the detection circuit 40 to control operations of these circuits. The detection control circuit 11 supplies various control signals such as a start signal STV, a clock signal CK, and a reset signal RST1 to the gate line drive circuit 15. The detection control circuit 11 also supplies various control signals such as a selection signal ASW to the signal line selection circuit 16. The detection control circuit 11 also supplies various control signals to the first and the second light sources 61 and 62 to control the lighting and the non-lighting of each group of the first and the second light sources 61 and 62.
The gate line drive circuit 15 is a circuit that drives a plurality of gate lines GCL (refer to
The signal line selection circuit 16 is a switch circuit that sequentially or simultaneously selects a plurality of signal lines SGL (refer to
The detection circuit 40 includes the AFE circuit 48, a signal processing circuit 44, a storage circuit 46, and a detection timing control circuit 47. The detection timing control circuit 47 controls the AFE circuit 48 and the signal processing circuit 44 such that they operate in synchronization with each other, based on the control signal supplied from the detection control circuit 11.
The AFE circuit 48 detects the detection signals of the optical sensors PD output from the sensor area 10 in chronological order. The AFE circuit 48 is an analog front-end IC, for example.
The AFE circuit 48 is a signal processing circuit having functions of at least a detection signal amplifying circuit 42 and an analog-to-digital (A/D) conversion circuit 43. The detection signal amplifying circuit 42 amplifies the detection signals Vdet. The A/D conversion circuit 43 converts the analog signals output from the detection signal amplifying circuit 42 into digital signals at a predetermined sampling interval.
In the present disclosure, the control circuit 122 includes the signal processing circuit 44 and the storage circuit 46.
The signal processing circuit 44 acquires the biometric data for generating the information on the living body based on the detection values of the optical sensors PD output from the AFE circuit 48. In the present disclosure, the information on the living body includes the pulse waves acquired using the infrared light and/or red light.
The storage circuit 46 temporarily stores therein the signals processed by the signal processing circuit 44. In the present disclosure, the storage circuit 46 stores therein a biometric data acquisition area that is set in a biometric data acquisition area setting process flow (to be described later) when the signal processing circuit 44 acquires the biometric data, and also stores therein various types of setting information. The storage circuit 46 may have a configuration including a random-access memory (RAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), and the like. The storage circuit 46 may be a register circuit or the like.
The following describes a circuit configuration example of the detection device 1.
The gate lines GCL extend in the first direction Dx and are each coupled to the partial detection areas PAA arranged in the first direction Dx. A plurality of gate lines GCL(1), GCL(2), . . . , GCL(8) are arranged in the second direction Dy and are each coupled to the gate line drive circuit 15. In the following description, the gate lines GCL(1), GCL(2), . . . , GCL(8) will each be simply referred to as the gate line GCL when they need not be distinguished from one another. To facilitate understanding of the description,
The signal lines SGL extend in the second direction Dy and are each coupled to the optical sensors PD of the partial detection areas PAA arranged in the second direction Dy. A plurality of signal lines SGL(1), SGL(2), . . . , SGL(12) are arranged in the first direction Dx and are each coupled to the signal line selection circuit 16 and a reset circuit 17. In the following description, the signal lines SGL(1), SGL(2), . . . , SGL(12) will each be simply referred to as the signal line SGL when they need not be distinguished from one another.
To facilitate understanding of the description, 12 signal lines SGL are illustrated. However, this is merely an example, and N signal lines SGL may be arranged (where N is a natural number, such as 252). In
The gate line drive circuit 15 receives the various control signals such as the start signal STV, the clock signal CK, and the reset signal RST1 from the control circuit 122 (refer to
The gate line drive circuit 15 may perform different driving for each of detection modes including the detection of the fingerprint and the detection of a plurality of different items of information on a living body (including, for example, the pulse waves, the pulsation, the vascular image, and the blood oxygen level, which are hereinafter called also simply “biometric information”). For example, the gate line drive circuit 15 may collectively drive more than one of the gate lines GCL.
Specifically, the gate line drive circuit 15 simultaneously selects a predetermined number of the gate lines GCL from among the gate lines GCL(1), GCL(2), . . . , GCL(8) based on the control signals. For example, the gate line drive circuit 15 simultaneously selects six gate lines GCL(1) to GCL(6) and supplies thereto the gate drive signals Vgcl. The gate line drive circuit 15 supplies the gate drive signals Vgcl via the selected six gate lines GCL to the first switching elements Tr. This operation selects block units PAG1 and PAG2 each including more than one of the partial detection areas PAA arranged in the first direction Dx and the second direction Dy as the detection targets. The gate line drive circuit 15 collectively drives the predetermined number of the gate lines GCL and sequentially supplies the gate drive signals Vgcl to each unit of the predetermined number of the gate lines GCL.
The signal line selection circuit 16 includes a plurality of selection signal lines Lsel, a plurality of output signal lines Lout, and third switching elements TrS. The third switching elements TrS are provided correspondingly to the respective signal lines SGL. Six signal lines SGL(1), SGL(2), . . . , SGL(6) are coupled to the same output signal line Lout1. Six signal lines SGL(7), SGL(8), . . . , SGL(12) are coupled to the same output signal line Lout2. The output signal lines Lout1 and Lout2 are each coupled to the AFE circuit 48.
The signal lines SGL(1), SGL(2), . . . , SGL(6) are grouped into a first signal line block, and the signal lines SGL(7), SGL(8), . . . , SGL(12) are grouped into a second signal line block. The selection signal lines Lsel are coupled to the gates of the respective third switching elements TrS included in one of the signal line blocks. One of the selection signal lines Lsel is coupled to the gates of the third switching elements TrS in the signal line blocks.
Specifically, selection signal lines Lsel1, Lsel2, . . . , Lsel6 are coupled to the third switching elements TrS corresponding to the signal lines SGL(1), SGL(2), . . . , SGL(6), respectively. The selection signal line Lsel1 is coupled to the third switching element TrS corresponding to the signal line SGL(1) and the third switching element TrS corresponding to the signal line SGL(7). The selection signal line Lsel2 is coupled to the third switching element TrS corresponding to the signal line SGL(2) and the third switching element TrS corresponding to the signal line SGL(8).
The control circuit 122 (refer to
The signal line selection circuit 16 may collectively couple more than one of the signal lines SGL to the AFE circuit 48. Specifically, the control circuit 122 (refer to
The detection is performed for each of the block units PAG1 and PAG2 by the operations of the gate line drive circuit 15 and the signal line selection circuit 16. As a result, the intensity of the detection signal Vdet obtained by a one-time detection operation increases, so that the sensor sensitivity can be improved.
In the present disclosure, the detection device 1 can change the number of the partial detection areas PAA (optical sensors PD) included in each of the block units PAG1 and PAG2. Thus, the value of resolution per inch (pixels per inch (ppi), hereinafter, referred to as “definition”) can be set depending on the information to be acquired.
For example, the number of the partial detection areas PAA (optical sensors PD) included in each of the block units PAG1 and PAG2 is relatively reduced. While this setting results in a longer detection time and a lower frame rate (for example, 20 frames per second (fps) or lower), the detection can be performed at a higher definition (for example, at 300 ppi or higher). Hereafter, the term “first mode” denotes a mode of performing the detection at a lower frame rate and a higher definition. By selecting the first mode of performing the detection at a lower frame rate and a higher definition, for example, the fingerprint on the surface of a finger can be acquired at a higher definition.
Alternatively, for example, the number of the partial detection areas PAA (optical sensors PD) included in each of the block units PAG1 and PAG2 is relatively increased. While this setting results in a lower definition (for example, 50 ppi or lower), the detection can be performed at a higher frame rate (for example, at 100 fps or higher) that allows the detection to be repeatedly performed in a shorter time in one frame. Hereafter, the term “second mode” denotes a mode of performing the detection at a higher frame rate and a lower definition. By selecting the second mode of performing the detection at a higher frame rate and a lower definition, for example, a change in pulse wave with time can be more accurately detected. In the second mode, calculation of a pulse wave velocity, calculation of blood pressure, and the like are enabled by using the pulse waves acquired at a higher frame rate (for example, 1000 fps or higher).
For example, when acquiring the vascular image (vein pattern), the number of the partial detection areas PAA (optical sensors PD) included in each of the block units PAG1 and PAG2 is set to an intermediate value between those of the first mode and the second mode. This setting allows the detection to be performed at a medium frame rate that is higher than that of the first mode and lower than that of the second mode (for example, higher than 20 fps and lower than 100 fps) and at a medium definition that is lower than that of the first mode and higher than that of the second mode (for example, higher than 50 ppi and lower than 300 ppi). Hereafter, the term “third mode” denotes a mode of performing the detection at the medium frame rate and the medium definition. The third mode of performing the detection at the medium frame rate and the medium definition is suitable, for example, for acquiring the vascular pattern of veins and the like.
As illustrated in
The control circuit 122 supplies a reset signal RST2 to the reset signal line Lrst. This operation turns on the fourth switching elements TrR to electrically couple the signal lines SGL to the reference signal line Lvr. The power supply circuit 123 supplies a reference signal COM to the reference signal line Lvr. This operation supplies the reference signal COM to a capacitive element Ca (refer to
Each of the first switching elements Tr is provided correspondingly to the optical sensor PD. The first switching element Tr is made of a thin-film transistor, and in this example, made of an n-channel metal oxide semiconductor (MOS) thin-film transistor (TFT).
The gates of the first switching elements Tr belonging to the partial detection areas PAA arranged in the first direction Dx are coupled to the gate line GCL. The sources of the first switching elements Tr belonging to the partial detection areas PAA arranged in the second direction Dy are coupled to the signal line SGL. The drain of the first switching element Tr is coupled to the cathode of the optical sensor PD and the capacitive element Ca.
The anode of the optical sensor PD is supplied with the sensor power supply signal (potential) VDDSNS from the power supply circuit 123. The cathode of the optical sensor PD is supplied with the reference signal COM that serves as an initial potential of the signal line SGL and the capacitive element Ca from the power supply circuit 123.
When the partial detection area PAA is irradiated with light, a current corresponding to an amount of light flows through the optical sensor PD. As a result, an electric charge corresponding to the amount of light is stored in the capacitive element Ca. Turning on the first switching element Tr causes a current corresponding to the electric charge stored in the capacitive element Ca to flow through the signal line SGL. The signal line SGL is coupled to the AFE circuit 48 through a corresponding one of the third switching elements TrS of the signal line selection circuit 16. Thus, the detection device 1 can detect a signal corresponding to the amount of light irradiating the optical sensor PD in each of the partial detection areas PAA or signals corresponding to the amounts of light irradiating the optical sensors PD in each of the block units PAG1 and PAG2.
During a readout period Pdet (refer to
The following describes a configuration of the optical sensor PD.
The sensor structure 22 includes a TFT layer 221, an anode electrode (lower electrode) 222, the optical sensor PD, and a cathode electrode (upper electrode) 226.
The TFT layer 221 is provided with various types of wiring such as the gate lines GCL and the signal lines SGL. The sensor base member 21 and the TFT layer 221 are a drive circuit that drives the sensor and are also called a backplane.
The optical sensor PD includes an active layer 224, an electron transport layer (lower buffer layer) 223, and a hole transport layer (upper buffer layer) 225. The electron transport layer 223 is provided between the active layer 224 and the anode electrode (lower electrode) 222. The hole transport layer 225 is provided between the active layer 224 and the cathode electrode (upper electrode) 226. In other words, the electron transport layer (lower buffer layer) 223, the active layer 224, and the hole transport layer (upper buffer layer) 225 of the optical sensor PD are stacked in this order in a direction orthogonal to the sensor base member 21.
The active layer 224 changes in characteristics (for example, voltage-current characteristics and resistance value) according to light emitted thereto. An organic material is used as a material of the active layer 224. Specifically, the active layer 224 has a bulk heterostructure containing a mixture of a p-type organic semiconductor and an n-type fullerene derivative ((6,6)-phenyl-C61-butyric acid methyl ester (PCBM)) that is an n-type organic semiconductor. As the active layer 224, low-molecular-weight organic materials can be used including, for example, fullerene (C60), phenyl-C61-butyric acid methyl ester (PCBM), copper phthalocyanine (CuPc), fluorinated copper phthalocyanine (F16CuPc), 5,6,11,12-tetraphenyltetracene (rubrene), and perylene diimide (PDI) (derivative of perylene).
The active layer 224 can be formed by a vapor deposition process (dry process) using any of the low-molecular-weight organic materials listed above. In this case, the active layer 224 may be, for example, a multilayered film of CuPc and F16CuPc, or a multilayered film of rubrene and C60. The active layer 224 can also be formed by a coating process (wet process). In this case, the active layer 224 is made using a material obtained by combining any of the above-listed low-molecular-weight organic materials with a high-molecular-weight organic material. As the high-molecular-weight organic material, for example, poly(3-hexylthiophene) (P3HT) and F8-alt-benzothiadiazole (F8BT) can be used. The active layer 224 can be a film made of a mixture of P3HT and PCBM, or a film made of a mixture of F8BT and PDI.
The electron transport layer (lower buffer layer) 223 and the hole transport layer (upper buffer layer) 225 are provided to facilitate electrons and holes generated in the active layer 224 to reach the anode electrode (lower electrode) 222 or the cathode electrode (upper electrode) 226. The electron transport layer (lower buffer layer) 223 is in direct contact with the top of the anode electrode (lower electrode) 222. The active layer 224 is in direct contact with the top of the electron transport layer (lower buffer layer) 223. Polyethylenimine ethoxylated (PEIE) is used as a material of the electron transport layer (lower buffer layer) 223.
The hole transport layer (upper buffer layer) 225 is in direct contact with the top of the active layer 224, and the cathode electrode (upper electrode) 226 is in direct contact with the top of the hole transport layer (upper buffer layer) 225. The hole transport layer (upper buffer layer) 225 is a metal oxide layer. For example, tungsten oxide (WO3) or molybdenum oxide is used as the metal oxide layer.
The materials and the manufacturing methods of the electron transport layer (lower buffer layer) 223, the active layer 224, and the hole transport layer (upper buffer layer) 225 are merely exemplary, and other materials and manufacturing methods may be used.
The anode electrode (lower electrode) 222 faces the cathode electrode (upper electrode) 226 with the optical sensor PD interposed therebetween. A light-transmitting conductive material such as indium tin oxide (ITO) is used as the cathode electrode (upper electrode) 226. A metal material such as silver (Ag) or aluminum (Al) is used as the anode electrode (lower electrode) 222. Alternatively, the anode electrode (lower electrode) 222 may be an alloy material containing at least one or more of these metal materials.
The anode electrode (lower electrode) 222 can be formed as a light-transmitting transflective electrode by controlling the film thickness of the anode electrode (lower electrode) 222. For example, the anode electrode (lower electrode) 222 is formed of an Ag thin film having a thickness of 10 nm so as to have light transmittance of approximately 60%. In this case, the optical sensor PD can detect, for example, first light LD entering from a first surface FD side.
The protective film 23 is provided on a second surface FU so as to cover the cathode electrode (upper electrode) 226. The protective film 23 is a passivation film and is provided to protect the optical sensor PD.
As illustrated in
In the present disclosure, the optical sensor PD is not limited to an organic photodiode (OPD). The optical sensor PD may be a silicon photodiode (SiPD), for example.
The following describes an operation example of the detection device 1.
As illustrated in
During the reset period Prst, the gate line drive circuit 15 sequentially selects the gate lines GCL based on the start signal STV, the clock signal CK, and the reset signal RST1. The gate line drive circuit 15 sequentially supplies gate drive signals Vgcl {Vgcl(1), . . . , Vgcl(M)} to the gate lines GCL. Each of the gate drive signals Vgcl has a pulsed waveform having a power supply voltage VDD serving as a high-level voltage and a power supply voltage VSS serving as a low-level voltage. In
Specifically, as illustrated in
In the same way, the gate line drive circuit 15 supplies the gate drive signals Vgcl(2), . . . , Vgcl(M−1), Vgcl(M) at the high-level voltage to gate lines GCL(2), . . . , GCL(M−1), GCL(M) during periods V(2), . . . , V(M−1), V(M), respectively.
Thus, during the reset period Prst, the capacitive elements Ca of all the partial detection areas PAA are sequentially electrically coupled to the signal lines SGL and are supplied with the reference signal COM. As a result, the capacitance of the capacitive elements Ca is reset. The capacitance of the capacitive elements Ca of some of the partial detection areas PAA can be reset by selecting some of the gate lines and some of the signal lines SGL.
Examples of the method of controlling the exposure include a method of controlling the exposure during non-selection of the gate lines and a method of always controlling the exposure. In the method of controlling the exposure during non-selection of the gate lines, the gate drive signals {Vgcl(1), . . . , Vgcl(M)} are sequentially supplied to all the gate lines GCL coupled to the optical sensors PD serving as the detection targets, and all the optical sensors PD serving as the detection targets are supplied with the reset voltage. Then, after all the gate lines GCL coupled to the optical sensors PD serving as the detection targets are set to a low voltage (the first switching elements Tr are turned off), the exposure starts and the exposure is performed during the exposure period Pex. After the exposure ends, the gate drive signals {Vgcl(1), . . . , Vgcl(M)} are sequentially supplied to the gate lines GCL coupled to the optical sensors PD serving as the detection targets as described above, and reading is performed during the readout period Pdet. In the method of always controlling the exposure, the control for performing the exposure can also be performed during the reset period Prst and the readout period Pdet (the exposure is always controlled). In this case, the exposure period Pex(1) starts after the gate drive signal Vgcl(1) is supplied to the gate line GCL during the reset period Prst. The exposure periods Pex {(1), . . . , (M)} are substantial exposure periods during which the capacitive elements Ca are charged from the optical sensors PD. No light is emitted except in these periods. The electric charge stored in the capacitive element Ca during the reset period Prst causes a reverse-directional current (from cathode to anode) to flow through the optical sensor PD due to light irradiation, and the potential difference between both ends of the capacitive element Ca decreases. The start timing and the end timing of the substantial exposure periods Pex(1), . . . , Pex(M) are different among the partial detection areas PAA corresponding to the gate lines GCL. Each of the exposure periods Pex(1), . . . , Pex(M) starts when the gate drive signal Vgcl changes from the power supply voltage VDD serving as the high-level voltage to the power supply voltage VSS serving as the low-level voltage during the reset period Prst. Each of the exposure periods Pex(1), . . . , Pex(M) ends when the gate drive signal Vgcl changes from the power supply voltage VSS to the power supply voltage VDD during the readout period Pdet. The lengths of exposure time of the exposure periods Pex(1), . . . , Pex(M) are equal.
During the exposure periods Pex {(1) . . . (M)}, current flows correspondingly to the light received by the optical sensor PD in each of the partial detection areas PAA. As a result, an electric charge is stored in each of the capacitive elements Ca.
At a time before the readout period Pdet starts, the control circuit 122 sets the reset signal RST2 to a low-level voltage. This operation stops the operation of the reset circuit 17. The reset signal may be set to a high-level voltage only during the reset period Prst. During the readout period Pdet, the gate line drive circuit 15 sequentially supplies the gate drive signals Vgcl(1), . . . , Vgcl(M) to the gate lines GCL in the same way as during the reset period Prst.
Specifically, as illustrated in
In the same way, the gate line drive circuit 15 supplies the gate drive signals Vgcl(2), . . . , Vgcl(M−1), Vgcl(M) at the high-level voltage to the gate lines GCL(2), . . . . GCL(M−1), GCL(M) during row readout periods VR(2), . . . , VR(M−1), VR(M), respectively. That is, the gate line drive circuit 15 supplies the gate drive signal Vgcl to the gate line GCL during each of the row readout periods VR(1), VR(2), . . . . VR(M−1), VR(M). The signal line selection circuit 16 sequentially or simultaneously selects the signal lines SGL based on the selection signal ASW during each period in which the gate drive signal Vgcl is set to the high-level voltage. The signal line selection circuit 16 sequentially or simultaneously couples each of the signal lines SGL to one AFE circuit 48. Thus, the detection device 1 can output the detection signals Vdet of all the partial detection areas PAA to the AFE circuit 48 during the readout period Pdet.
With reference to
As illustrated in
Specifically, after the period t4 of turning on the switch SSW starts, the electric charge is transferred from the capacitance (capacitive element Ca) of the partial detection area PAA to the capacitance (capacitive element Cb) of the detection signal amplifying circuit 42 of the AFE circuit 48. At this time, the non-inverting input (+) of the detection signal amplifying circuit 42 is set to the reference potential (Vref) voltage (at 0.75 V, for example). Therefore, the output (Vout) of the third switching element TrS is also set to the reference potential (Vref) voltage due to the virtual short-circuit between the inputs of the detection signal amplifying circuit 42. The voltage of the capacitive element Cb is set to a voltage corresponding to the electric charge stored in the capacitance (capacitive element Ca) of the partial detection area PAA at a location where the third switching element TrS is turned on in response to the selection signal ASW(k). After the output (Vout) of the third switching element TrS is set to the reference potential (Vref) voltage due to the virtual short-circuit, the output of the detection signal amplifying circuit 42 reaches a voltage corresponding to the capacitance of the capacitive element Cb. The A/D conversion circuit 43 reads the output voltage of the detection signal amplifying circuit 42. The voltage of the capacitive element Cb is, for example, a voltage between two electrodes provided on a capacitor constituting the capacitive element Cb.
In a first example illustrated in
The control circuit 122 can control the lighting and the non-lighting of the light sources according to the detection target.
As illustrated in
As a specific example of the information on the living body acquired by the detection device 1, the following describes an example of acquiring the pulse waves serving as the biometric information for calculating the oxygen saturation level in blood (hereinafter, called “blood oxygen saturation level (SpO2)”).
When acquiring the pulse waves for calculating the blood oxygen saturation level (SpO2), for example, the red visible light (red light) having a wavelength of 600 nm to 700 nm, specifically, approximately 660 nm, is employed as the first light emitted from the first light sources 61, and the infrared light having a wavelength of 780 nm to 950 nm, specifically, approximately 850 nm, is employed as the second light emitted from the second light sources 62. When acquiring the human blood oxygen saturation level (SpO2), a pulse wave acquired using the first light (red light) and a pulse wave acquired using the second light (infrared light) are used.
Since the amount of light absorbed changes depending on the amount of oxygen taken in by hemoglobin, the optical sensor PD detects the amount of light obtained by subtracting the light absorbed by blood (hemoglobin) from the applied first and second light. Most of the oxygen in blood is reversibly bound to hemoglobin in red blood cells, and a small portion thereof is dissolved in blood plasma. More specifically, the value indicating what percentage of a permissible amount of oxygen is bound to blood as a whole is called the oxygen saturation level (SpO2). At the two wavelengths of the first light and the second light, the blood oxygen saturation level can be calculated from the amount obtained by subtracting the light absorbed by blood (hemoglobin) from the applied light.
In the first example illustrated in
In the first example illustrated in
Thus, in the first example illustrated in
In calculating the blood oxygen saturation level (SpO2), the pulse wave acquired using the first light and the pulse wave acquired using the second light are used. Therefore, the gap between the detection timing of the first detection value detected based on the first light and the detection timing of the second detection value detected based on the second light is preferably smaller. The following describes an operation example that can reduce the gap between the detection timing of the first detection value detected based on the first light and the detection timing of the second detection value detected based on the second light, with reference to
In the second example illustrated in
In the second example illustrated in
In the second example illustrated in
In the second example illustrated in
Specifically, the control circuit 122 sequentially supplies the selection signals ASW1, . . . , ASW6 to the signal line selection circuit 16 during the period in which the gate drive signal Vgcl(1) is at the high-level voltage (power supply voltage VDD). The third switching elements TrS are sequentially brought into the coupled state in response to the selection signals ASW1, . . . , ASW6. That is, during the period of reading out each row (row readout period VR(1)), when the first switching elements Tr of the signal-supplied row are in the coupled state, the signal line selection circuit 16 couples the signal lines SGL to the AFE circuit 48 column by column in a predetermined order. As a result, the detection signal Vdet of each of the partial detection areas PAA is supplied to the AFE circuit 48.
As illustrated in
After the completion of the readout period of the given row (row readout period VR(1)) and before the start of the readout period of a row next to the given row (row readout period VR(2)), a reset potential (reference signal COM) is supplied to the optical sensors PD and the signal lines SGL belonging to the given row. Specifically, the control circuit 122 supplies the reset signal RST2 to the reset signal line Lrst at time t22. This operation turns on the fourth switching elements TrR to supply the reference signal COM to the optical sensors PD corresponding to the gate line GCL(1) and the signal lines SGL.
In the second example illustrated in
Then, at time t23, the gate line drive circuit 15 sets the gate drive signal Vgcl(1) to the low-level voltage. This operation brings the first switching elements Tr of the given row into a non-coupled state. At time t24, the control circuit 122 sets the reset signal RST2 to the low-level voltage. This operation ends the readout period Pdet and the reset period Prst of the first row. The capacitance Cb of the AFE circuit 48 is reset by setting the reset switch RSW from the off state through the transition from the on state to the off state between time t22 and t24.
Then, at time t25, the gate line drive circuit 15 supplies the gate drive signal Vgcl(2) at the high-level voltage (power supply voltage VDD) to the gate line GCL(2) of the second row. Subsequently, in the same way as in the first row, the readout period Pdet and the reset period Prst of the second row are executed from time t26 to time t28. The detection for one frame (1F) can be performed by repeating the scanning operation described above to the last row (gate line GCL(256)).
In the second example illustrated in
The following describes application examples of the detection device 1 according to the first embodiment.
For the biometric information such as the pulse waves and blood flow, time-domain data in which the detection values acquired by the optical sensor PD are arranged in chronological order needs to be acquired. In the present embodiment, the following describes a specific example of processing that can acquire the chronologically varying biometric information, such as the pulse waves and the blood flow, as the image information in a detection area in each of the devices 200 and 200a to which the detection device 1 having the configuration described above is applied (refer to
The signal processing circuit 44 first stores the detection value of each of the optical sensors PD acquired by the AFE circuit 48 during a predetermined period P as first time-domain data in the storage circuit 46 (first time-domain data acquisition process, Step S100).
The predetermined period P for acquiring the detection value of each of the optical sensors PD is set to a length suitable for the pulsation frequency of the biometric information to be detected in the detection device 1. Specifically, when the biometric information to be detected in the detection device 1 is the pulse waves or the blood flow, the predetermined period P for acquiring the detection value of each of the optical sensors PD is, for example, 10 seconds to 20 seconds. When the biometric information to be detected in the detection device 1 is the pulse waves, a pulsation frequency of approximately 1 Hz to 1.5 Hz is exemplified. When the biometric information to be detected in the detection device 1 is the blood flow, a pulsation frequency of approximately 0.05 Hz to 0.15 Hz is exemplified.
The signal processing circuit 44 converts the acquired first time-domain data into a time-domain matrix A having elements representing the detection values (n, m, p) arranged in (P/t) columns by (N×M) rows given in Expression (1) below (matrix transform process, Step S200). In the time-domain matrix A, the detection values (n, m, p) are arranged in descending order of time in the row direction, and the detection values (n, m, p) are arranged in the order of spatial arrangement in the column direction. The order of arrangement in the column direction of the detection values (n, m, p) in the time-domain matrix A is not limited to Expression (1) below.
The signal processing circuit 44 reads the first time-domain data acquired in chronological order for each of the optical sensors PD from the storage circuit 46 and performs a fast Fourier transform (FFT) process on the first time-domain data at each preset unit frequency to calculate a power spectral density (PSD) (power spectral analysis process, Step S300).
Based on the results of the power spectral analysis process, the signal processing circuit 44 performs singular value decomposition (SVD) on the time-domain matrix A given in Expression (1) above, as indicated in Expression (2) below (singular value decomposition process, Step S400).
A matrix S in Expression (2) above represents a K column by K row singular value matrix in which λ1, λ2, . . . , AK are arranged as diagonal elements in descending order of the power spectral density obtained by the power spectral analysis process, and the elements other than the diagonal elements are “0”. λk is a singular value of the time-domain matrix A that represents the unit frequency in the power spectral analysis process, and a number K of singular values λk is a value determined according to the unit frequency in the power spectral analysis process.
U in Expression (2) above indicates a spatial distribution and represents a left singular vector in which elements u′k corresponding to the respective singular values λk are arranged in the row direction.
V in Expression (2) above indicates a time distribution and represents a right singular vector in which elements v′k corresponding to the respective singular values λk are arranged in the column direction.
In the orthogonal matrix U representing the spatial distribution, the elements u*k corresponding to the respective singular values λk are arranged in the row direction in descending order of the power spectral density (u*1, u*2, . . . , u*K). In each of the elements u*k of the orthogonal matrix U, spatial components uk of fluctuations in components corresponding to the respective singular values λk are arranged in the column direction (in space (N×M) direction) in the same order as that in the column direction of the time-domain matrix A.
In the orthogonal matrix V representing the time distribution, the elements v*k corresponding to the respective singular values λk are arranged in the column direction in descending order of the power spectral density (v*1, v*2, . . . , v*K). In each of the elements v*k of the orthogonal matrix V, time-periodic components vk of the fluctuations in the components corresponding to the respective singular values λk are arranged in the row direction (time (P/t) direction).
The left-hand side (time-domain matrix A) and the right-hand side (USVT) illustrated in
In the present disclosure, acquisition conditions for acquiring the desired biometric information such as the pulse waves and the blood flow are stored in advance in the storage circuit 46 as the biometric information acquisition conditions. The signal processing circuit 44 reads out the biometric information acquisition conditions stored in the storage circuit 46 and inversely calculates second time-domain data that satisfies the biometric information acquisition conditions using Expressions (1) and (2) given above (second time-domain data inverse calculation process, Step S500).
In Expression (2) given above, the signal processing circuit 44 inversely calculates the second time-domain data based on singular values, which are included in the singular values λ1, λ2, . . . , λK included in the singular value matrix S, that satisfy the biometric information acquisition conditions described above. More specifically, among the singular values λ1, λ2, . . . , λK included in the singular value matrix S, the singular values that do not satisfy the biometric information acquisition conditions are set to “0” while leaving unchanged the singular values that satisfy the above-described biometric information acquisition conditions, and the time-domain matrix A given in Expression (1) above is inversely calculated. The obtained time-domain matrix A is inversely transformed into the second time-domain data in the state illustrated in
Biometric information acquisition conditions according to characteristics of the human body are set as the biometric information acquisition conditions. For example, as the biometric information acquisition conditions when measuring a blood flow velocity, a frequency range may be set such that the time for phase inversion of outputs at two points determined from the known blood flow velocity for each wearing part falls within a predetermined range, and the singular values corresponding to the other frequencies may be set to “0”. For example, as the biometric information acquisition conditions when measuring the pulse waves, a predetermined frequency range including the frequency of a peak of a waveform may be set, and the singular values corresponding to the other frequencies may be set to “0”. In this case, the predetermined frequency range is set within a range that can be considered as the pulse wave of the human body. Alternatively, for example, a singular value corresponding to a frequency component that can be identified in advance as a noise component may be set to “0”. For example, as the frequency component that can be identified in advance as a noise component, the frequency component of a singular value λb illustrated in
The signal processing circuit 44 uses the second time-domain data of each of the optical sensors PD obtained by the second time-domain data inverse calculation process to generate the biometric information to be detected in the detection device 1 (biometric information generation process, Step S600).
The process described above can remove, for example, noise components other than the biometric information to be detected in the detection device 1, such as the pulse waves and the blood flow (for example, body motion noise caused by human body movement or the like, biosignals not intended to be detected, or noise components at an alternating-current frequency of a commercial power supply (for example, at 50 Hz or 60 Hz)).
As illustrated in
The FFT process can obtain the magnitude (amplitude) of the frequency components of the pulse wave included in the pulse waveform, but, as illustrated in
In contrast, as illustrated in
In the above-described process according to the present embodiment, the singular value decomposition process (Step S400 in
The time-domain data (second time-domain data) of each of the optical sensors PD obtained by the second time-domain data inverse calculation process is used to perform the biometric information generation process (Step S600 in
The process described above can obtain the desired biometric information, such as the pulse waves and the blood flow, as the image information in the detection area.
In the embodiment described above, an example of performing the biometric information generation process (Step S600 in
While the preferred embodiment has been described above, the present invention is not limited to such an embodiment. The content disclosed in the embodiment is merely an example, and can be variously modified within the scope not departing from the gist of the present invention. Any modifications appropriately made within the scope not departing from the gist of the present invention also naturally belong to the technical scope of the present invention. At least one of various omissions, substitutions, and changes of the components can be made without departing from the gist of the embodiment described above and the modifications thereof.
| Number | Date | Country | Kind |
|---|---|---|---|
| 2022-110345 | Jul 2022 | JP | national |
This application claims the benefit of priority from Japanese Patent Application No. 2022-110345 filed on Jul. 8, 2022 and International Patent Application No. PCT/JP2023/024123 filed on Jun. 29, 2023, the entire contents of which are incorporated herein by reference.
| Number | Date | Country | |
|---|---|---|---|
| Parent | PCT/JP2023/024123 | Jun 2023 | WO |
| Child | 19009680 | US |
