This application claims priority to Korean Patent Application No. 10-2019-0057878, filed on May 17, 2019 in the Korean Intellectual Property Office, the disclosure of which is incorporated herein by reference in its entirety.
Apparatuses and methods consistent with one or more exemplary embodiments relate to a biometric signal measuring device.
Recently, there has been growing demand for electronic devices having healthcare functions. Thus, a biometric signal measuring device capable of measuring various biometric signals, such as a user's heart rate and blood oxygen saturation (SpO2), has been gaining attention. A biometric signal measuring device is desired to have a significantly reduced size and power consumption as an electronic device. Accordingly, research has been actively conducted on an apparatus and a method for measuring a biological signal using an image sensor.
Aspects of one or more exemplary embodiments provide a biometric signal measuring device in which miniaturization thereof may be implemented using a pixel array of an image sensor and a user's biometric signals may be measured with relatively low power.
According to an aspect of an exemplary embodiment, a biometric signal measuring device includes: a light source configured to output a light signal to an object; a pixel array connected to a plurality of row lines and a plurality of column lines, and including a plurality of unit pixels configured to sense a reflected light signal corresponding to a reflection of the output light signal by the object; a binning controller configured to group the plurality of unit pixels to obtain one or more macro pixels; and a controller configured to drive the light source and to obtain a biometric signal of the object, based on a pixel signal generated by each of the one or more macro pixels according to the reflected light signal.
According to an aspect of an exemplary embodiment, a biometric signal measuring device includes: a pixel array including a plurality of unit pixels, the plurality of unit pixels being grouped into one or more macro pixels; a readout circuit configured to detect, during a first time period, a pixel signal output by each of the one or more macro pixels based on an external light signal, and to convert the detected pixel signal into a digital signal during a second time period; and a control logic configured to control operation timings of the pixel array and the readout circuit, and to sum the digital signal to generate a biometric signal.
According to an aspect of an exemplary embodiment, a biometric signal measuring device includes: a plurality of unit pixels connected to a plurality of row lines and a plurality of column lines; a binning controller configured to group the plurality of unit pixels to obtain one or more macro pixels; and a controller configured to convert a pixel signal generated by each of the one or more macro pixels based on an external light signal into a digital signal, and to generate a biometric signal by applying a predetermined weight to the digital signal.
According to an aspect of an exemplary embodiment, a biometric signal measuring method using a biometric signal measuring device, includes: controlling to output a light single from a light source of the biometric signal measuring device to an object; controlling to sense, by a plurality of unit pixels in a pixel array, a reflected light signal corresponding to a reflection of the output light signal by the object; and obtaining a biometric signal of the object based on a pixel signal generated by each of one or more macro pixels according to the reflected light signal, the one or more macro pixels each including unit pixels grouped together from among the plurality of unit pixels.
The above and other aspects, features, and advantages of certain exemplary embodiments will be more clearly understood from the following detailed description, taken in conjunction with the accompanying drawings, in which:
Hereinafter, exemplary embodiments will be described in detail with reference to the accompanying drawings. The same reference numerals are used for the same constituent elements in the drawings, and redundant description of the same constituent elements will be omitted.
Referring to
The light source 10 may include at least one light emitting element. For example, the light source 10 may include at least one of a light emitting diode (LED), a laser diode, a vertical cavity surface emitting laser (VCSEL), and a phosphor. A plurality of light emitting elements included in the light source 10 may be arranged in an array form.
The sensing device 20 may detect a light signal scattered or reflected by living tissue of the object 2 to generate a living body signal. The sensing device 20 may include a pixel array that generates an electric signal in response to the detected light signal, and a controller that generates a living body signal using the electric signal generated by the pixel array. In an exemplary embodiment, the biometric signals may include a pulse wave signal (PPG), an electrocardiography (ECG) signal and an electromyography (EMG) signal. The object 2 may be a body part of the user who touches or is adjacent to the sensing device 20 and a body part that facilitates generation of a biometric signal. For example, in the case of measuring the pulse wave signal PPG, the object 2 may be a wrist, an ear, or the like, which is a body part relatively thin in skin tissue and high in blood vessel density.
Referring first to
The pixel array 100 may include a plurality of unit pixels PX. When a plurality of unit pixels PX are arranged in a matrix form, the unit pixels PX may be disposed at the intersections of the plurality of row lines and the plurality of column lines. An example of a pixel array 100 that may be applied to one or more exemplary embodiments is illustrated in
Referring to
Specific examples of the unit pixel PX are as shown in
Referring to
When the light signal output from the light source is reflected from the object and is incident on the pixel array 100, the photodiode PD may generate charges in response to the incident light. The charge generated by the photodiode PD may be accumulated in the floating diffusion FD.
When the reset transistor RX is turned on by the reset control signal RG, the voltage of the floating diffusion FD may be reset to the power supply voltage VDD. When the voltage of the floating diffusion FD is reset, the selection transistor SX is turned on by the selection control signal SEL so that the reset voltage may be output to the column line COL through the pixel node PN.
If the transfer transistor TX is turned on by the transfer control signal TG after the reset voltage is output to the column line COL, the charge generated by the photodiode PD may be transferred to the floating diffusion FD.
The driving transistor DX may operate as a source-follower amplifier that amplifies the voltage of the floating diffusion FD. When the selection transistor SX is turned on by the selection control signal SEL, the pixel voltage corresponding to the amount of charge generated by the photodiode PD may be output to the column line COL through the pixel node PN.
Referring to
The first photodiode PD1 and the second photodiode PD2 may share a floating diffusion FD and a reset transistor RX. The charge generated by the first photodiode PD1 and the second photodiode PD2 may be accumulated in the floating diffusion FD. When the selection transistor SX is turned on by the selection control signal SEL, the pixel voltage (pixel signal) corresponding to the total amount of charges generated by the first photodiode PD1 and the second photodiode PD2 is supplied to the pixel node PN and the column line COL.
Referring to
The photoelectric element 510 (or optoelectronic device) may be disposed below the microlens 530 and the pixel circuit 520 may be disposed below the photoelectric element 510. In some exemplary embodiments, the unit pixels PX may share a microlens in a predetermined unit. For example, referring to
Referring to
Referring back to
The binning controller 200 may also constitute a macro pixel MP by using a unit pixel PX sharing a floating diffusion FD as shown in
The biometric signal measuring device 1 may reduce the number of unit pixels PX read out from the pixel array 100 by constructing the macro pixel MP to reduce the readout time and the number of analog-to-digital converters.
In addition, the biometric signal measurement apparatus 1 may perform spatial oversampling by constructing the macro pixel MP, and may increase the resolution of the output voltage through the pixel node of the macro pixel MP. For example, when a macro pixel MP having a size of 2×1 including two unit pixels PX is constructed, the resolution of the output voltage through the pixel node of the macro pixel MP may be increased by 0.5 bits. When a macro pixel (MP) having a size of 128×128 is configured for the pixel array having a dynamic range (DR) of 12-bits, the dynamic range DR of the pixel array is set to 19.
Referring to
Referring to
Referring to
The binning operation for the first row line RO[1] and the second row line ROW[2] is performed by simultaneously applying the first row line (ROW[1]) select signal and the second row line (ROW[2]) select signal.
The binning operation for the column lines COL[1]-COL[n] is performed by the switching operation of the first through the n-th switches SW1-SWn arranged between the column lines COL[1]-COL[n]. For example, when the first switch SW1 between the first column line COL[1] and the second column line COL[2] is turned on, the first column line COL[1] and the second column line COL[2] may be grouped by each row line ROW[1]-ROW[m].
The biometric signal measuring device 1 may configure the macro pixels MP in various shapes and sizes based on the type of biometric signals, the dynamic range of the image sensor, and the like. For example, the biometric signal measuring device 1 may use a 14-bit dynamic range image sensor to measure 16-bit or more dynamic range suitable for measurement of the pulse wave signal PPG and sampling over 100 samples per sec (SPS). To achieve this rate, a macro pixel (MP) in the form of a square having a size of 4×4 may be constructed as shown in
In addition, the biometric signal measuring device 1 may configure the macro pixels MP in various shapes and sizes based on the distance from the center of the pixel array 100. For example, in order to uniformly maintain the light sensitivity of the unit pixels PX, the biometric signal measuring device 1 may be configured to have a relatively large distance from the center of the pixel array 100. As shown in
Referring to
The row driver 310 may drive the pixel array 100 on a row-by-row basis. For example, the row driver 310 may generate a transfer control signal for controlling the transfer transistor of the unit pixel PX, a reset control signal for controlling the reset transistor, a selection control signal for controlling the selection transistor, and the like on a row-by-row basis. In an exemplary embodiment, the row driver 310 may drive at least some of the plurality of row lines under the control of the binning controller 200 in synchronization. For example, when the first row line and the second row line are binned, the row driver 310 may simultaneously drive the first row line and the second row line. In this case, the unit pixels PX connected to the respective column lines are grouped to constitute the macro pixels MP, and the pixel signals of the grouped unit pixels PX are summed and output.
The readout circuit 320 may detect a pixel signal output from the unit pixels PX included in the pixel array 100 and convert the pixel signal into a digital signal. Examples of the readout circuit 320 according to various exemplary embodiments are shown in
Referring to
The amplifier 321 may amplify the pixel signals output from the respective macro pixels MP with a predetermined amplification ratio.
In an exemplary embodiment, the amplification ratio of the amplifier 321 may be changed dynamically based on the current light amount of the light source 10. For example, when the current light amount of the light source 10 is equal to or greater than a predetermined threshold value, the control logic 330 may set the amplification ratio of the amplifier 321 to a first value. Conversely, when the current amount of light in the light source 10 is below a predetermined threshold, the control logic 330 may set the amplification ratio of the amplifier 321 to a second value greater than the first value.
In an exemplary embodiment, the amplifier 321 may analog-sum-amplify the pixel signals output from the macro pixels MP. For example, the amplifier 321 may sum the pixel signals output from the macro pixels MP, and amplify the summed pixel signals at a predetermined amplification ratio.
The analog-to-digital converter 323 may convert the pixel signal amplified by the amplifier 321 into a digital signal. The pixel signal converted into the digital signal may be stored in the buffer 325.
Referring to
Referring back to
The timing generator 331 may generate various timing signals for controlling the operation of the controller 300. For example, the timing generator 331 may generate a timing signal for synchronizing the driving timing of the light source 10 with the detection timing of the pixel signal, and output the timing signal to the light source driver 340. The biometric signal measurement apparatus 1 may minimize the driving time and power consumption of the light source 10 by synchronizing the driving timing of the light source 10 with the detection timing of the pixel signal.
The digital binning unit 332 may generate a biometric signal using the digital signals stored in the buffer 325. For example, the digital binning unit 332 may generate a living body (or biometric) signal by cumulatively averaging the digital values of the digital signals stored in the buffer 325.
The biometric signal generated by the digital binning unit 332 may be transmitted to an external processor or the like and be used to acquire biometric information of the object. For example, the external processor may analyze biometric signals transmitted from the digital binning unit 332 to obtain biological information such as blood pressure, blood vessel age, arteriosclerosis, aortic pressure waveform, and stress index of the object.
The light source driver 340 may generate a predetermined pulse signal to drive the light source 10. For example, the light source driver 340 may determine the period, the duty ratio, and the duration of the pulse signal based on the timing signal generated by the timing generator 331. In an exemplary embodiment, the light source driver 340 may drive the light source 10 in synchronization with the detection time of the pixel signal.
Referring to
In the first section D1, the biometric signal measuring device 1 may drive a light source (e.g., LED) 10 and output a light signal toward a part of the user's body. The biometric signal measuring device 1 according to one or more exemplary embodiments controls the first section D1 to a very short time (e.g., 100 μs) in comparison with the second section D2 and the third section D3, thereby minimizing driving of the light source 10.
A photodiode (PD) of a biometric signal measuring device 1 may receive a light signal by a rolling shutter method or a global shutter method to generate charges. The biometric signal measuring device 1 may convert the charge generated by the photodiode PD into a pixel signal, and then add pixel signals by a predetermined pixel unit through an analog binning operation. In an exemplary embodiment, the biometric signal measuring device 1 may perform an analog binning operation by grouping unit pixels PX and constructing at least one macro pixel (MP). In this case, the biometric signal measuring device 1 may perform the readout operation for each macro pixel (MP), thereby reducing the readout time.
In the first section D1, the biometric signal measurement device 1 may perform a first operation for detecting pixel signals of the respective macro pixels MP. In an exemplary embodiment, the biometric signal measuring device 1 may store the detected pixel signal in the S&H circuit 327 provided for each macro pixel (MP).
In the second section D2, the biometric signal measurement device 1 may perform a second operation of converting the pixel signal detected from each macro pixel MP into a digital signal.
In an exemplary embodiment, the biometric signal measurement device 1 may sequentially perform the second operation on the pixel signal detected from each macro pixel (MP). Referring to
In an exemplary embodiment, the biometric signal measurement device 1 performs an analog summing amplification operation on all or a part of the pixel signals detected from the macro pixels MP, and the second operation may be performed simultaneously. For example, in the above example, when performing an analog summing amplification operation on pixel signals of all macro pixels, the second section D2 may be 100 μs. As a result, the manufacturing cost of the biometric signal measuring device 1 may be reduced by minimizing the total number of analog-to-digital converters (ADCs) required to convert the pixel signals into digital signals.
In the third section D3, the biometric signal measuring device 1 may perform a third operation of calculating the biometric signal using the pixel signals converted into the digital signal. In an exemplary embodiment, the biometric signal measurement device 1 may minimize power consumption by operating in a power-down mode that deactivates the functions of various components related to the light-emitting operation of the light source 10 and the light-receiving operation of the photodiode PD. In the example of
The first to third operations of the biometric signal measuring device are summarized as shown in
Referring to
Thereafter, the biometric signal measurement device 1 may generate the biometric signal D by performing a digital binning operation on the digital signals D1-Dn stored in the buffers BUF1-BUFn. For example, the biometric signal measuring device 1 may generate the biometric signal D by cumulatively averaging the digital values of the digital signals D1-Dn.
In an exemplary embodiment, the biometric signal measurement device 1 may perform a digital binning operation after applying predetermined weights to the digital values of each of the digital signals D1-Dn. For example, the biometric signal measurement device 1 applies a first weight to the first macro pixel MP, and a second weight, greater than the first weight, to the second macro pixel MP2, which has a larger distance from the center of the pixel array than the first macro pixel MP1, after which a digital binning operation may be performed. The biometric signal measurement apparatus may improve the measurement accuracy by applying different weights to each of the digital signals (D1-Dn) based on the distance from the center of the pixel array and the like.
The biometric signal D generated from the pixel signals A1-An may be transmitted to a processor (e.g., external processor) and be used to acquire biometric information of the user.
Referring to
The biometric signal measurement device 1210 is as described above with reference to
The input/output device 1220 may include an input device such as a keyboard, a mouse, a touch screen, etc., and an output device, such as a display, an audio output, etc.
The memory 1230 may be a storage medium for storing data used in the operation of the electronic device 1200, or multimedia data. Memory 1230 may include volatile memory, or non-volatile memory such as flash memory and the like. Also, the memory 1230 may include at least one of a solid state drive (SSD), a hard disk drive (HDD), and an optical drive (ODD) as a storage device.
A processor 1240 (e.g., at least one processor) may perform particular operations, commands, tasks, and so on. The processor 1240 may be a central processing unit (CPU) or microprocessor unit (MCU), a system on chip (SoC), etc., and may be coupled to the biometric signal measurement device 1210, the display 1220, 1230, as well as other devices connected through the communication module 1250 (e.g., communicator, communication interface, etc.).
First, referring to
The electronic device 1300 may include a body and a wrist strap. A display is provided on the front of the main body, and various application screens including time information, received message information, and the like may be displayed. The user may wear the electronic device 1300 on the wrist using the strap.
A biometric signal measuring device 1310 may be disposed on the rear surface of the main body. The living body or biometric signal measuring device 1310 may output a light signal to a body region that is in contact with the back surface of the main body such as a wrist of a user, and may measure the living body or biometric signal by sensing the reflected light reflected from the body region. The electronic device 1300 may analyze the biometric signal measured by the biometric signal measuring device 1310 to acquire biometric information of the user such as blood pressure, blood vessel age, arteriosclerosis, aortic pressure waveform, a stress index, or the like.
Referring to
The electronic device 1400 may include a housing and a display panel.
The housing may form the appearance of the electronic device 1400. The housing may include a first surface, a second surface opposite the first surface, and a side surface surrounding the space between the first surface and the second surface.
A display panel and a cover glass may be sequentially arranged on the first surface of the housing. The display panel may be exposed to the outside through the cover glass.
On the second side of the housing, a biometric signal measuring device 1410, a camera module and an infrared sensor may be disposed.
When a user requests biometric information by executing an application or the like mounted on the electronic device 1400, the biometric signal measuring device 1410 may measure the biometric signal by sensing reflected light obtained from a part of the user's body. The electronic device 1400 may acquire biometric information of the user by analyzing the biometric signal measured by the biometric signal measuring device 1410.
Referring to
The electronic device 1500 may include a body and an ear strap.
The user may wear the electronic device 1500 by wearing the ear strap on the auricle. With the user wearing the electronic device 1500, the main body may be inserted into the external auditory meatus of the user.
The body may be equipped with a biometric signal measuring device 1510. The living body or biometric signal measuring device 1510 may output a light signal to a body region that is in contact with the body, such as a wall surface of a user's ear canal, and may measure a living body signal by sensing reflected light reflected from the body region. The wall of the external ear canal of the user is thinner than other areas of the body, so that it is easy to measure the biometric signals such as blood flow. The electronic device 1500 may acquire biometric information of the user by analyzing the biometric signal measured by the biometric signal measuring device 1510.
As set forth above, an apparatus for measuring biometric signals according to one or more exemplary embodiments may minimize size and power consumption by using an image sensor.
In addition, the biometric signal measuring device according to one or more exemplary embodiments minimizes the number of analog-digital converters required to generate a biometric signal, thereby minimizing the manufacturing cost.
Furthermore, the biometric signal measuring device according to one or more exemplary embodiments may minimize the size and noise by implementing the light receiving unit and the driving circuit on a single chip.
The various advantages and effects of the present inventive concept are not limited to the above description.
While exemplary embodiments have been shown and described above, it will be apparent to those skilled in the art that modifications and variations could be made without departing from the scope of the present inventive concept as defined by at least the appended claims.
Number | Date | Country | Kind |
---|---|---|---|
1020190057878 | May 2019 | KR | national |