Patent Application
20210386375
This application is based on and claims priority under 35 U.S.C. § 119 to Korean Patent Application No. 10-2020-0072334, filed on Jun. 15, 2020, in the Korean Intellectual Property Office, the disclosure of which is incorporated by reference herein in its entirety.
The following description relates to a foldable electronic device, and technology for estimating bio-information by using the same.
Generally, methods of non-invasively measuring blood pressure without damaging a human body include a method to measure blood pressure by measuring a cuff-based pressure and a method to estimate blood pressure by measuring pulse waves without the use of a cuff.
A Korotkoff-sound method is one of the cuff-based blood pressure measurement methods, in which a pressure in a cuff wound around an upper arm is increased and blood pressure is measured by listening to the sound generated in the blood vessel through a stethoscope while decreasing the pressure. Another cuff-based blood pressure measurement method is an oscillometric method using an automated machine, in which a cuff is wound around an upper arm, a pressure in the cuff is increased, a pressure in the cuff is continuously measured while the cuff pressure is gradually decreased, and blood pressure is measured based on a point where a change in a pressure signal is large.
Cuffless blood pressure measurement methods generally include a method of estimating blood pressure by calculating a Pulse Transit Time (PTT), and a Pulse Wave Analysis (PWA) method of estimating blood pressure by analyzing a pulse wave shape.
According to an aspect of an example embodiment, a foldable electronic device may include a main body part, having a first main body and a second main body, configured to fold along a folding line; a first sensor provided on the first main body, and configured to obtain a contact image of an object of a user; a second sensor provided on the first main body, and configured to measure a degree of folding of the main body part; and a processor configured to estimate bio-information of the user, based on the contact image of the object and the degree of folding.
The first sensor may include an image sensor configured to obtain the contact image of the object based on the second main body rotating to press the object while the object is in contact with the first sensor.
The image sensor may include a complementary metal-oxide semiconductor (CMOS) image sensor (CIS).
The second sensor may include a Hall sensor configured to measure the degree of folding of the main body part based on the second main body rotating to press the object while the object contacts the first sensor.
The foldable electronic device may include a display part having a first display provided on an inner surface of the first main body, and a second display provided on an inner surface of the second main body.
The first display and the second display may be integrally formed with each other, and the display part may be configured to fold along the folding line.
The processor may control the display part to output a processing result.
The processor may control the first display to display a bio-information estimation result, and may control the second display to output information for estimating the bio-information.
The processor may control the second display to display a bio-information estimation history, and based on a user selecting an estimation history of a specific time, may control the first display to display a bio-information estimation result of the specific time.
The display part may include a third display provided on an outer surface of the second main body, and the processor may the third display to display information for guiding contact pressure between the object and the first sensor based on the degree of folding measured by the second sensor.
The processor may obtain a pulse wave signal based on the contact image, and obtain contact pressure, exerted by the second main body on the object, based on the degree of folding.
The processor may obtain an oscillometric waveform envelope based on the pulse wave signal and the contact pressure, and estimate bio-information based on the obtained oscillometric waveform envelope.
The bio-information may include one or more of blood pressure, vascular age, arterial stiffness, aortic pressure waveform, vascular compliance, stress index, and fatigue level.
According to an aspect of an example embodiment, a method of estimating bio-information of a user, by a foldable electronic device comprising a foldable main body part having a first main body and a second main body, may include obtaining a contact image of an object of the user by a first sensor provided on the first main body; measuring a degree of folding of the main body part by a second sensor provided on the first main body; and estimating the bio-information of the user, based on the contact image and the degree of folding.
The obtaining the contact image may include obtaining the contact image of the object based on the second main body rotating to press the object while the object contacts the first sensor.
The measuring the degree of folding may include measuring the degree of folding of the main body part based on the second main body rotating to press the object while the object contacts the first sensor.
The estimating the bio-information may include obtaining a pulse wave signal based on the contact image; obtaining contact pressure, exerted by the second main body on the object when the second main body rotates, based on the degree of folding; and estimating the bio-information of the user, based on the pulse wave signal and the contact pressure.
The estimating the bio-information may include obtaining an oscillometric waveform envelope based on the pulse wave signal and the contact pressure; and estimating the bio-information based on the obtained oscillometric waveform envelope.
The method may include outputting information for guiding contact pressure between the object and the first sensor based on the measured degree of folding, on a display provided on an outer surface of the second main body.
The method may include outputting an estimation result of the bio-information.
The above and other aspects, features, and advantages of certain embodiments of the present disclosure will be more apparent from the following description taken in conjunction with the accompanying drawings, in which:
Details of example embodiments are included in the following detailed description and drawings. Advantages and features of the present disclosure, and a method of achieving the same will be more clearly understood from the following embodiments described in detail with reference to the accompanying drawings. Throughout the drawings and the detailed description, unless otherwise described, the same drawing reference numerals will be understood to refer to the same elements, features, and structures.
It will be understood that, although the terms “first,” “second,” etc. may be used herein to describe various elements, these elements should not be limited by these terms. These terms are used to distinguish one element from another. Also, the singular forms of terms are intended to include the plural forms of the terms as well, unless the context clearly indicates otherwise. It will be further understood that when an element is referred to as “comprising” another element, the element is intended not to exclude one or more other elements, and may further include one or more other elements, unless explicitly described to the contrary. In the following description, terms such as “unit” and “module” indicate a unit for processing at least one function or operation and units may be implemented by using hardware, software, or a combination thereof.
Hereinafter, embodiments of a foldable electronic device and a method of estimating bio-information using the same will be described in detail with reference to the accompanying drawings.
The main body part 110 includes a first main body 111 and a second main body 112, which are disposed along a folding line 113, and may be connected to each other via a hinge. Further, the first main body 111 and the second main body 112 may be folded and unfolded by rotating with respect to the folding line 113. In this case, while the first main body 111 and the second main body 112 rotate, a display surface thereof may be folded inward. The display surface of the main body part 110 refers to a surface on which a first display 131 and a second display 132 of the display part 130 are disposed. However, the display surface is not limited thereto and may be folded in an opposite direction.
As illustrated herein, the sensor part 120 may be disposed in a predetermined area of the first main body 111. The sensor part 120 may include a first sensor 121 and a second sensor 122. The first sensor 121 may obtain a contact image of an object when the object is in contact with the sensor. In this case, the object may be a user's finger, and the contact image of the object may be a fingerprint image of the finger.
The first sensor 121 may be an optical image sensor, such as a complementary metal-oxide semiconductor (CMOS) image sensor (CIS), but is not limited thereto. The first sensor 121 may include a light source for emitting light onto the object in contact with the first sensor 121. The light source may include a light emitting diode (LED), a laser diode, and the like, and may be formed as one or more arrays of a plurality of light sources. However, the light source is not limited thereto, and the first sensor 121 may use, for example, light emanating from the first display 131 and/or the second display 132 of the display part 130 as an external light source.
Further, the first sensor 121 may include a pixel array, and each pixel of the pixel array may include a detector such as a photo diode, a photo transistor, and the like. A detector of each pixel may detect light scattered or reflected from the object, and may output pixel data representing a contact image of the object.
The second sensor 122 may measure a degree of folding when the first main body 111 and the second main body 112 are folded by rotating. For example, the second sensor 122 may include a Hall sensor, which outputs a value indicative of a degree of folding of the main body part 110 by using the Hall effect occurring when the second main body 112 rotates. However, the second sensor 122 is not limited thereto.
Although not illustrated herein, the sensor part 120 may further include one or more additional image sensors in the first main body 111 and/or the second main body 112. For example, an additional image sensor may perform various functions, including a function of guiding a contact position of the object by photographing the object while the first sensor 121 acquires a contact image of the object.
The display part 130 includes the first display 131 and the second display 132, which may be disposed on a display surface of the first main body 111 and the second main body 112, respectively. The first display 131 and the second display 132 may be integrally formed with each other, and may be made of a flexible material so that the first display 131 and the second display 132 may be folded and unfolded when the first main body 111 and the second main body 112 rotate along the folding line 113. However, the first display 131 and the second display 132 are not limited thereto, and may be separated from each other to be disposed at the first main body 111 and the second main body 112, respectively. The first display 131 and the second display 132 may include a touch screen for receiving a touch input. Further, the first display 131 and the second display 132 may also include a fingerprint sensor for acquiring a fingerprint image when a user's body part touches the sensor.
Referring to
While the main body part 110 is folded, a user may input instructions, such as a request for estimating bio-information, a request for displaying bio-information estimation history, and a request for outputting a health monitoring result, through the third display 133. Furthermore, when a user unfolds the third display 133 during an operation for estimating bio-information through the third display 133, information displayed on the third display 133 may be enlarged on the first display 131 and the second display 132.
The foldable electronic device 100 may include a processor for processing various functions thereof, and various other hardware modules, such as a communication module, a storage module, and the like, which are mounted in the main body part 110. For example, the processor may control the sensor part 120 in response to a request for estimating bio-information, and may estimate bio-information based on the contact image of the object, obtained by the first sensor 121, and a degree of folding of the main body part 110, output by the second sensor 122. In this case, bio-information may include, but is not limited to, blood pressure, vascular age, arterial stiffness, aortic pressure waveform, vascular compliance, stress index, fatigue level, skin elasticity, skin age, and the like.
Referring to
Based on receiving a request for estimating bio-information, the processor 230 may control the first sensor 210 and the second sensor 220, and may estimate bio-information based on a contact image and a degree of folding, which are obtained by the first sensor 210 and the second sensor 220, respectively.
Referring to
The processor 230 may obtain a pulse wave signal based on a change in pixel intensity of the contact image received from the first sensor 210. Referring to
For example, the processor 230 may convert the pixel intensity at each time point of the predetermined period of time into a pulse wave amplitude at each time point by using an amplitude conversion model which represents a correlation between the pixel intensity and the amplitude. For example, the amplitude conversion model may be an equation for calculating an average of the pixel intensities, but is not limited thereto. In addition, the processor 230 may set a region of interest by using the contact image, and may obtain an amplitude based on the intensity of pixels in the set region of interest. In this case, the processor 230 may set the region of interest based on a predetermined characteristic point, such as a center point of a fingerprint or a center point of the contact image.
Referring back to
In
Based on obtaining the pulse wave signal and the contact pressure, the processor 230 may estimate bio-information based on the obtained pulse wave signal and contact pressure. For example, the processor 230 may estimate blood pressure using oscillometry based on the pulse wave signal and the contact pressure.
Referring to
Based on extracting the features, the processor 230 may estimate blood pressure by applying a pre-defined blood pressure estimation model. The blood pressure estimation model may be defined as various linear or non-linear combination functions, such as addition, subtraction, division, multiplication, logarithmic value, regression equation, and the like, with no specific limitation. For example, the following Equation 1 represents a simple linear function.
y=ax+b [Equation 1]
Herein, “y” denotes an estimated blood pressure value to be obtained; “x” denotes an extracted feature value; and “a” and “b” are values pre-obtained by preprocessing, and may be values personalized for each user. For example, by using the above Equation 1 which is defined for each of mean arterial pressure (MAP), diastolic blood pressure (DBP), and systolic blood pressure (SBP), the processor 230 may independently estimate each blood pressure. For example, by inputting the extracted feature values MP, DP, and SP into the function, which is defined for each of the feature values, the processor 230 may obtain MAP, DBP, and SBP independently.
As illustrated in
Referring to
For example, the processor 430 may output information, such as “place your finger on the first sensor,” by voice using a speaker of the foldable electronic device 400.
In another example, when the main body is folded, the processor 430 may display a visual marker, indicating a contact position of the finger, on the third display 133 disposed on the cover surface of the second main body 112; and when the main body is unfolded, the processor 430 may display the visual marker on the first display or the second display disposed on the display surface of the first main body 111 and the second main body 112.
In yet another example, when the user places the object OBJ on the first sensor 410, the processor 430 may display voice or visual information, such as “fold the main body to press the finger,” through a speaker or a display as guide information for the user. In this case, when the user rotates and folds the second main body 112, the guide information displayed on the first display or the second display may be moved to the third display 133 to be continuously displayed thereon.
In still another example, as illustrated in
Referring to
For example, as illustrated in
In another example, as illustrated in
The storage 450 may store a variety of information related to bio-information such as contact image data, a degree of folding between the first and second main bodies, a pulse wave signal, contact pressure, an estimated bio-information value, health condition monitoring information, and the like. Furthermore, the storage 450 may store reference information for estimating bio-information such as user characteristics information, such as a user's age, sex, stature, weight, health condition, and the like, a bio-information estimation model, an amplitude conversion model, and the like, but the reference information is not limited thereto.
In this case, the storage 450 may include at least one storage medium of a flash memory type memory, a hard disk type memory, a multimedia card micro type memory, a card type memory (e.g., a secure digital (SD) memory, an extreme digital (XD) memory, etc.), a Random Access Memory (RAM), a Static Random Access Memory (SRAM), a Read Only Memory (ROM), an Electrically Erasable Programmable Read Only Memory (EEPROM), a Programmable Read Only Memory (PROM), a magnetic memory, a magnetic disk, and an optical disk, and the like, but is not limited thereto.
The communication interface 460 may communicate with an external device by using wired or wireless communication techniques under the control of the processor 430, and may transmit and receive various data to and from the external device. For example, the communication interface 460 may transmit a bio-information estimation result to the external device, and may receive, from the external device, a variety of reference information for estimating bio-information. In this case, the external device may include a cuff-type blood pressure measuring device, and an information processing device such as a smartphone, a tablet personal computer (PC), a desktop computer, a laptop computer, and the like.
In this case, examples of the communication techniques may include Bluetooth communication, Bluetooth Low Energy (BLE) communication, Near Field Communication (NFC), WLAN communication, Zigbee communication, Infrared Data Association (IrDA) communication, wireless fidelity (Wi-Fi) Direct (WFD) communication, Ultra-Wideband (UWB) communication, Ant+ communication, Wi-Fi communication, Radio Frequency Identification (RFID) communication, 3G communication, 4G communication, 5G communication, and the like. However, this is merely exemplary and is not intended to be limiting.
The foldable electronic devices 200 and 400 may obtain a contact image of an object by using the first sensor disposed at the first main body of the main body part in operation 610. In this case, the first sensor may include a CIS, and may obtain a contact image of the object when a user places the object on the first sensor and then applies gradually increasing or decreasing pressure to the object by rotating the second main body.
The foldable electronic devices 200 and 400 may measure a degree of folding between the first main body and the second main body in operation 620 by using the second sensor while the user changes pressure, applied to the object, by rotating the second main body. In this case, the degree of folding may be an angle formed between the first main body and the second main body, and the second sensor may include a Hall sensor.
Then, the foldable electronic devices 200 and 400 may obtain a pulse wave signal in operation 630 based on the contact image obtained in operation 610. In this case, the foldable electronic devices 200 and 400 may extract a change in pulse wave amplitude based on a change in pixel intensity of the contact images of the object which are obtained successively during a predetermined period of time.
Subsequently, the foldable electronic devices 200 and 400 may obtain contact pressure in operation 640 based on the degree of folding of the main bodies which is measured in operation 620. By using a pre-defined contact pressure conversion model, the foldable electronic devices 200 and 400 may convert a change in the degree of folding during a predetermined period of time into a change in the contact pressure.
Next, the foldable electronic devices 200 and 400 may estimate bio-information based on the pulse wave signal and the contact pressure in operation 650. For example, based on a relationship between the amplitude of the pulse wave signal and the contact pressure, the foldable electronic devices 200 and 400 may estimate blood pressure using oscillometry.
Then, the foldable electronic devices 200 and 400 may output a bio-information estimation result in operation 660. The foldable electronic devices 200 and 400 may output the bio-information estimation result in various manners by using a plurality of displays disposed on a display surface of the first main body, a display surface of the second main body, a cover surface of the second main body, and the like. Further, the foldable electronic devices 200 and 400 may provide a user with the bio-information estimation result, a health condition, and the like, by voice, vibrations, tactile sensation, and the like, by using a speaker, a haptic module, and the like.
The example embodiments of the present disclosure can be implemented by computer-readable code, written on a non-transitory computer-readable medium, that is executed by a processor. The computer-readable medium may be any type of recording device in which data is stored in a computer-readable manner.
Examples of the computer-readable medium include a ROM, a RAM, a CD-ROM, a magnetic tape, a floppy disc, an optical data storage, and a carrier wave (e.g., data transmission through the Internet). The computer-readable medium can be distributed over a plurality of computer systems connected to a network so that computer-readable code is written thereto and executed therefrom in a decentralized manner. Functional programs, code, and code segments for implementing the example embodiments of the present disclosure can be deduced by programmers in the technical field to which the present disclosure pertains.
The present disclosure has been described herein with regard to various example embodiments. However, it will be obvious to those skilled in the art that various changes and modifications can be made without changing the technical conception and features of the present disclosure. Thus, it is clear that the above-described embodiments are illustrative in all aspects and are not intended to limit the present disclosure.
| Number | Date | Country | Kind |
|---|---|---|---|
| 10-2020-0072334 | Jun 2020 | KR | national |
