Patent Application
20190053715
The present disclosure relates to a biological information extracting device and a biological information extracting system for extracting biological information from information obtained without coming into contact with a human body.
Regarding measurement of a pulse of a person, there are a method in which a measuring person (nurse or the like) puts his/her finger on a wrist of a subject to personally check pulsation, a method in which a dedicated measuring instrument is attached to a wrist, a finger, or the like of a subject to automatically measure pulsation, and the like. On the other hand, in such a measuring method, since free movement of the subject is temporarily restricted or it is necessary to attach the measuring instrument to the subject, a technique for estimating (measuring) a pulse without coming into contact with the subject (human body) has been developed. For example, a technique for extracting a pulse wave signal that is time series data of a pulse wave of the subject from image data obtained by capturing the subject and estimating (measuring) the pulse based on the pulse wave signal is known (see PTL 1).
In addition, for example, at a construction site, a building site, or the like, in order to prevent occurrence of accident caused by a health condition (for example, drowsiness, stress, or poor condition) of a worker, it is preferable that the biological information (for example, the pulse wave or the pulse) of the worker is obtained and the health condition of the worker is monitored and managed based on the biological information. Therefore, a technique for determining a health condition by capturing a front surface of a face of the subject by a camera installed in a flange of a front surface of a helmet mounted on a head of the subject (measurement target person), wirelessly transmitting the image data (video information) to a management computer, and extracting the biological information of the subject based on the image data in the management computer is proposed (see PTL 2).
However, when the image data obtained by capturing the front surface of the face of the subject is created and transmitted, in a case where a trouble such as erroneous transmission or information leakage occurs, a privacy of the subject cannot be protected. In addition, in a case where the subject wears a shielding object such as sunglasses or a mask on the face, the biological information cannot be accurately extracted based on the image obtained by capturing the front surface of the face of the subject, and in some cases, the biological information cannot be extracted.
Furthermore, since the image data has a large data amount, in order to secure a large amount of a communication band or to reduce a transmission data amount, it is necessary to compress the image data or reduce an image size by using a known image compression technique such as H. 264. However, in a case where the image data is compressed or the image size is reduced, extraction accuracy of the biological information based on the image data is lowered.
An object of the present disclosure is to protect a privacy of a subject, to extract biological information of the subject even in a case where the subject wears a shielding object such as sunglasses or a mask on a face, and to reduce a transmission data amount without lowering extraction accuracy of the biological information.
PTL 1: Japanese Patent Unexamined Publication No. 2012-239661
PTL 2: International Publication No. 2015/098977
A biological information extracting device of the present disclosure which is provided at a head-mounted member mounted on a head of a subject and extracts biological information of the subject from information obtained without coming into contact with the subject including: a camera that captures temporally successive images including at least a part of a face of the subject as an object; a region selector that selects a skin color region from the image captured by the camera; and a data transmitter that transmits biomaterial-related data relating to a biomaterial of the subject extracted based on the skin color region to a predetermined transmission destination. The camera is disposed to face a side surface of the face of the subject.
According to the present disclosure, it is possible to protect a privacy of a subject, to extract biological information of the subject even in a case where the subject wears a shielding object such as sunglasses or a mask on the face, and to reduce a transmission data amount without lowering extraction accuracy of the biological information.
Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. However, detailed explanation may be omitted more than necessary. For example, there are cases where detailed explanation of well-known matters and redundant explanation on substantially the same configuration is omitted. This is to avoid unnecessary redundancy of the following description and to facilitate understanding by those skilled in the art. The accompanying drawings and the following description are provided to enable those skilled in the art to fully understand the present disclosure, and are not intended to limit the subject described in claims.
Biological information extracting system 1 is provided for extracting biological information (for example, a pulse wave or a pulse) from information (captured image) obtained without coming into contact with a human body, and as illustrated in
In the embodiment, person H is a worker at a construction site, a building site, or the like and head-mounted member 2 is a helmet. Hereinafter, head-mounted member 2 is also referred to as “helmet”. In the example of
As illustrated in
Biological information extracting device 3 has a thin and substantially rectangular housing, and accommodates therein hardware for executing a function of each unit which is described later. A lens of camera 11 is exposed on a side surface of the housing. Upper end portion 3a of biological information extracting device 3 is fixedly attached to a portion on the side of the face of person H at flange 2a of helmet 2 by using a known attachment mechanism (not illustrated). Therefore, camera 11 is disposed toward the side surface of the face of person H. In the example of
Camera 11 is a camera having a known configuration and includes an image pickup element for converting light of a formed image into an electric signal by forming an image of light from the object (person H) obtained through the lens in an image sensor (CCD, CMOS, or the like) which is not illustrated. As described above, since camera 11 is disposed to face the side surface of the face of person H, camera 11 captures an image of the side surface of the face of person H. In addition, image capturing region R of camera 11 is set not to an entire side of the side surface of the face of person H but to a part of the side surface of the face. Therefore, image data capable of identifying the face of person H is not created.
In the embodiment, the placing position of camera 11 is set so as to capture an image of a cheek of the face of person H. That is, image capturing region R of camera 11 becomes the cheek of the face. As will be described in detail later, in a biological information acquisition technique used in the present disclosure, pulse wave signal W as the time series data is extracted based on a captured image of the skin of person H captured by camera 11. Pulse wave signal W is a signal extracted based on a change in a light absorption amount (specifically, a hemoglobin concentration in the blood) caused by a change in a volume of the blood vessel due to contraction of the heart. Therefore, it is preferable that image capturing region R is a thin portion of the skin so that capturing of the image of the skin of person H is easy and the change in the light absorption amount can be accurately extracted. The cheek of the face is flat, an area thereof is wide, the skin is thin, so that the cheek of the face is optimal as image capturing region R for extracting pulse wave signal W. Pulse wave signal W of person H can be accurately extracted by setting the cheek of the face as image capturing region R.
In a case where region extractor 12 obtains a representative value representing the color (or luminance) of a pixel block for each pixel block of a predetermined size in a captured image (frame image), and the representative value coincides with a predetermined skin color value, the pixel block or a region within a predetermined range from the pixel block is extracted as the skin color region. The skin color value is automatically set based on a representative value of a skin color candidate region measured by using a known image recognition technique from the captured image. Moreover, the skin color value varies depending on each individual and a light condition when an image is captured, so that the skin color value may be set when person H wears helmet 2 and at each predetermined time interval.
Pulse wave measurer 13 calculates, for example, pixel values (0-255 gradations) of each component of RGB regarding each pixel configuring the skin color region extracted in the temporarily continuous captured image, and generates time series data of a representative value (here, an average value of respective pixels) as pulse wave signal W. In this case, it is possible to generate pulse wave signal W that is the time series data based on a pixel value of only a green component (G) of which variation is particularly large due to pulsation. Pulse wave signal W is wave form data (see
Data transmitter 14 can transmit the biomaterial-related data (in the embodiment, pulse wave signal W generated by pulse wave measurer 13) regarding a biomaterial of person H extracted based on the skin color region by using a known wireless communication technique such as a wireless LAN or a Bluetooth (registered trademark) to biological information analyzing device 4. Moreover, the wireless communication technique used in data transmitter 15 is not particularly limited and various known wireless communication techniques can be used.
As illustrated in
Similar to data transmitter 15 of biological information extracting device 3, data receiver 21 can receive the biomaterial-related data transmitted from data transmitter 15 by using a known wireless communication technique such as a wireless LAN or a Bluetooth (registered trademark). Moreover, the wireless communication technique used by data receiver 21 is not particularly limited as long as the biomaterial-related data transmitted from data transmitter 15 can be received.
For example, as illustrated in
An estimated result (heart rate) in pulse estimator 22 is stored in data storage unit 23 as pulse data. In addition, various information (for example, a wave form of the pulse wave or the like) including the estimated result by pulse estimator 22 is output and displayed on a display device (known display device) which is not illustrated.
Next, flows of processes by biological information extracting device 3 and biological information analyzing device 4 according to the first embodiment are respectively described with reference to flowcharts of
In addition, in
As illustrated in
As described above, according to biological information extracting system 1 according to the first embodiment, a configuration, in which the side surface of the face of person H is captured by camera 11 disposed toward the side surface of the face of person H, is provided, so that even in a case where person H wears the shielding object such as the sunglasses or the mask on the face, it is possible to extract the biological information of person H. In addition, a configuration, in which an image of a part (in the embodiment, the cheek of the face) of the side surface of the face of person H is captured by camera 11, is provided, so that image data capable of identifying the face of person H is not created. Therefore, even in a case where troubles such as erroneous transmission of the image data and information leakage occur, it is possible to protect the privacy of person H. Furthermore, a configuration, in which pulse wave signal W is generated in biological information extracting device 3 and is transmitted to biological information analyzing device 4, is provided, so that it is possible to greatly reduce the transmission data amount without lowering the extraction accuracy of the biological information compared to a case where the image data is transmitted.
In the embodiment, image capturing region R of camera 11 is the cheek of the face, but image capturing region R may be a portion other than the cheek of the face as long as image capturing region R is a part of the side surface of the face of person H. For example, region R1 above mask 52 on the face of person H as illustrated in
In addition, in the first embodiment, in order to reduce a processing amount in biological information extracting device 3, pulse wave signal W before executing the filtering process is transmitted from biological information extracting device 3 to biological information analyzing device 4. However, the pulse wave signal (that is, the pulse wave signal from which noise component is removed) after executing the filtering process may be transmitted. Also in this case, it is possible to greatly reduce the transmission data amount compared to a case where the image data is transmitted. Moreover, in this case, the filtering process of pulse wave signal W is executed in biological information extracting device 3.
As illustrated in
According to biological information extracting system 1 according to the second embodiment, pulse data M is generated in biological information extracting device 3 and pulse data M is transmitted to biological information analyzing device 4, so that it is possible to greatly reduce a transmission data amount without lowering the extraction accuracy of the biological information compared to a case where the image data is transmitted. In addition, since it is possible to transmit pulse data M which is the biological information from biological information extracting device 3, a degree of freedom of the configuration of biological information extracting system 1 is enhanced. Therefore, it is possible to construct biological information extracting system 1 having various configurations. In addition, a configuration, in which an image of a part (in the embodiment, the cheek of the face) of the side surface of the face of person H is captured by camera 11 disposed toward the side surface of the face of person H, is provided, so that similar to the case of the first embodiment which is described above, it is possible to protect the privacy of person H and extract biological information of person H even in a case where person H wears a shielding object such as sunglasses or a mask on the face.
As illustrated in
According to biological information extracting system 1 according to the third embodiment, image data P that is the time series image data of the skin color region is generated in biological information extracting device 3 and pulse data P is transmitted to biological information analyzing device 4, so that it is possible to greatly reduce a transmission data amount without lowering the extraction accuracy of the biological information compared to a case where the whole face image is transmitted. In addition, a configuration, in which an image of a part (in the embodiment, the cheek of the face) of the side surface of the face of person H is captured by camera 11 disposed toward the side surface of the face of person H, is provided, so that similar to the case of the first embodiment which is described above, it is possible to protect the privacy of person H and extract biological information of person H even in a case where person H wears a shielding object such as sunglasses or a mask on the face.
In
Biological information extracting system 1 according to the fourth embodiment is different from that of the case of the first embodiment in that headset 2 is used instead of helmet 2 as the head-mounted member. Headset 2 includes band portion 61 mounted on the head of person H, headphone 62 that is disposed at lower end 61a of band portion 61, arm 63 that extends from lower end 61a of band portion 61, and microphone 64 that is disposed at tip 63a of arm 63. Biological information extracting device 3 is fixedly attached to a portion positioned toward the cheek of the face of person H in arm 63 by using a known attachment mechanism (not illustrated) such as a clip. The placing position of camera 11 is set so as to capture an image of the cheek of the face of person H. That is, image capturing region R of camera 11 becomes the cheek of the face.
Biological information extracting system 1 according to the fourth embodiment can be applied to a telephone operator of a call center or the like. According to biological information extracting system 1 according to the fourth embodiment, a configuration, in which an image of a part (in the embodiment, the cheek of the face) of the side surface of the face of person H is captured by camera 11 disposed toward the side surface of the face of person H, is provided, so that similar to the case of the first embodiment which is described above, it is possible to protect the privacy of person H and extract biological information of person H even in a case where person H wears a shielding object such as sunglasses or a mask on the face.
Although the present disclosure is described based on specific embodiments, these embodiments are merely examples, and the present disclosure is not limited by these embodiments. For example, the biological information extracting device and the biological information extracting system according to the present disclosure can be applied not only to a medical application but also to various uses such as face recognition, monitoring, life and death confirmation, and the like. Each configuration element of the biological information extracting device and the biological information extracting system is not necessarily essential, and can be appropriately selected at least as long as the element does not depart from the scope of the present disclosure.
The biological information extracting device and the biological information extracting system according to the present disclosure are useful as a biological information extracting device, a biological information extracting system, and the like capable of protecting the privacy of the subject, extracting the biological information of the subject even in a case where the subject wears the shielding object such as sunglasses or a mask on the face, and reducing the transmission data amount without lowering the extraction accuracy of the biological information.
| Number | Date | Country | Kind |
|---|---|---|---|
| 2016-040890 | Mar 2016 | JP | national |
| Filing Document | Filing Date | Country | Kind |
|---|---|---|---|
| PCT/JP2017/005214 | 2/14/2017 | WO | 00 |
