Patent Application
20240237932
The present disclosure relates to a physiological information measurement system that measures brain activities.
In order to provide goods and services in consideration of feelings of individuals, a system that can analyze information on feelings of a subject has recently been proposed. For example, PTL 1 discloses an apparatus that automatically obtains evaluation of a sample (wine) by shooting moving images of a subject who tries the sample with the use of a camera provided in a wine dispenser where the sample is contained and analyzing the subject's facial expression from the moving images to estimate the subject's feeling.
In order to estimate information on sensibility of a subject, a plurality of types of sensors are preferably used. For example, as described in Japanese Patent Laying-Open No. 2019-159941, in addition to a camera that shoots a subject's face, another sensor that obtains physiological information of the subject may be used. In this literature, a face of a worker on production lines of a factory is shot with a camera, an electroencephalograph is attached to a head of the worker, and a degree of concentration of the worker is estimated from a shot image and outputs from the electroencephalograph.
In general, ease in attachment to a subject, of a sensor that measures brain activities is an issue. There is a room for improvement in a method of efficiently obtaining physiological information of a subject with a plurality of types of sensors including such a sensor as measuring brain activities.
The present disclosure was made to solve such a problem, and provides a physiological information measurement system that achieves improved ease in attachment to a subject, of a plurality of types of sensors including a sensor that measures brain activities.
A physiological information measurement system in the present disclosure includes a base attachable to and removable from a head of a subject, a plurality of sensors held by the base, and a signal processing unit that synchronizes signals outputted from the plurality of sensors. The plurality of sensors include a first sensor that measures brain activities and a second sensor that obtains an image of a face of the subject.
According to the physiological information measurement system, a plurality of sensors are held by the base removably fixed to the head of the subject, so that ease in attachment to the subject is improved. Furthermore, since the second sensor can obtain an image of the facial expression of the subject, the image of the facial expression of the subject can be obtained without being affected by a motion of the subject.
The present embodiment will be described in detail below with reference to the drawings. The same or corresponding elements in the drawings have the same reference characters allotted and description thereof will not be repeated.
A head-mounted measurement system that measures brain activities will be described below as an exemplary physiological information measurement system.
The fixing portion shown in
A sensor that measures brain activities is attached to head-mounted device 1. In order to correctly measure brain activities, the sensor should be arranged at a prescribed position of the head of the subject. Therefore, for arrangement of the sensor at the prescribed position of the head of the subject at the time of attachment of head-mounted device 1 to the head of the subject, an alignment portion 14 is provided in vertex band 11. By bringing alignment portion 14 in alignment with a vertex point of the subject, the sensor is arranged at the prescribed position of the head of the subject. By providing alignment portion 14, anybody including the subject himself/herself can easily attach head-mounted device 1 to a correct position.
A sensor (first sensor) that measures brain activities and an ultra-wide-angle camera 31 (second sensor) which is an image sensor that obtains an image of the face of the subject are attached to housing 10. Ultra-wide-angle camera 31 is arranged at a position out of a field of view of the subject. For example, housing 10 is provided with a protruding portion that protrudes forward from a frontal portion of the subject, and as ultra-wide-angle camera 31 is attached to the protruding portion, it can be arranged at a position outside the field of view of the subject. By thus arranging ultra-wide-angle camera 31 out of the field of view of the subject, ultra-wide-angle camera 31 does not block a line of sight of the subject. Examples of the sensor that measures brain activities include an NIRS probe 21 that measures a cerebral blood flow with near-infrared spectroscopy (NIRS), a brain wave sensor 22 (EEG), and a magnetoencephalograph (MEG). Though NIRS probe 21 and brain wave sensor 22 are attached to housing 10 shown in
NIRS probe 21 is a sensor that non-invasively measures variation in amount of a cerebral blood flow with the use of near infrared light having such a property as passing through a skin tissue or a bone tissue but being absorbed by oxyhemoglobin and deoxyhemoglobin in blood. NIRS probe 21 detects signal light by emitting near infrared light from a plurality of light transmission portions (light transmission fibers) arranged at prescribed intervals toward a scalp of the subject and receiving light diffused and reflected by the brain (cerebral cortex) with a plurality of light reception portions (light reception fibers) on the scalp. A concentration of oxyhemoglobin and a concentration of deoxyhemoglobin at a measurement site located between the light transmission portions and the light reception portions are found based on this detection light. Brain activities are then measured based on variation in concentration of hemoglobin.
In head-mounted device 1 shown in
Head-mounted device 1 shown in
Brain wave sensor 22 is a sensor that obtains a weak current generated by activities in the brain with an electrode attached to a skull, amplifies the current, and measures the current as brain waves. A shape or a configuration of brain wave sensor 22 is not particularly limited so long as the brain wave sensor is capable of measuring brain waves. Housing 10 may include a mechanism that allows change in position where brain wave sensor 22 is to be held by housing 10 in accordance with a brain activity to be measured.
Not only ultra-wide-angle camera 31 but also other sensors such as a temperature sensor 32 capable of measuring a body temperature of the subject and an acceleration sensor 33 that measures a motion of the head of the subject may be attached to housing 10 as the second sensor. Other sensors may include any sensor so long as the sensor measures a target different from the target of the sensor that measures brain activities, and a sensor that detects a motion of the subject, a sensor that measures a heartbeat, and a sensor that measures respiration may be applicable. Though temperature sensor 32 and acceleration sensor 33 other than ultra-wide-angle camera 31 are attached to housing 10 in head-mounted device 1 shown in
Other than NIRS probe 21, brain wave sensor 22, ultra-wide-angle camera 31, temperature sensor 32, and acceleration sensor 33, housing 10 is provided with a control unit that controls these sensors.
Control unit 16 includes a processor such as a central processing unit (CPU) and a memory such as a read only memory (ROM) and a random access memory (RAM). A control program is stored in the memory. As the processor executes the control program, it controls such sensors as NIRS probe 21 and ultra-wide-angle camera 31.
Control unit 16 performs signal processing for synchronizing a signal (first signal) resulting from measurement by NIRS probe 21 with a signal (second signal) resulting from image obtainment by ultra-wide-angle camera 31. In other words, control unit 16 corresponds to a signal processing unit that performs signal processing for synchronizing the signal (first signal) resulting from measurement by NIRS probe 21 with the signal (second signal) resulting from image obtainment by ultra-wide-angle camera 31, and it is provided in housing 10. Control unit 16 synchronizes the signal resulting from measurement by NIRS probe 21 with the signal resulting from image obtainment by ultra-wide-angle camera 31 based on time stamps of these signals. Signal processing for synchronizing the signals is not limited to processing based on the time stamp of the signal. Control unit 16 may generate a synchronization signal and supply the synchronization signal to sensors to synchronize signals from the sensors with each other.
Battery 17 may be, for example, a secondary battery such as a nickel metal hydride battery or a lithium ion secondary battery or a primary battery such as a dry cell. Naturally, head-mounted device 1 may be provided with no battery 17, or may be driven by a commercial power supply together with battery 17.
Communication unit 18 is a communication interface, and can communicate with external equipment 500 by establishing wireless or wired communication. Therefore, external equipment 500 can receive through communication unit 18, the signal (first signal) resulting from measurement by NIRS probe 21 and the signal (second signal) resulting from image obtainment by ultra-wide-angle camera 31 as signals synchronized by control unit 16. Since the signal resulting from image obtainment by ultra-wide-angle camera 31 includes a signal resulting at least from image obtainment of the facial expression of the subject, external equipment 500 can obtain the facial expression of the subject and the signal resulting from measurement of brain activities at that time (the signal resulting from measurement by NIRS probe 21 and the signal resulting from measurement by brain wave sensor 22). Therefore, a researcher can study relation between delight, anger, sorrow and pleasure emotions of the subject and the brain activities based on the signals obtained by external equipment 500. In addition, the researcher can know a state (for example, sleeping, being nervous, or the like) of the subject at the time point of measurement of the brain activities based on the signals obtained by external equipment 500 and then study the brain activities.
The signal resulting from image obtainment by ultra-wide-angle camera 31 is described as including the signal resulting at least from image obtainment of the facial expression of the subject. In other words, a range of shooting by ultra-wide-angle camera 31 includes the subject's face without fail.
Since ultra-wide-angle camera 31 is attached as facing the outside of the subject in
As a result of synchronization between the signal resulting from image obtainment of the operation performed by the subject and the signal resulting from measurement of brain activities (the signal resulting from measurement by NIRS probe 21 and the signal resulting from measurement by brain wave sensor 22), external equipment 500 can obtain the operation performed by the subject and the signal resulting from measurement of brain activities at that time. Therefore, the researcher can study about relation between an action of the subject and the brain activities based on the signals obtained by external equipment 500. In addition, the researcher can know a behavior (for example, wiggling a leg or the like) of the subject at the time point of measurement of the brain activities based on the signals obtained by external equipment 500 and then study about the brain activities.
As a result of synchronization between the signal resulting from image obtainment of the environment around the subject and the signal resulting from measurement of brain activities (the signal resulting from measurement by NIRS probe 21 and the signal resulting from measurement by brain wave sensor 22), external equipment 500 can obtain the environment around the subject and the signal resulting from measurement of brain activities at that time. Therefore, the researcher can study about relation between the environment (for example, a facial expression of a person with whom the subject talks) around the subject and the brain activities based on the signals obtained by external equipment 500. In addition, the researcher can know the environment (for example, playing with a pet) around the subject at the time point of measurement of the brain activities based on the signals obtained by external equipment 500 and then study about the brain activities.
Though ultra-wide-angle camera 31 is described as being used as the image sensor, an image sensor to be attached to head-mounted device 1 is not limited to ultra-wide-angle camera 31. Head-mounted device 1 may be provided, for example, with two image sensors of an image sensor that obtains an image of a facial expression of the subject and an image sensor that obtains an image of an environment around the subject. Alternatively, head-mounted device 1 may include a mechanism capable of moving a single image sensor to a position for image obtainment of a facial expression of the subject or to a position for image obtainment of an environment around the subject to have the image sensor obtain an image.
In the description above, control unit 16 is described as performing processing for synchronizing the signal (first signal) resulting from measurement by NIRS probe 21 with the signal (second signal) resulting from image obtainment by ultra-wide-angle camera 31. Control unit 16, however, may not only synchronize the signal (first signal) resulting from measurement by NIRS probe 21 with the signal (second signal) resulting from image obtainment by ultra-wide-angle camera 31 but also may generate a signal associated with cognition or sensibility obtained from the synchronized signals and transmit the signal to external equipment 500 through communication unit 18.
In the description above, control unit 16 is described as performing processing for synchronizing the signal (first signal) resulting from measurement by NIRS probe 21 with the signal (second signal) resulting from image obtainment by ultra-wide-angle camera 31. Control unit 16, however, may perform processing for synchronizing a signal (second signal) resulting from measurement by temperature sensor 32 or acceleration sensor 33 other than the signal resulting from image obtainment by ultra-wide-angle camera 31 with the signal (first signal) resulting from measurement by NIRS probe 21. External equipment 500 can thus obtain the signal resulting from measurement by temperature sensor 32 or acceleration sensor 33 and the signal resulting from measurement of brain activities (the signal resulting from measurement by NIRS probe 21 and the signal resulting from measurement by brain wave sensor 22) at that time, rather than the signal resulting from image obtainment by ultra-wide-angle camera 31.
Head-mounted measurement system 100 according to the first embodiment is described as performing processing for synchronizing the signal (first signal) resulting from measurement by NIRS probe 21 with the signal (second signal) resulting from image obtainment by ultra-wide-angle camera 31 by means of control unit 16 provided in housing 10 of head-mounted device 1. In a head-mounted measurement system according to a second embodiment, processing for synchronizing the signal (first signal) resulting from measurement by the NIRS probe with the signal (second signal) resulting from image obtainment by the ultra-wide-angle camera is performed in an apparatus different from the housing of the head-mounted device.
synchronizing the signal (first signal) resulting from measurement by NIRS probe 21 with the signal (second signal) resulting from image obtainment by ultra-wide-angle camera 31 is not performed by control unit 16 but the signals are merely transmitted to signal processing apparatus 2 through communication unit 18.
Signal processing apparatus 2 performs processing for synchronizing the signal (first signal) resulting from measurement by NIRS probe 21 with the signal (second signal) resulting from image obtainment by ultra-wide-angle camera 31 that are received from head-mounted device 1. In other words, signal processing apparatus 2 corresponds to the signal processing unit that performs signal processing for synchronizing the signal (first signal) resulting from measurement by NIRS probe 21 with the signal (second signal) resulting from image obtainment by ultra-wide-angle camera 31, and it is provided in an apparatus different from housing 10. Signal processing apparatus 2 synchronizes the signal resulting from measurement by NIRS probe 21 with the signal resulting from image obtainment by ultra-wide-angle camera 31 based on time stamps of these signals. Signal processing for synchronizing the signals is not limited to processing based on the time stamp of the signal.
Signal processing apparatus 2 may not only synchronize the signal (first signal) resulting from measurement by NIRS probe 21 with the signal (second signal) resulting from image obtainment by ultra-wide-angle camera 31 but also may generate a signal associated with cognition or sensibility obtained from the synchronized signals.
Signal processing apparatus 2 may perform processing for synchronizing the signal (second signal) resulting from measurement by temperature sensor 32 or acceleration sensor 33 other than the signal resulting from image obtainment by ultra-wide-angle camera 31 with the signal (first signal) resulting from measurement by NIRS probe 21. Signal processing apparatus 2 can thus obtain the signal resulting from measurement by temperature sensor 32 or acceleration sensor 33 and the signal resulting from measurement of brain activities (the signal resulting from measurement by NIRS probe 21 and the signal resulting from brain wave sensor 22) at that time, rather than the signal resulting from image obtainment by ultra-wide-angle camera 31.
Processor 202 is implemented by a CPU or a GPU, and can read a program (by way of example, an OS 2102 and a program 2104) stored in storage 210, develop the program on main memory 204, and execute the program. Processor 202 executes program 2104 for synchronizing a signal from head-mounted device 1 which is received by input unit 206.
Main memory 204 is implemented by a volatile storage device such as a DRAM or an SRAM. Storage 210 is implemented, for example, by a non-volatile storage device such as an HDD or an SSD.
In addition to OS 2102 for performing a basic function, program 2104 for providing a function as signal processing apparatus 2 is stored in storage 210. In other words, as program 2104 is executed by processor 202 of signal processing apparatus 2, processing for synchronizing signals or generating a signal associated with cognition or sensibility obtained from the synchronized signals is performed.
Input unit 206 includes an input interface connected to head-mounted device 1 wirelessly or through a wire to receive a signal from head-mounted device 1. Input unit 206 can be implemented by a keyboard or a mouse and can further receive an operation from a measurer.
Output unit 208 includes an output interface that outputs to external equipment through communication or USB connection, a signal synchronized by processor 202 or a signal associated with cognition or sensibility which is generated from synchronized signals. Output unit 208 can output a result of processing by processor 202 to a display, various indicators, or a printer connected to signal processing apparatus 2, and can further output a signal to an earphone-microphone 300 for giving an auditory stimulus to the subject or output a signal to display 400 for giving a visual stimulus to the subject.
Signal processing apparatus 2 includes external drive 212, which allows reading of a program stored in a recording medium (for example, an optical recording medium such as a digital versatile disc (DVD) or a USB memory) in which a computer readable program is stored and installation of the program in storage 210 or the like. Communication controller 220 exchanges a signal with another apparatus or the like through wired or wireless communication. Signal processing apparatus 2 may exchange with head-mounted device 1 through communication controller 220, synchronized signals or a signal associated with cognition or sensibility generated from the synchronized signals.
Though program 2104 or the like executed by signal processing apparatus 2 may be installed through a computer readable recording medium, it may be installed by being downloaded from a server apparatus on a network. A function provided by signal processing apparatus 2 may be implemented by using a part of a module provided by the OS.
Signal processing apparatus 2 may present an audio stimulus to the subject with the use of earphone-microphone 300 (an audio device) shown in
Signal processing apparatus 2 may present a visual stimulus to the subject with the use of display 400 (a display device) shown in
Earphone-microphone 300 may be used as a voice collector that collects oral communication between the subject and the measurer. Signal processing apparatus 2 can synchronize the oral communication between the subject and the measurer collected by earphone-microphone 300, the signal (first signal) resulting from measurement by NIRS probe 21, and the signal (second signal) resulting from image obtainment by ultra-wide-angle camera 31 with one another. Signal processing apparatus 2 can obtain the facial expression of the subject at the time of conversation with the subject and the signal resulting from measurement of brain activities. Therefore, the researcher can study about relation between contents of the conversation with the subject and the brain activities based on the signals obtained by signal processing apparatus 2.
Signal processing apparatus 2 is described as synchronizing the timing of presentation of the audio stimulus or the visual stimulus, the signal (first signal) resulting from measurement by NIRS probe 21, and the signal (second signal) resulting from image obtainment by ultra-wide-angle camera 31 with one another. Without being limited as such, as described in the first embodiment, control unit 16 provided in housing 10 of head-mounted device 1 may synchronize the timing of presentation of the audio stimulus or the visual stimulus, the signal (first signal) resulting from measurement by NIRS probe 21, and the signal (second signal) resulting from image obtainment by ultra-wide-angle camera 31 with one another.
The embodiments described above are understood by a person skilled in the art as specific examples of aspects below.
A physiological information measurement system according to one aspect includes a base attachable to and removable from a head of a subject, a plurality of sensors held by the base, and a signal processing unit that synchronizes signals outputted from the plurality of sensors. The plurality of sensors include a first sensor that measures brain activities and a second sensor that obtain an image of a face of the subject.
According to the physiological information measurement system described in Clause 1, as the plurality of sensors are held by the base removably fixed to the head of the subject, ease in attachment to the subject is improved. Furthermore, since the second sensor can obtain the image of the facial expression of the subject, the image of the facial expression of the subject can be obtained without being affected by a motion of the subject.
In the physiological information measurement system described in Clause 2, the second sensor is arranged at a position out of a field of view of the subject.
According to the physiological information measurement system described in Clause 2, the second sensor does not block the line of sight of the subject.
In the physiological information measurement system described in Clause 3, the base includes a protruding portion that protrudes forward from a frontal portion of the subject, and the second sensor is attached to the protruding portion.
According to the physiological information measurement system described in Clause 3, the second sensor can be arranged at a position where it does not block the line of sight of the subject.
The physiological information measurement system described in Clause 4 includes a fixing portion that removably fixes the base to the head of the subject. The fixing portion includes a vertex band for prevention of slip-off, a head band that fixes a position of the base, and a size adjustment portion that allows adjustment of a length of at least the head band in accordance with a size of the head.
According to the physiological information measurement system described in Clause 4, the base can reliably be fixed to the head of the subject without slipping from the head of the subject.
In the physiological information measurement system described in Clause 5, the vertex band includes an alignment portion aligned with a vertex point.
According to the physiological information measurement system described in Clause 5, the base can easily be attached such that the first sensor and the second sensor are arranged at prescribed positions (correct positions) in the head of the subject.
In the physiological information measurement system described in Clause 6, the base includes a mechanism that changes a position where the first sensor is held by the base in accordance with a brain activity to be measured.
According to the physiological information measurement system described in Clause 6, the first sensor can be arranged at a position in accordance with a brain activity of interest so as to measure the brain activity.
In the physiological information measurement system described in Clause 7, the signal processing unit is provided in the base.
According to the physiological information measurement system described in Clause 7, since the signal processing unit provided in the base can synchronize the first signal resulting from measurement by the first sensor with the second signal resulting from measurement by the second sensor, no other apparatus is required.
In the physiological information measurement system described in Clause 8, the signal processing unit is provided in an apparatus different from the base.
According to the physiological information measurement system described in Clause 8, since an apparatus different from the base can synchronize the first signal resulting from measurement by the first sensor with the second signal resulting from measurement by the second sensor, the base can be lighter in weight.
In the physiological information measurement system described in Clause 9, the signal processing unit synchronizes a first signal from the first sensor and a second signal from the second sensor with each other based on a time stamp of the first signal and a time stamp of the second signal.
According to the physiological information measurement system described in Clause 9, the first signal and the second signal can easily be synchronized with each other.
In the physiological information measurement system described in Clause 10, the first sensor is a sensor that measures brain waves or a sensor that measures a cerebral blood flow, the second sensor is an image apparatus, and the physiological information measurement system further includes at least one sensor selected from the group consisting of a sensor that detects a motion of the subject, a sensor that measures a heartbeat, a sensor that measures a body temperature, and a sensor that measures respiration.
According to the physiological information measurement system described in Clause 10, various types of sensors can be adopted as the first sensor and the second sensor.
In the physiological information measurement system described in Clause 11, the image apparatus is configured to obtain image of at least one of an operation performed by the subject and an environment around the subject as combined with a facial expression of the subject.
According to the physiological information measurement system described in Clause 11, a signal resulting from shooting of an operation performed by the subject or an environment around the subject other than the facial expression of the subject can be used.
The physiological information measurement system described in Clause 12 further includes a voice collector that collects oral communication between the subject and a measurer.
According to the physiological information measurement system described in Clause 12, oral communication between the subject and the measurer collected by the voice collector can be used for measurement of brain activities.
The physiological information measurement system described in Clause 13 further includes an audio device that presents an audio stimulus to the subject.
According to the physiological information measurement system described in Clause 13, brain activities at the time of presentation of the audio stimulus to the subject can be measured.
The physiological information measurement system described in Clause 14 further includes a display device that presents a visual stimulus to the subject.
According to the physiological information measurement system described in Clause 14, brain activities at the time of presentation of the visual stimulus to the subject can be measured.
The physiological information measurement system described in Clause 15 further includes a communication unit that communicates data with external equipment.
According to the physiological information measurement system described in Clause 15, the first signal and the second signal in synchronization with external equipment can be transmitted.
A physiological information measurement system according to one aspect includes a base attachable to and removable from a head of a subject, a plurality of sensors held by the base, and a signal processing unit that processes signals outputted from the plurality of sensors. The plurality of sensors include a brain sensor that measures brain activities.
According to the physiological information measurement system described in Clause 16, a time period for attachment of a plurality of sensors to the subject and preparation for measurement can be reduced.
It should be understood that the embodiments disclosed herein are illustrative and non-restrictive in every respect. The scope of the present invention is defined by the terms of the claims rather than the description of the embodiments above and is intended to include any modifications within the scope and meaning equivalent to the terms of the claims.
| Number | Date | Country | Kind |
|---|---|---|---|
| 2020-162869 | Sep 2020 | JP | national |
| Filing Document | Filing Date | Country | Kind |
|---|---|---|---|
| PCT/JP2021/033118 | 9/9/2021 | WO |
