Patent Application
20240423504
The present invention relates to a measurement processing terminal, a measurement processing method, and a computer program that measure a motion of a finger including a finger tapping motion or the like and process the measurement result.
With the aging of society, the number of patients with Alzheimer's dementia is increasing year by year. If patients are detected early, medication can slow the progression of the disease. Since it may be difficult to distinguish between symptoms associated with aging such as forgetfulness and the disease, there are many cases where patients go to a hospital only after the disease becomes severe.
In such a situation, as a screening test for early detection of Alzheimer's dementia, a blood test, a smell test, a test that reproduce doctor's interviews on a tablet terminal and the like have been conventionally performed. But there has been problems such as pain during blood sampling and long examination times, which places a heavy burden on the subject. On the other hand, as an examination with a small burden on the subject, cognitive function evaluation is also performed by button pressing or finger motion measurement of one hand using a tablet terminal, but there is a problem that sufficient examination accuracy cannot be obtained. If a simple screening examination can be performed with high accuracy with a small burden on the subject, it leads to early detection of Alzheimer's dementia, and can also contribute to improvement of the quality of life of the patient and reduction of medical and care costs.
On the other hand, in recent years, it has been revealed that a motion pattern peculiar to Alzheimer's dementia can be extracted from an opening/closing motion (finger tapping motion) of thumbs and index fingers of both hands, and it has been confirmed that there is a high correlation with the dementia examination by motion measurement of the fingers and general interview. These are said to be results of capturing a decrease in the rhythm motion function of fingers of both hands caused by atrophy of the brain in Alzheimer's dementia by finger tapping motion measurement. In addition, a finger is said to be a second brain, and many regions in the brain are related to the function of the finger, and the motion of the finger is not limited to Alzheimer's dementia, but is also said to be related to dementia such as cerebrovascular disease, Lewy body disease, Parkinson's disease, developmental coordination disorder (such as inability to skip or jump rope), and the like. That is, the state of the brain can be known from the tapping motion of the finger. Furthermore, since an ability of finger dexterity can be quantified by utilizing the tapping motion of the finger as a “measure” indicating the health condition of the brain, the tapping motion of the finger can be used in various fields such as healthcare, rehabilitation, and life support.
As a method for accurately measuring and evaluating a finger tapping motion, for example, Patent Literature 1 and Patent Literature 2 disclose a motion function evaluation system and a method including a motion function measurement apparatus that calculates motion data based on a relative distance between a pair of a transmission coil and a reception coil attached to a movable part of a living body, and an evaluation apparatus that evaluates a motion function of the living body based on motion data received from the motion function measurement apparatus. That is, these patent documents show that a state of brain function is known by converting a change in magnetic force fluctuated by tapping motion by two fingers into an electrical signal using a magnetic sensor attached to a fingertip, measuring and quantifying the motion, and capturing a feature amount indicating the feature of the finger motion.
In addition, in the rehabilitation field, Simple Test for Evaluating Hand Function (STEF) is also conducted to confirm the effect of postoperative rehabilitation of stroke and cerebral infarction patients by measuring the time required for a series of gripping and moving objects of different sizes and shapes with a stopwatch.
However, in a finger tapping apparatus using a magnetic sensor attached to a fingertip as disclosed in Patent Literature described above, since a motion of a joint of a finger cannot be recognized, a bending/stretching motion of the finger such as a “pinching” motion cannot be quantitatively evaluated. In addition, in a case where it is difficult to attach the sensor due to injury, deformation, or the like of the finger, measurement cannot be performed.
In addition, in the Simple Test for Evaluating Hand Function (STEF) described above, since a motion of an arm and fingers is performed by visual observation by a doctor, it is largely subject to a subjectivity of a measurer (doctor), and the examination results may vary depending on the measurer.
In addition, in order to objectively, finely and accurately evaluate a motor function of upper limb, it is necessary to perform an evaluation in synchronization with a motion of not only the finger but also other parts of a body such as the arm and the eye (For example, the interlocking property (relationship) of the motion of the finger and the arm and the interlocking property (relationship) of the motion of the finger and the eye are quantitatively evaluated.). However, in a method of visually confirming the motion as in a conventional examination method, it is difficult for the measurer to simultaneously evaluate the motion of different parts.
Furthermore, in order to accurately grasp the degree of recovery of the motor function of upper limb, or in order to prevent variations in an examination and measurement, it is also necessary to unify the examination and measurement environment to make the examination and measurement conditions uniform. Depending on the examination and measurement method, if such examination and measurement conditions (or environment) are not unified, the examination and measurement results vary among subjects, and accurate examination and measurement may not be performed.
The present invention has been made in view of the above circumstances, and an object of the present invention is to provide a measurement processing terminal, a method, and a computer program capable of not only quantitatively evaluating a bending/stretching motion function of a finger and/or an opening/closing motion function of two fingers by capturing a motion of a finger joint, but also objectively and finely evaluating the motor function of upper limb with high accuracy.
In order to solve the above problems, the present invention is a measurement processing terminal that measures a motion of a finger of a subject and processes a measurement result, the measurement processing terminal including: an imaging data collector that collects imaging data obtained by capturing the motion of the finger of the subject; a hand tracking data generator that implements a hand tracking function of detecting and tracking a position of the finger on the basis of the imaging data and generates chronological hand tracking data by the hand tracking function from the imaging data; and a data processor that processes the hand tracking data obtained from the hand tracking data generator and generates quantitative data regarding a bending/stretching motion of the finger and/or an opening/closing motion of two fingers accompanying the motion of the finger joint.
According to the above configuration of the present invention, the chronological hand tracking data can be generated from the imaging data obtained by capturing the motion of the finger of the subject by the hand tracking function, and the hand tracking data can be processed to generate quantitative data regarding the bending/stretching motion of the finger accompanying the motion of the finger joint. Therefore, it is possible to accurately capture the motion of the joint that cannot be recognized by a conventional magnetic sensor type apparatus, and quantitatively evaluate the bending/stretching motion of the finger such as a “pinching” motion. As a result, in evaluation of an ability of finger dexterity, more detailed analysis and evaluation can be performed by combining the information of distance between two fingers and the information of movement (joint angle) of each joint of the finger.
Note that the measurement processing terminal having such a function may have any form. For example, the measurement processing terminal may be configured as a small terminal such as a smartphone, may be in the form of a tablet-type thin computer, a personal computer, or the like, or may be in the form of a head mounted display (Head Mounted Display; HMD, hereinafter referred to as HMD) or the like.
Furthermore, in the above configuration, the data processor may calculate and analyze a feature amount leading to brain function evaluation of the subject. In addition, the data processor may evaluate (for example, by comparison with data of a healthy subject,) brain function and cognitive function of the subject from the calculated feature amount. Such assessment may be effective as an early stage screening to discriminate dementia and may aid in the detection of dementia. In addition, the application of the measurement processing terminal with such a data processor is not limited to the clinical field, and for example, the measurement processing terminal can also contribute to determination of judgment in driving a car, can be applied to a brain training-like game and the like, and has a wide application range.
Furthermore, in the above configuration, a movement distance measuring instrument that measures a chronological movement distance in which a hand moves with a motion of an arm, or a gaze detector that detects a line-of-sight of an eye of the subject may be further provided. In this case, the data processor preferably processes the hand tracking data obtained from the hand tracking data generator, distance data and time data obtained from the movement distance measuring instrument, and line-of-sight data obtained from the gaze detector, and generates correlation data obtained by quantifying the correlation between these data. According to this, for example, when the arm is moved to grip the object, the motion of the arm and the opening/closing motion of the finger can be quantitatively evaluated at the same time (interlocking property (relationship) between the motion of the finger and the motion of the arm can be quantitatively evaluated), and furthermore, the interlocking property (relationship) between the motion of the finger and the motion of the eye can be quantitatively evaluated. That is, the finger can be evaluated in synchronization with the motion of another part of the body, and thus the motor function of upper limb can be objectively, finely and accurately evaluated.
As the correlation data generated by the data processor, for example, graph data indicating the relationship between the movement distance of the hand from the measurement start position to the object to be gripped and the time together with the opening/closing timing of the finger can be exemplified. Such correlation data makes it possible to grasp the movement distance of the hand when the subject tries to open the finger, the distance from the measurement start position to the object, and the time required to grip the object. In addition, graph data indicating the relationship between the distance between two fingers and time can be cited as correlation data. Such correlation data makes it possible to grasp the opening/closing timing of the finger (the moment of gripping the object). Furthermore, examples of the correlation data include graph data indicating a relationship between a joint angle and time obtained from the hand tracking data and the relationship between the distance from the measurement start position and the distance between two fingers. The graph data indicating the relationship between the joint angle and time makes it possible to know the change in the joint angle with time, and the graph data indicating the relationship between the distance from the measurement start position and the distance between two fingers makes it possible to know how far the hand has moved from the measurement start position before the finger opening action is performed. Furthermore, in order to be able to evaluate the motion of the finger in synchronization with the motion of the eye detected by the eye tracking of the gaze detector, for example, correlation data that can display the deviation of the line-of-sight position of the eye with respect to the object to be gripped as a scatter diagram may be generated.
In the above configuration, the data processor may generate image data related to a measurement reference position and/or a measurement history. According to this, measurement conditions can be made uniform among a plurality of examinations and measurements, and the change in the motor function of a finger (upper limb) can be known among the plurality of examinations and measurements. Note that examples of the image data related to the measurement reference position include image data for marking the measurement start position (such as a position to place a hand) on a display screen, and image data for performing guide display indicating a contour of the finger that defines a direction and a position of the hand at the time of starting measurement on the display screen. Examples of the image data related to the measurement history include image data for displaying a past measurement result on the display screen with a dotted line, a contour of the hand, or the like.
In the above configuration, a condition setter that sets the measurement condition may be further provided, and in this case, the data processor preferably generates image data and/or audio data corresponding to the measurement condition set by the condition setter. According to this, by setting the measurement conditions, for example, measurement error (variation) can be reduced, examination and measurement environment can be unified, and the examination and measurement conditions can be made uniform (unified). As a result, problems unique to the case of using a photography data are solved (For example, in consideration of a parameter that can vary as the distance from the camera as an imaging unit to an imaging target changes, references are made uniform in the Z-axis direction, and the like.), external factors (noise and the like) are reduced as much as possible, or a plurality of subjects can perform measurement under the same environment as much as possible. Therefore, for example, it is possible to accurately know the degree of recovery of the motor function of upper limb or to prevent variation in the examination and measurement states among the subjects. In addition, if the set conditions are visualized by image or made audible by voice, the measurement conditions are reliably recognized by the subject (Alternatively, external factors that adversely affect the measurement are eliminated.), an appropriate measurement environment is prepared, and an accurate measurement result can be obtained.
Here, the measurement conditions set by the condition setter may be associated with the measurement environment and the subject's field of view, in which case the data processor preferably generates audio data that eliminates noise in the measurement environment and/or image data that limits the subject's field of view. Examples of the audio data for eliminating noise in the measurement environment include audio data for outputting music for canceling surrounding noise from a speaker, and examples of the image data for limiting the subject's field of view include image data for inserting a virtual object between the finger and a surrounding object and hiding the object on the display screen.
In addition, the measurement condition set by the condition setter may be associated with the initial setting of the measurement. Examples of such initial setting include setting of a measurement reference position (positions of the head, the finger, and the line-of-sight of the subject, and the like) and setting of an acquisition position of imaging data (for example, the position of the camera) that is first performed for each measurement when measurement is executed for the same subject a plurality of times in order to accurately grasp the degree of recovery of the motor function of upper limb. In addition, such condition setting can also be performed as preprocessing of measurement processing (examination), and can contribute to unification of measurement conditions.
Furthermore, in the above configuration, the imaging data collector may include a camera that captures the motion of the finger. Such a camera can capture the measurement environment. In the above configuration, the measurement processing terminal may further include an output interface that outputs data generated by the hand tracking data generator and/or the data processor. Examples of such an output interface include a display apparatus that displays an image, text, or the like, and an audio output apparatus such as a headphone or a speaker.
In addition to the measurement processing terminal described above, the present invention also provides a method and a computer program for measuring a motion of a finger and processing a measurement result.
According to the present invention, a hand tracking function is implemented, and hand tracking data, movement distance data of a finger, and time data are processed to obtain correlation data obtained by quantifying correlation of these data. Therefore, not only a motion of a finger joint can be captured and a bending/stretching motion function of a finger and/or an opening/closing motion function of two fingers can be quantitatively evaluated, but also a motor function of upper limb can be objectively and finely and accurately evaluated.
Hereinafter, embodiments of the present invention will be described with reference to the drawings. The present embodiment contributes to the development of medical care and the realization of a healthy society with advanced technologies by providing the following technologies. The implementation of the measurement processing terminal contributes to the “9. Build resilient infrastructure, promote inclusive and sustainable industrialization and foster innovation” of the Sustainable Development Goals (SDGs) proposed by the United Nations.
Note that, in the following, the measurement processing terminal according to the present invention that performs measurement processing of a motion of a finger will be described as a head mounted display (HMD) (first embodiment) or as a smartphone (second embodiment). However, the measurement processing terminal of the present invention may have a form such as a thin tablet computer, a personal computer, or the like, or a use form in which the measurement processing terminal is connected to a server via a communication means (network) is also conceivable, and any structure form and use form may be adopted.
In addition, in the following embodiments, a measurement processing terminal that includes a camera, a display, or the like so as to be able to acquire captured imaging data and measure and display the motion of the finger by itself is illustrated. However, the present invention may be embodied as a terminal or a method that enables measurement processing of the motion of the finger in cooperation with a separate imaging camera and a display, or may be configured as a computer program that enables such measurement processing to be performed by a computer.
As illustrated in
The first camera 6 is a rear camera built in the HMD 50 to capture the motion of the finger of the subject including a measurement environment (surrounding object or scene), and the second camera 8 is built in the HMD 50 as a front camera to capture an eye of the subject for eye tracking by gaze detectors 12 and 14. Each of the cameras 6 and 8 captures an image of an object and captures the captured image (imaging data).
The distance detection sensor 10 constitutes a movement distance measuring instrument that measures a chronological movement distance in which a hand moves with motion of an arm, and is a sensor capable of capturing a shape of an object such as a person or an object as a three-dimensional shape (Alternatively, a timer that measures time may be separately provided.). Examples of such a sensor include a Light Detection and Ranging (LiDAR) sensor that irradiates an object with laser light such as infrared light, measures reflected scattered light, and analyzes and detects a distance to an object at a long distance and a state of the object, a Time Of Flight (TOF) sensor that measures a reflection time of pulsed light applied to the object for each pixel and measures a distance, and a millimeter wave radar that emits a millimeter wave, captures the reflected wave, and detects a distance to the reflecting object and a state of the object. In particular, the distance detection sensor 10 of the present embodiment can detect the distance to the finger of the subject and the angle thereof, and enables chronological measurement of each distance.
The right-eye gaze detector 12 and the left-eye gaze detector 14 detect a line-of-sight of the right eye and the left eye of the subject, respectively. Note that the processing of detecting the line-of-sight may use a well-known technique generally used as eye tracking processing. For example, in a method using corneal reflection, there is known a technique of irradiating a face with an infrared Light Emitting Diode (LED) and photographing a face with an infrared camera, and detecting the line of sight on the basis of a position of a pupil with respect to a position of corneal reflection with a position (corneal reflection) of reflected light generated by irradiation of the infrared LED as a reference point.
The geomagnetic sensor 25 is a sensor (gravity sensor) that detects the magnetic force of the earth, and detects the direction in which the HMD 50 is facing (an angle of a neck of the subject). As the geomagnetic sensor, it is also possible to detect the motion of the HMD 50 (the angle of the neck of the subject) by capturing a geomagnetic change with respect to a motion of the HMD 50 using a three-axis type that also detects geomagnetism in vertical direction in addition to front-back direction and left-right direction.
The condition setter 24 is for setting a measurement condition, and includes, for example, a user interface for selecting an examination mode such as a motor function of upper limb or a finger tapping motion function or selecting measurement conditions prepared for each examination mode, and the measurement condition can be set by displaying the user interface on a display screen of the HMD 50, performing gesture operation or voice input, or performing input via an input means such as a keyboard, a key button, or a touch key. Examples of the measurement condition in the finger tapping motion function prepared as the user interface include selection of a measurement mode such as both hands at the same time, both hands alternately, only one hand (right hand), and only one hand (left hand), selection of whether or not to show a finger part to the subject at the time of measurement (on/off of the function of the transmission masking of the finger), and selection of whether or not to suppress noise relating to the measurement environment sensed by the eye and ear of the subject (for example, on/off of a function of inserting a virtual object between the finger and a surrounding object to hide surrounding information). In addition, as an example of the measurement condition prepared as another user interface, for example, a function of displaying a past measurement result or a target standard with a finger contour, a dotted line, or the like may be set so that a change (In the case of rehabilitation training, its rehabilitation effect) in the motor function of a finger (upper limb) can be grasped.
In addition, the hand tracking data generator 26 implements a hand tracking function of detecting and tracking the position of the finger based on the imaging data acquired by the camera 6, and generates the chronological hand tracking data from the imaging data by the hand tracking function. As the hand tracking (skeleton detection) function, for example, an open source machine learning tool “MediaPipe” provided by Google LLC (U.S.A.) may be utilized.
In addition, the data processor 27 not only processes the hand tracking data obtained from the hand tracking data generator 26 to generate quantitative data related to a bending/stretching motion of the finger and/or an opening/closing motion of the two fingers accompanying the motion of the finger joint, but also processes the hand tracking data obtained from the hand tracking data generator 26, the distance data and the time data obtained from the distance detection sensor 10, and the line-of-sight data obtained from the gaze detectors 12 and 14 to generate correlation data obtained by quantifying the correlation between these data. Further, the data processor 27 generates image data related to a measurement reference position and/or a measurement history, and also generates image data and/or audio data corresponding to the measurement condition set by the condition setter 24. In particular, when the measurement condition is associated with the measurement environment and the subject's field of view, the data processor 27 generates audio data that eliminates noise in the measurement environment and/or image data that limits the subject's field of view. Note that, in the present embodiment, the data processor 27 also constitutes a controller of the HMD 50, and is constituted by a CPU or the like, and executes the program 29 such as an Operating System (OS) 30 and various operation control applications 31 stored in the memory 28 to perform operation control processing of the entire HMD 50 and control startup operation of various applications.
The memory 28 includes a flash memory or the like, and stores the program 29 such as the operating system 30 and the operation control application 31 for various processes such as an image, audio, document, display, and measurement. The memory 28 also stores information data 32 such as base data 33 necessary for basic operation by the operating system 30 or the like, and file data 34 used by the various applications 31 or the like. For example, the image processing application is activated, the image is captured by the camera, and the captured file data is stored. Note that the processing in the data processor 27 may be stored as an application A, and the measurement processing of the motion of the finger and the calculation analysis of various feature amounts may be performed by activating the application A. In addition, a measurement result measured from the information processing terminal may be received by an external server apparatus or the like having high calculation performance and a large capacity, and the feature amount may be calculated and analyzed.
The display apparatus 16 is an output interface that outputs data generated by the hand tracking data generator 26 and/or the data processor 27, and can display, in particular, a processing result processed by the data processor 27. In the case of an optically transmissive HMD, the display apparatus 16 includes, for example, a projection unit that projects various types of information such as reproduction information by a start-up application and notification information to the subject, and a transparent half mirror that forms and displays the projected various types of information in front of the subject's eyes. In addition, in the case of a video transmissive HMD, the HMD includes a display such as a liquid crystal panel that displays a real space object in front of the subject captured by the first camera 6 and various types of information together. This allows the subject to visually recognize and view not only the image in the field of view in front of the subject but also image information from other sources.
Note that, in the second embodiment to be described later, since the measurement processing terminal is a smartphone, the display apparatus 16 includes a liquid crystal panel or the like, and can display notification information to the subject such as a remaining amount of battery capacity, various alarms and time, and an icon of an application to be activated in the display screen, and the like, together with display of an image and a video.
The operation input interface 19 of the HMD 50 often uses gesture operation or audio input, but for example, an input means such as a keyboard, a key button, or a touch key may be used, and information desired to be input by the subject is set and input. In the second embodiment to be described later, since the measurement processing terminal is the smartphone, the operation input interface 19 can be provided in the terminal itself. However, in the present embodiment, the operation input interface 19 may be provided at a position or in a form in which the subject easily performs an input operation in the HMD 50, or may be in a form of being connected in a wired or wireless manner separately from the main body of the HMD 50. In addition, an input operation screen may be displayed in the display screen of the display apparatus 16, and input operation information may be fetched according to the position on the input operation screen to which the line-of-sight detected by the right-eye gaze detector 12 and the left-eye gaze detector 14 is directed, or a pointer may be displayed on the input operation screen and the pointer may be operated by the operation input interface 19 to fetch the input operation information. In addition, the subject may utter a voice indicating the input operation, collect the voice with the microphone 18, and capture the input operation information.
The microphone 18 can also constitute an output interface that outputs audio data generated by the data processor 27, and collects a voice from the outside or a user's own vocalization. Furthermore, the speaker 20 outputs a voice to the outside to inform the user of a voice such as notification information or music. In addition, the instruction related to the measurement of the finger motion may be transmitted to the subject by voice through the speaker 20.
The communication interface 22 is a communication interface that performs wireless communication with a server apparatus or the like at another location by short-range wireless communication, wireless LAN, or base station communication, and transmits and receives measurement data, analyzed and calculated feature amount, and the like to and from the server apparatus or the like via the transmission/reception antenna 23 at the time of wireless communication. Note that the short-range wireless communication is performed using, for example, an electronic tag, but is not limited thereto, and may be performed using a wireless LAN such as Bluetooth (registered trademark), IrDA (Infrared Data Association, registered trademark), Zigbee (registered trademark), HomeRF (Home Radio Frequency, registered trademark), or Wi-Fi (registered trademark) as long as the measurement processing terminal can perform at least wireless communication when the measurement processing terminal is near another information terminal. Further, as the base station communication, wireless communication over a long distance such as W-CDMA (Wideband Code Division Multiple Access) or GSM (Global System for Mobile communications, registered trademark) may be used. It is also possible to detect a positional relationship and an orientation between terminals by using an Ultra Wide Band (UWB). Although not illustrated, the communication interface 22 may use another method such as communication using optical communication sonic communication and the like as a means of wireless communication. In this case, instead of the transmission/reception antenna 23, a light emitter/receiver and a sound wave output/sound wave input interface are used, respectively.
Note that, in the present embodiment, the HMD 50 individually includes the above-described respective components as described above, but may include a functional unit that integrates at least some or all of these components, and in short, may have any configuration as long as the functions of these respective components are secured.
Next, the measurement processing operation of the HMD 50 in which measurement processing of the motion of the finger is performed will be described with reference to
In addition, in a case where the HMD 50 is shared by a plurality of persons, unique identification information such as a fingerprint and a palm print may be registered in advance and collated before measurement so that the information of the contour of the finger of the subject may be read and displayed.
When the measurement condition is set by the condition setter 24 at the start of the examination (condition setting step S3 in
In the present embodiment, the measurement condition by the condition setter 24 is associated with the measurement environment and the field of view of the subject 60, and the data processor 27 generates audio data that eliminates noise of the measurement environment and/or image data that limits the field of view of the subject 60. For example, in order to suppress unnecessary information from the ear and the eye, the data processor 27 generates audio data for outputting, from the speaker 20, music for canceling surrounding noise as the audio data for eliminating noise in the measurement environment, and generates image data for inserting and hiding a virtual object between the finger of the subject 60 and the surrounding object on the screen 16a, as the image data for limiting the field of view of the subject 60. That is, by playing videos and music on the HMD 50, the subject can relax and take measurements.
An example of image data generation related to such measurement condition setting is illustrated in
In addition,
As described above, in the present embodiment, the display/non-display of the hand portion can be switched according to the measurement application. That is, in the case of measurement in which it is desired to show the motion of the finger, the video captured by the first camera 6 is displayed only for the part of the hand 63 as illustrated in
Note that, in a case where the gaze alignment marker 85 is displayed as illustrated in
Then, as illustrated in
An example of such correlation data is illustrated in
The correlation data as described above can be displayed on the screen 16a together with other data generated by the hand tracking data generator 26 and/or the data processor 27 (output step S10 in
Next, a second embodiment of the present invention will be described with reference to
As described above, according to the first and second embodiments, the chronological hand tracking data can be generated by the hand tracking function from the imaging data obtained by capturing the motion of the finger of the subject 60, and the hand tracking data can be processed to generate quantitative data regarding the bending/stretching motion of the finger and/or the opening/closing motion of the two fingers accompanying the motion of the finger joint. Therefore, the motion of the joint that cannot be recognized by the conventional magnetic sensor type apparatus can be accurately grasped, and the bending/stretching motion of the finger such as a “pinching” motion can be quantitatively evaluated. Unlike the conventional magnetic sensor type apparatus, it is not necessary to attach the sensor to a fingertip.
In addition, according to the first and second embodiments, since the hand tracking data obtained from the hand tracking data generator 26, the distance data and the time data obtained from the distance detection sensor 10, and the line-of-sight data obtained from the gaze detectors 12 and 14 can be processed to generate correlation data obtained by quantifying the correlation between these data, for example, when an arm is moved to grip an object, the motion of the arm and the opening/closing motion of the finger can be quantitatively evaluated at the same time (the interlocking property (relationship) between the motion of the finger and the motion of the arm can be quantitatively evaluated), and furthermore, the interlocking property (relationship) between the motion of the finger and the motion of the eye can be quantitatively evaluated. That is, the finger can be evaluated in synchronization with the motion of another part of the body, and thus the motor function of upper limb can be objectively, finely and accurately evaluated.
Although the embodiments of the present invention have been described above with reference to the drawings, the present invention is not limited to the above-described embodiments and can include various modifications. For example, the above-described embodiments have been described in detail in order to describe the present invention in an easy-to-understand manner, and are not necessarily limited to those having all the described configurations. Further, a part of the configuration of one embodiment can be replaced with the configuration of another embodiment, and the configuration of another embodiment can be added to the configuration of one embodiment. In addition, it is possible to add, delete, and replace other configurations for a part of the configuration of each embodiment.
In addition, some or all of the above-described configurations, functions, processers, processing means, and the like may be realized by hardware, for example, by designing with an integrated circuit. In addition, each of the above-described configurations, functions, and the like may be realized by software by a processor interpreting and executing a program for realizing each function. Information such as a program, a table, and a file for realizing each function may be stored in a recording apparatus such as a memory, a hard disk, and a Solid State Drive (SSD), or a recording medium such as an IC card, an SD card, and a DVD, or may be stored in an apparatus on a communication network.
In addition, the control lines and the information lines indicate what is considered to be necessary for the description, and do not necessarily indicate all the control lines and the information lines on the product. In practice, it may be considered that almost all the configurations are connected to each other.
| Filing Document | Filing Date | Country | Kind |
|---|---|---|---|
| PCT/JP2021/034132 | 9/16/2021 | WO |
