Patent Application
20230190138
This application is based upon and claims the benefit of priority from Japanese patent application No. 2021-207174, filed on Dec. 21, 2021, the disclosure of which is incorporated herein in its entirety by reference.
The present disclosure relates to an information processing apparatus, an information processing method, a program, an information processing system, and a generation method.
A technology for diagnosing (determining) an abnormality in a lower limb based on data measured by a sensor(s) has been known (see, for example, Japanese Unexamined Patent Application Publications No. 2004-261525 and No. 2019-150229).
However, the technology disclosed in Japanese Unexamined Patent Application Publications No. 2004-261525 and No. 2019-150229 has a problem that, in some cases, it cannot be properly determined that, for example, there is an abnormality in a lower limb.
In view of the above-described problem, an example object of the present disclosure is to provide an information processing apparatus, an information processing method, a program, and an information processing system capable of making a proper decision about an abnormality in a lower limb.
In a first example aspect, an information processing apparatus includes: an acquisition unit configured to acquire information based on a sensor attached to a foot of a user; a determination unit configured to make a decision about an abnormality in walking of the user based on an acceleration of the user in a walking direction, acquired by the acquisition unit; and an output unit configured to output information based on a result of the decision made by the determination unit.
Further, in a second example aspect, an information processing method includes: acquiring information based on a sensor attached to a foot of a user; making a decision about an abnormality in walking of the user based on an acquired acceleration of the user in a walking direction; and outputting information based on a result of the decision.
Further, in a third example aspect, a program causes an information processing apparatus to perform: a process for acquiring information based on a sensor attached to a foot of a user; a process for making a decision about an abnormality in walking of the user based on an acquired acceleration of the user in a walking direction; and a process for outputting information based on a result of the decision.
Further, in a fourth example aspect, an information processing system includes a sensor attached to a foot of a user, an information processing apparatus, and an information processing terminal. This information processing system includes: an acquisition unit configured to acquire information based on an acceleration of the user in a walking direction, measured by the sensor; a determination unit configured to make a decision about an abnormality in walking of the user based on an acceleration of the user in a walking direction, acquired by the acquisition unit; and an output unit configured to make the information processing terminal output information based on a result of the decision made by the determination unit.
Further, in a fifth example aspect, a generation method includes: acquiring a data set containing a combination of information based on an acceleration of a user in a walking direction, measured by a sensor attached to a foot of the user, and information indicating an abnormality in walking of the user; generating, based on an acquired acceleration of the user in the walking direction, a trained model from information measured by the sensor attached to the foot of the user, the trained model being configured to make a decision about an abnormality in walking of the user; and outputting the generated trained model.
The above and other aspects, features, and advantages of the present disclosure will become more apparent from the following description of certain example embodiments when taken in conjunction with the accompanying drawings, in which:
A principle of the present disclosure will be described with reference to several illustrative example embodiments. It should be understood that these example embodiments are described only for an illustrative purpose and will assist those skilled in the art in understanding and carrying out the present disclosure without suggesting any limitations in regard to the scope of the disclosure. Disclosures described in this specification can also be implemented in a variety of ways other than those described below.
In the following description and the claims, unless otherwise defined, all technical and scientific terms used in this specification have the same meanings as those generally understood by those skilled in the technical field to which the present disclosure belongs.
An example embodiment according to the present disclosure will be described hereinafter with reference to the drawings.
«Information Processing Apparatus 10A that Performs Generation Process»
A configuration of an information processing apparatus 10A that performs a generation process according to an example embodiment will be described with reference to
The acquisition unit 11 acquires various types of information from a storage unit disposed inside the information processing apparatus 10A or from an external apparatus. The acquisition unit 11 acquires, for example, a data set containing a combination of information based on an acceleration in the walking direction (the traveling direction) of a user, measured by a sensor(s) attached to a foot (or feet) of the user, and information indicating an abnormality in the walking of the user.
Based on the information acquired by the acquisition unit 11, the generation unit 12 generates a trained model that makes a decision about an abnormality in the walking of the user from the information measured (i.e., obtained) by the sensor attached to the foot of the user. The output unit 13 outputs (transmits or records) various types of information to the storage unit disposed inside the information processing apparatus 10A or to the external apparatus. For example, the output unit 13 outputs the trained model generated by the generation unit 12.
«Information Processing Apparatus 10B that Performs Determination Process»
Next, a configuration of an information processing apparatus 10B according to an example embodiment will be described with reference to
The acquisition unit 15 acquires various types of information from a storage unit disposed inside the information processing apparatus 10B or from an external apparatus. The acquisition unit 15 acquires, for example, information based on an acceleration of a user in the walking direction, measured by a sensor(s) attached to a foot (or feet) of the user.
The determination unit 16 makes a decision about an abnormality in the walking of the user based on the information acquired by the acquisition unit 15. The output unit 17 outputs (transmits, displays, reports, or records) various types of information to the storage unit disposed inside the information processing apparatus 10B or to the external apparatus. For example, the output unit 17 outputs information based on the result of the decision made by the determination unit 16.
Next, a configuration of an information processing system 1 according to an example embodiment will be described with reference to
An example in which a process for generating (training) a trained model, and a determination (estimation or inference) process are performed in the server 40 will be described hereinafter. Note that the generation process may be performed in at least one of the measurement apparatuses 20, the user terminal 30, the server 40, and the hospital terminal 50. Further, the determination process may be performed in at least one of the measurement apparatuses 20, the user terminal 30, the server 40, and the hospital terminal 50. The generation process and the determination process may be performed in the same apparatus or in different apparatuses. The measurement apparatuses 20 and the user terminal 30 may be connected to each other through, for example, short-range wireless communication such as BLE (Bluetooth (Registered Trademark) Low Energy) or through a cable so that they can communicate with each other.
In the example shown in
The measurement apparatus 20 (i.e., each of the measurement apparatuses 20) includes a sensor 21 that is attached to a foot of a user. The measurement apparatus 20 outputs data measured (i.e., obtained) by using the sensor 21 to an external apparatus such as the user terminal 30.
The user terminal 30 may be, for example, an apparatus such as a smartphone, a tablet-type computer, a personal computer, an Internet of Things (IoT) communication device, and a mobile phone. The user terminal 30 transmits the data acquired from the measurement apparatus 20 to the server 40.
The server 40 is, for example, an apparatus such as a server, a cloud computing system, a personal computer, and a smartphone. The server 40 makes a decision about an abnormality in the walking of the user based on the data measured by the measurement apparatus 20. Further, the server 40 transmits information based on the result of the decision to the hospital terminal 50.
The hospital terminal 50 may be, for example, an apparatus such as a smartphone, a tablet-type computer, a personal computer, and a mobile phone. The hospital terminal 50 may be a terminal used in a facility such as a hospital. The hospital terminal 50 displays information received from the server 40 on a screen.
These components may be connected to each other through a bus or the like. The memory 102 stores at least a part of a program 104. The communication interface 103 includes an interface necessary for communicating with other network elements.
When the program 104 is executed through the cooperation of the processor 101, the memory 102, and the like, at least one of the processes of the example embodiment according to the present disclosure is performed by the computer 100. The memory 102 may be any type of memory suitable for a local technology network. The memory 102 may be, but is not limited to, a non-transitory computer readable storage medium. Further, the memory 102 may be implemented by using any suitable data storage technology such as a semiconductor-based memory device, a magnetic memory device and a system, an optical memory device and a system, a fixed memory, and a removable memory. Although only one memory 102 is provided in the computer 100, a plurality of physically different memory modules may be provided in the computer 100. The processor 101 may be any type of processor. The processor 101 may include at least one of a general-purpose computer, a dedicated computer, a microprocessor, and a digital signal processor (DSP: Digital Signal Processor), and may also include, but is not limited to, at least one processor based on a multi-core processor architecture. The computer 100 may include a plurality of processors, such as an application-specific integrated circuit chip that is temporally dependent on a clock for synchronizing the main processor.
An example embodiment according to the present disclosure may be implemented by hardware, a dedicated circuit, software, a logic, or any combination thereof. In some aspects, an example embodiment may be implemented by hardware, while in other aspects, an example embodiment may be implemented by firmware or software that may be executed by a controller, a microprocessor, or other computing devices. The present disclosure also provides at least one computer program product that is tangibly stored in a non-transitory computer readable storage medium. The computer program product contains computer executable instructions, such as those contained in program modules, and is executed by a target real processor or by a device on a virtual processor, so that a process(es) or a method according to the present disclosure is performed. The program module contains routines, programs, libraries, objects, classes, components, and data structures for performing specific tasks or implementing specific abstract data types. The functions of the program module may be combined with those of the other program modules, or divided into a plurality of program modules as desired in various example embodiments. The machine executable instructions in the program module can be executed locally or in a distributed device(s). In the distributed device, the program module can be disposed on both local and remote storage media.
The program codes for performing the method according to the present disclosure may be written in any combination of at least one programming language. These program codes are provided to a processor or a controller of a general-purpose computer, a dedicated computer, or other programmable data processing apparatuses. When such a program code is executed by the processor or the controller, a function/operation in a flowchart and/or a block diagram to be implemented is executed. The program code is entirely executed in a machine, partially executed in a machine as a standalone software package, partially executed in a machine, partially executed in a remote machine, or entirely executed in a remote machine or a server.
The program can be stored and provided to a computer using any type of non-transitory computer readable media. Non-transitory computer readable media include any type of tangible storage media. Examples of non-transitory computer readable media include magnetic storage media (such as floppy disks, magnetic tapes, hard disk drives, etc.), optical magnetic storage media (e.g., magneto-optical disks), CD-ROM (compact disc read only memory), CD-R (compact disc recordable), CD-R/W (compact disc rewritable), and semiconductor memories (such as mask ROM, PROM (programmable ROM), EPROM (erasable PROM), flash ROM, RAM (random access memory), etc.). The program may be provided to a computer using any type of transitory computer readable media. Examples of transitory computer readable media include electric signals, optical signals, and electromagnetic waves. Transitory computer readable media can provide the program to a computer via a wired communication line (e.g., electric wires, and optical fibers) or a wireless communication line.
Next, an example of a measurement apparatus 20 according to an example embodiment will be described with reference to
In the example shown in
In the example shown in
Next, with reference to
In a step S101, the acquisition unit 11 of the information processing apparatus 10A acquires a data set 801 for learning (hereinafter also referred to as the learning data set 801). In the example shown in
The correct answer label is a correct answer label used in supervised learning. The correct answer label may contain, for example, information indicating the presence/absence of an abnormality. Further, the correct answer label may include, for example, information indicating the severity of an abnormality. In this case, for example, the severity of an abnormality may be expressed by a number on a scale of 10 from 0 to 9, in which the severity “0” may indicate that there is no abnormality and the severity “9” may indicate the highest severity. The correct answer label may be, for example, designated (set or registered) by an operator of the server 40, a user of the hospital terminal 50 (e.g., a doctor or the like), or a user of the user terminal 30.
Further, at a time point 913, as shown in
Further, at a time point 914, the foot is sharply decelerated by decisively returning the knee from the flexed state to the stretched state before the foot lands on the ground surface. In the following description, the time point at which the foot of the user lands on the ground surface is also referred to as a “time point C”. Further, the value of the acceleration in the walking direction at the time point C is also referred to as an “acceleration C” (an example of the “third acceleration”). (Example in which acceleration B is used)
The items of the data included in the records of the learning data set 801 may include, for example, an item for data based on the acceleration B. This is because it is considered that since a user (a patient) suffering from osteoarthritis (OA) or the like may not be able to flex his/her knee sufficiently, an acceleration in the walking direction when the toe comes off the ground surface is degenerated (decreased) as compared to that of a healthy person.
In this case, the aforementioned item may be an item based on the acceleration B, and at least one of the acceleration A and the walking speed of the user. In this case, the item may include an item based on a ratio between the accelerations A and B (e.g., the value of the ratio between the accelerations A and B, or a reciprocal value thereof). This is because since there is no significant difference between the acceleration A of a patient suffering from OA or the like and that of a healthy person, the effect that would otherwise be caused by the difference between walking speeds is reduced for the value of the ratio between the accelerations A and B, or the reciprocal value thereof. Further, the item for data based on the acceleration B may include an item based on a ratio between the walking speed and the acceleration B (e.g., a radio expressed as “walking speed/acceleration B” or a reciprocal value thereof). Note that the walking speed may be calculated based on changes in the acceleration in the walking direction, measured by the sensor 21. Note that the symbol “/” indicates a division. Therefore, for example, an expression “X/Y” refers to the value of a ratio between X and Y.
(Example in which acceleration C is used)
The items of the data included in the records of the learning data set 801 may include, for example, an item for data based on the acceleration C. This is because it is considered that since a patient suffering from osteoarthritis (OA) or the like does not have an ability to move his/her knee sufficiently, he/she gradually decreases the walking speed during the swing phase, so that the acceleration C is degenerated (decreased) as compared to that of a healthy person. In this case, the aforementioned item may be an item based on the acceleration C, and at least one of the acceleration A and the walking speed of the user. In this case, the item may include an item based on a ratio between the accelerations A and C (e.g., a value expressed as “acceleration A/acceleration C”, or a reciprocal value thereof). This is because since there is no significant difference between the acceleration A of a patient suffering from OA or the like and that of a healthy person, the effect that would otherwise be caused by the difference between walking speeds is reduced for the value of the ratio between the accelerations A and C, or the reciprocal value thereof. Further, the item for data based on the acceleration C may include an item based on a ratio between the walking speed and the acceleration C (e.g., a radio expressed as “walking speed/acceleration C” or a reciprocal value thereof).
Further, the items of the data included in the records of the learning data set 801 may include, for example, an item for the magnitude of a one-vibration component of an acceleration of the user in the walking direction during a period in which the user makes one step. Further, the items of the data included in the records of the learning data set 801 may include, for example, an item based on the angle between the sole of the foot and the ground surface in the walking direction during a period in which the user makes one step. This is because it is considered that since a patient suffering from osteoarthritis (OA) or the like may not be able to flex (dorsiflex and plantar-flex) his/her ankle sufficiently, the angle between the sole of the foot and the ground surface in the walking direction is degenerated (decreased) as compared to that of a healthy person.
Further, the items of the data included in the records of the learning data set 801 may include, for example, the magnitude of a five-vibration component of an acceleration of the user in the vertical direction during a period in which the user makes one step.
Next, the generation unit 12 of the information processing apparatus 10A generates, based on the machine-learning data set 801, a trained model by using a machine learning method through supervised learning (Step S102). Note that the information processing apparatus 10A may use, for example, a weighted K-Nearest Neighbor Algorithm. In this case, the information processing apparatus 10A may generate, for example, a trained model in which weights that are inversely proportional to the Euclidean distance are added, and a K value is 10 (nearby 10 points).
Further, the information processing apparatus 10A may generate, for example, a trained model by using an arbitrary machine learning method through supervised learning, such as, a neural network (Neural Network: NN), a random forest, a linear regression, and a support vector machine.
Next, the output unit 13 of the information processing apparatus 10A outputs the trained model generated by the generation unit 12 (Step S103). Note that the information processing apparatus 10A may store the generated trained model in a storage unit provided therein. Alternatively or additionally, the information processing apparatus 10A may transmit (distribute) the generated trained model to an external apparatus(es) and make the external apparatus(es) record the trained model therein.
Next, an example of a determination process performed by an information processing apparatus 10B according to an example embodiment will be described with reference to
16 shows an example of changes in the angle between the sole of a foot and the ground surface in the walking direction during a period in which a healthy person makes one step and those during a period in which a patient makes one step based on data measured by the sensor 21 according to the example embodiment.
In a step S201, the acquisition unit 15 of the information processing apparatus 10B acquires data about walking of a user, measured by the sensor 21. Next, the determination unit 16 of an information processing apparatus 10B extracts data about the walking of the user during a period in which the user makes one step (Step S202). Note that the information processing apparatus 10B may determine, for example, a start point and an end point of the period in which the user makes one step based on changes of the angle between the sole of the user's foot and the ground surface in the walking direction. In this case, the information processing apparatus 10B may determine the start point and the end point of the period in which the user makes one step based on, for example, a period from when the angle between the sole of the user's foot and the ground surface in the walking direction reaches the maximum value to when the angle, after reaching the minimum value, reaches the maximum value again.
Alternatively, the information processing apparatus 10B may determine the start point and the end point of the period in which the user makes one step based on, for example, changes in the acceleration in the walking direction. In this case, the information processing apparatus 10B may determine the start point and the end point of the period in which the user makes one step based on, for example, a period from when the acceleration in the walking direction reaches the maximum value to when the acceleration, after reaching the minimum value, reaches the maximum value again.
Then, the information processing apparatus 10B may normalize the length of the time period in which the user makes one step (i.e., the walking cycle) as indicated by the horizontal axis in
In the case where, after the length of the time period in which the user makes one step is normalized into a value irrelevant to the time, for example, there are two maximum values of the acceleration in the walking direction side by side in a predetermined range, the information processing apparatus 10B may determine the second maximum value as the time point A. In this way, for example, it is possible to appropriately determine the time point A for the healthy person. Alternatively, the information processing apparatus 10B may determine, for example, a time point at which the acceleration in the walking direction has the maximum value in the period in which the user makes one step as the time point A.
Further, in the case where, after the length of the time period in which the user makes one step is normalized into a value irrelevant to the time, for example, there are two maximum values of the acceleration in the walking direction side by side in a predetermined range, the information processing apparatus 10B may determine the first maximum value as the time point B. In this way, for example, it is possible to appropriately determine the time point B for the healthy person. Further, the information processing apparatus 10B may determine, for example, after the length of the time period in which the user makes one step is normalized into a value irrelevant to the time, a time point that is a predetermined length (i.e., a time equivalent to 5% of the one walking cycle) earlier than the time point at which the acceleration in the walking direction has the maximum value as the time point B.
Alternatively, the information processing apparatus 10B may determine, for example, a time point at which the acceleration in the walking direction has the minimum value in the period in which the user makes one step as the time point C.
Next, the determination unit 16 of the information processing apparatus 10B calculates feature values related to the walking based on the data about the walking of the user during the period in which the user makes one step (Step S203). Note that the information processing apparatus 10B calculates data for at least one item that is the same as an item(s) of the data included in the records of the learning data set 801 shown in
Further, the information processing apparatus 10B may calculate, for example, the magnitude of a one-vibration component of an acceleration of the user in the walking direction during a period in which the user makes one step as shown in
Further, the information processing apparatus 10B may calculate an item based on, for example, the angle between the sole of the foot and the ground surface in the walking direction during a period in which the user makes one step. In this case, this item may include, for example, a difference between the maximum value and the minimum value of the angle between the sole of the foot and the ground surface in the walking direction during the period in which the user makes one step.
Further, the information processing apparatus 10B may calculate, for example, the magnitude of a five-vibration component of an acceleration of the user in the vertical direction during a period in which the user makes one step. In this case, the information processing apparatus 10B may first perform a Fourier transform for a waveform representing the changes in the acceleration of the user in the vertical direction during the period in which the user makes one step, and thereby converts the changes in the acceleration from the time-dependent waveform into a frequency-dependent waveform. Then, the information processing apparatus 10B may extract the value of this waveform at a frequency at which five vibrations occur during the period in which the user makes one step as the five-vibration component.
Next, the determination unit 16 of the information processing apparatus 10B makes a decision (an estimation or inference) an abnormality in the walking of the user based on the data of the calculated feature values and the trained model generated by the generation unit 12 (Step S204). Note that the information processing apparatus 10B infers correct answer labels (objective variables) by using the trained model while using the calculated feature values as explanatory variables. In this way, it is possible to, for example, determine the degree of the progress of the condition of the disease of the patient, compare the condition of the disease before a surgery and that after the surgery, and evaluate the degree of recovery.
Next, the output unit 17 of the information processing apparatus 10B outputs information based on the result of the decision made by the determination unit 16 (Step S205). Note that the information processing apparatus 10B may display, for example, a message indicating the presence/absence of an abnormality and the severity thereof. Further, the information processing apparatus 10B may report the presence/absence of an abnormality and the severity thereof by, for example, producing a sound (or a voice) and/or turning on a lamp or like. Further, the information processing apparatus 10B may record, for example, changes in the value indicating the degree of certainty of the presence/absence of an abnormality for a specific user, calculated in the step S204 at a plurality of dates/times (e.g., on a weekly basis). Then, the information processing apparatus 10B may output a message about advice on rehabilitation for the user based on the changes.
(Effect of Each Item Used for Inference)
Next, an example of the effect of each item (each feature value or each explanatory variable) used for inference will be described with reference to
The information processing apparatus 10 may be an apparatus housed in one housing, but the information processing apparatus 10 according to the present disclosure is not limited to such examples. Each unit of the information processing apparatus 10 may be implemented by, for example, cloud computing formed by at least one computer. Further, each unit of the information processing apparatus 10 may be implemented by two of more of the measurement apparatuses 20, the user terminal 30, the server 40, and hospital terminal 50. Such an information processing apparatus 10 is also included in examples of the “information processing apparatus” according to the present disclosure.
Each of above-described example embodiments can be combined as desirable by one of ordinary skill in art.
An example advantage according to above-described example embodiments is to be able to make a proper decision about an abnormality in a lower limb.
While the disclosure has been particularly shown and described with reference to example embodiments thereof, the disclosure is not limited to these example embodiments. It will be understood by those of ordinary skill in art that various changes in form and details may be made therein without departing from the spirit and scope of the present disclosure as defined by the claims.
The whole or part of the example embodiments disclosed above can be described as, but not limited to, the following supplementary notes.
An information processing apparatus comprising:
The information processing apparatus described in Supplementary note 1, wherein the acquisition unit acquires information based on a sensor attached at a position located between an arch of the user's foot and a heel thereof.
The information processing apparatus described in Supplementary note 1 or 2, wherein the determination unit makes a decision about an abnormality in walking of the user based on at least one of a first acceleration in the walking direction that occurs when a toe of the foot comes off a ground surface, a second acceleration in the walking direction that occurs when the foot is swung in the walking direction by contraction of a thigh muscle of the user, and a walking speed of the user.
The information processing apparatus described in any one of
Supplementary notes 1 to 3, wherein the determination unit makes a decision about an abnormality in walking of the user based on at least one of a third acceleration in the walking direction that occurs when the foot lands on a ground surface, a second acceleration in the walking direction that occurs when the foot is swung in the walking direction by contraction of a thigh muscle of the user, and a walking speed of the user.
The information processing apparatus described in any one of Supplementary notes 1 to 4, wherein the determination unit makes a decision about an abnormality in walking of the user based on a magnitude of a one-vibration component of an acceleration of the user in the walking direction during a period in which the user makes one step.
The information processing apparatus described in any one of Supplementary notes 1 to 5, wherein the determination unit makes a decision about an abnormality in walking of the user based on at least one of an angle between a sole of the foot and a ground surface in the walking direction during a period in which the user makes one step, and a magnitude of a five-vibration component of an acceleration of the user in a vertical direction during the period in which the user makes one step.
An information processing method comprising:
A program for causing an information processing apparatus to perform:
A trained model configured to cause an information processing apparatus to perform a process for making a decision about an abnormality in walking of a user based on an acceleration of the user in a walking direction, measured by a sensor attached to a foot of the user.
An information processing system comprising a sensor attached to a foot of a user, an information processing apparatus, and an information processing terminal, wherein
The information processing system described in Supplementary note 10, wherein the acquisition unit acquires information based on a sensor attached at a position located between an arch of the user's foot and a heel thereof.
A generation method comprising:
An information processing apparatus comprising:
The information processing apparatus described in Supplementary note 13, wherein the acquisition unit acquires a data set containing a combination of information based on the acceleration of the user in the walking direction, measured by a sensor attached at a position located between an arch of the user's foot and a heel thereof, and information indicating an abnormality in walking of the user.
A program for causing an information processing apparatus to perform:
1 INFORMATION PROCESSING SYSTEM
10 INFORMATION PROCESSING APPARATUS
11 ACQUISITION UNIT
12 GENERATION UNIT
13 OUTPUT UNIT
15 ACQUISITION UNIT
16 DETERMINATION UNIT
17 OUTPUT UNIT
20 MEASUREMENT APPARATUS
21 SENSOR
22 CONTROL UNIT
23 COMMUNICATION UNIT
30 USER TERMINAL
40 SERVER
50 HOSPITAL TERMINAL
| Number | Date | Country | Kind |
|---|---|---|---|
| 2021-207174 | Dec 2021 | JP | national |
