Patent Application
20230355135
The present invention relates to a gait analyzer. And particularly, the present invention relates to an intelligent gait analyzing apparatus which calculates a health score based on the health parameters (HP) as a stride, a pace, a humpbacked value (standing or walking body tilt angle), a risk of falling (a distance between a gravity center (x,y) and a center (x,y) of two feet), and a time length from the last sitting on the chair then to next standing up.
As the health awareness rises and the trend of fitness is promoted, people are paying more attention to their health and spending time working out. Legs and feet are the fundamental of sport simply because healthy legs and feet are essential for the mobility in sports, such as running, swimming and playing badminton. If the legs and feet are disabled, such disability would restrain people from walking and playing the sport, and the enthusiasm about sport would be decreased. Hence, the monitoring the status of the capability of the legs and feet is quite important.
Currently, the conventional gait analyzer has been provided with the ability to analyze the walking behavior of people. The conventional gait analyzer utilizes a plurality of cameras to capture a walking image of one person and utilizes a plurality of sensors to sense the stress bearing by the joints and feet while the person is walking. The equipment for building up the conventional gait analyzer is complicated. The conventional gait analyzer is limited to analyzing the walking behavior of the person when the person walks, but it is difficult for the conventional gait analyzer to compare the current walking behavior with that in the past. If the conventional gait analyzer is utilized for a disabled person, a doctor needs to find the past walking behavior to evaluate the status of recovery of the disabled person and thus it reduces convenience.
Accordingly, the inventor of the present invention has designed an intelligent gait analyzing apparatus to overcome deficiencies in terms of current techniques so as to enhance the implementation and application in industries.
According to the above problems, an objective of the present invention is to provide an intelligent gait analyzing apparatus builds a complete intelligent gait analyzing and estimating system through the capturing of cameras, the operation of algorithms and establishing of databases. A doctor is able to learn a past walking situation and a current walking situation from the intelligent gait analyzing apparatus of the present invention. The intelligent gait analyzing apparatus of the present invention is beneficial to the doctor to determine a better treatment to cure the patient.
For the abovementioned purpose, the present invention provides an intelligent gait analyzing apparatus. The intelligent gait analyzing apparatus comprises a chair, a first camera and an electronic device. The chair is arranged for a patient to sit and weigh. The first camera captures a first video in which the patient walks from a starting point to the chair and a second video in which the patient changes a posture from standing up to sitting on the chair. The electronic device is electrically connected to the first camera to receive the first video and the second video and is provided with a graphic processor and a memory. The memory stores an algorithm of gravity center and a calibration of skeleton coordinate. The graphic processor establishes a 3D skeleton model and a moving diagram based on the first video. The graphic processor acquires a stride, a pace, a risk of falling and a humpbacked value based on the moving diagram. The graphic processor utilizes the algorithm of gravity center to acquire an original gravity center based on the moving diagram to obtain a risk of falling and acquire a time length from the last sitting on the chair then to next standing up based on the second video. The graphic processor generates a health score by performing a weight function on the normalizing stride, the normalizing pace, the risk of falling based on the normalizing original gravity center, the humpbacked value and the time length from the last sitting on the chair then to next standing up.
Optionally, the moving diagram is a dynamic record recoding a motion of the 3D skeleton model corresponding to the patient in the first video and the graphic processor creates a plurality of coordinates on the 3D skeleton model.
Optionally, the graphic processor calculates the stride and the pace based on two coordinates corresponding to two feet in the moving diagram.
Optionally, the graphic processor utilizes the algorithm of gravity center to calculate the original gravity center based on a moving path of the plurality of coordinates in the moving diagram.
Optionally, the graphic processor utilizes the correction algorithm to calculate a modified gravity center based on the original gravity center and the thickness of the patient's thoracic surface and belly surface.
Optionally, the present invention further comprises a second camera. The second camera is electrically connected to the electronic device and captures a patient's 3D image of the patient. The electronic device creates a point cloud diagram based on the patient's 3D image.
Optionally, the graphic processor acquires a projection distance (or thickness) based on a boundary point of the point cloud diagram and utilizes the algorithm of gravity center to calculate a modified gravity center based on a moving path of the plurality of coordinates in the moving diagram and the projection distance.
Optionally, the present invention further comprises a workstation. The workstation is in networked communication with the electronic device and is provided with a backup database. The backup database stores a reference 3D skeleton model with a plurality of reference coordinates.
Optionally, the workstation transmits the reference 3D skeleton model to the electronic device and the graphic processor utilizes a long short-term memory (LSTM) model or a transformer machine learning model to calculate a variation between the plurality of coordinates and the plurality of reference coordinates or the health parameters (HP).
Optionally, the present invention further comprises a workstation. The workstation is in networked communication with the electronic device and is provided with a history database. The history database stores a history 3D skeleton model with a plurality of history coordinates.
Optionally, the workstation transmits the history 3D skeleton model to the electronic device, and the graphic processor utilizes a long short-term memory model or a transformer machine learning model to calculate a similarity score between the plurality of coordinates and the plurality of history coordinates or the health parameters (HP) related to abnormal symptom classes from an electric medical record. The graphic processor compares the similarity score with a threshold value.
Optionally, the electronic device comprises a first database and a second database. The first database stores the health score and the second database stores the 3D skeleton model and the moving diagram.
Optionally, the time length from the last sitting on the chair then to next standing up is determined by the graphic processor calculating a time difference for a head and a pelvis of the body segments moving from a first coordinate of the head and the pelvis representing the patient is standing up and a second coordinate of the head and the pelvis representing the patient is sitting on the chair.
Optionally, the present invention further comprises a pathway. The pathway is provided with the starting point and is adjacent to the chair to provide the patient to walk from the starting point to the chair.
Optionally, the present invention further comprises a sensor. The sensor is electrically connected to the electronic device and senses an identification card of the patient to acquire an identification number. The electronic device receives the identification number.
Optionally, the present invention further comprises a medical care platform. The medical care platform is in networked communication with the electronic device to receive the identification number and the health score. The medical care platform transmits an electronic medical record to the electronic device.
In accordance with the above description, the intelligent gait analyzing apparatus of the present invention utilizes a first camera to capture a video in which the patient walks, sits down and stands up. Subsequently, the graphic processor acquires the stride, the pace, the humpbacked value, the risk of falling and the time length from the last sitting on the chair then to next standing up based on the video and utilizes the algorithm of gravity center to acquire an original gravity center. At last, the graphic processor calculates the health score by performing a weight function on the stride, the pace, the humpbacked value, the risk of falling and the time length from the last sitting on the chair then to next standing up. Doctor estimates recovery situation and muscle endurance of the patient.
The following embodiments of the present invention are herein described in detail with reference to the accompanying drawings. These drawings show specific examples of the embodiments of the present invention. It is to be acknowledged that these embodiments are exemplary implementations and are not to be construed as limiting the scope of the present invention in any way. Further modifications to the disclosed embodiments, as well as other embodiments, are also included within the scope of the appended claims. These embodiments are provided so that this disclosure is thorough and complete, and fully conveys the inventive concept to those skilled in the art. Regarding the drawings, the relative proportions and ratios of elements in the drawings may be exaggerated or diminished in size for the sake of clarity and convenience. Such arbitrary proportions are only illustrative and not limiting in any way. The same reference numbers are used in the drawings and description to refer to the same or like parts.
It is to be acknowledged that although the terms ‘first’, ‘second’, ‘third’, and so on, may be used herein to describe various elements, these elements should not be limited by these terms. These terms are used only for the purpose of distinguishing one component from another component. Thus, a first element discussed herein could be termed a second element without altering the description of the present disclosure. As used herein, the term “or” comprises any and all combinations of one or more of the associated listed items.
It will be acknowledged that when an element or layer is referred to as being “on,” “connected to” or “coupled to” another element or layer, it can be directly on, connected or coupled to the other element or layer, or intervening elements or layers may be present. In contrast, when an element is referred to as being “directly on,” “directly connected to” or “directly coupled to” another element or layer, there are no intervening elements or layers present.
In addition, unless explicitly described to the contrary, the word “comprise” and variations such as “comprises” or “comprising”, will be acknowledged to imply the inclusion of stated elements but not the exclusion of any other elements.
Please refer to
It is worthy to note that the maximum and the minimum values of the stride, the maximum and the minimum values of the pace, the maximum and the minimum values of the risk of falling, the maximum and the minimum values of the humpbacked value and the maximum and the minimum of the time length from the last sitting on the chair then to next standing up may be provided to the graphic processor 21 prior to the evaluation of the patient. The maximum and the minimum values of the stride, the pace, the risk of falling, the humpbacked value and the time length from the last sitting on the chair then to next standing up are respectively scaled proportionally from 0 to 100 for normalization of the measured values. The graphic processor 21 generates scores of the measured values of the stride, the pace, the risk of falling, the humpbacked value and the time length from the last sitting on the chair then to next standing up in proportion to each of the range of the respective maximum and the minimum values. Doctors may additionally customize weightings of the stride, the pace, the risk of falling, the humpbacked value and the time length from the last sitting on the chair then to next standing up. The graphic processor 21 performs a weight function on the stride, the pace, the risk of falling, the humpbacked value and the time length from the last sitting on the chair then to next standing up according to the weightings in order to acquire the overall combined health score HS.
The electronic device 20 may include a first database 23, a second database 24 and a third database 25. The first database 23 stores the health score HS. The second database 24 stores the 3D skeleton model DS and the moving diagram MD. The third database 25 stores the identification number. The first camera 10 may be a Kinect sensor and the electronic device 20 may be a desktop computer or a laptop computer. The electronic device 20 may be other devices with the function of processing, but is not limited thereto. It is worthy to mention that Azure Kinect may be used in the present invention as the first camera, for the high hardware performance of its GPU power, more data points in the point cloud may be provided for precision measurement, and a processing speed thereof may be increased dramatically, up to 30 times of speed, while comparing to conventional Kinect devices.
In the present embodiment, a pathway H, a sensor 30 and a workstation 40 may be arranged. The pathway H comprises a starting point SP and is adjacent to the chair C to provide the patient to walk from the starting point SP to the chair C. The handrails may be disposed on two sides of the pathway H according to the actual need, which may be beneficial for a disabled person to walk. For example, a size of the pathway H is 1.5 m in length and 0.6 m in width. The sensor 30 is electrically connected to the electronic device 20 and senses an identification card of the patient to acquire an identification number. The electronic device 20 receives the identification number. For example, the sensor 30 may be a radio frequency identification (RFID) sensor and the identification card of the patient may include an RFID tag. The workstation 40 may be in networked communication with the electronic device 20 and may be provided with a backup database 41. The backup database 41 may store a plurality of reference 3D skeleton models, each of which may include a plurality of reference coordinates defining a plurality of body segments. A plurality of the reference 3D skeleton models may be divided into a plurality of groups. Each group may have at least one reference 3D skeleton model and the plurality of reference coordinates thereof. The workstation 40 may be a server or a super computer. The workstation 40 may be other types of the electronic devices, but is not limited thereto.
Please refer to
When the indicator light is lit, the patient may start walking toward the chair C along the pathway H, the first camera 10 may then capture the walking progress of the patient as the first video F1. When the patient changes a posture from standing up to sitting on the chair C, the first camera 10 would capture the progress of changing posture of the patient as a second video F2. The first camera 10 transmits the first video F1 and the second video F2 to the electronic device 20. When the patient sits on the chair C, the chair C is able to measure weight of the patient and a screen SC displays weight of the patient. Afterwards, the graphic processor 21 may construct the 3D skeleton model DS and the moving diagram MD based on the first video F1. The graphic processor 21 would acquire the stride, the pace and the height variation based on the moving diagram MD. The graphic processor 21 may utilize the algorithm of gravity center GC to generate the original gravity center based on a moving path of the plurality of coordinates BC of the 3D skeleton model DS of the moving diagram MD. The graphic processor may further utilize the correction algorithm to calculate a variation of gravity center based on the original gravity center and the variation of the patient. The graphic processor 21 acquires a time length from the last sitting on the chair then to next standing up based on the second video F2. The graphic processor 21 generates a health score HS by performing a weight function on the scores of the measured stride, the pace, the humpbacked value, the risk of falling and the time length from the last sitting on the chair then to next standing up. Hereafter, the graphic processor 21 transmits the health score HS to the first database 23 and the backup database 41 and transmits the 3D skeleton model DS and the moving diagram MD to the second database 24 and the backup database 41.
And then, the workstation 40 may select the reference 3D skeleton model corresponding to the received 3D skeleton model DS and the plurality of reference coordinates thereof from the backup database 41. The workstation 40 may transmit the reference 3D skeleton model corresponding to the received 3D skeleton model DS and the plurality of reference coordinates thereof to the graphic processor 21. The graphic processor 21 would compare the received 3D skeleton model DS and the plurality of coordinates BC thereof with the corresponding reference 3D skeleton model and the plurality of reference coordinates thereof by the long short-term memory model or the transformer machine learning model to acquire a difference. The graphic processor 21 classifies the difference obtained from the long short-term memory model or the transformer machine learning model as one of the plurality of groups. The graphic processor 21 may then transmit the difference, the corresponding group, the health parameters (HP), the received 3D skeleton model DS and the plurality of coordinates BC thereof to the backup database 41. The backup database 41 may store the received 3D skeleton model DS, the health parameters (HP) and the plurality of coordinates BC thereof into the corresponding group. Alternatively, the graphic processor 21 may transmit the received 3D skeleton model DS, the health parameters (HP) and the plurality of coordinates BC thereof to the workstation 40. The workstation 40 may select the reference 3D skeleton model corresponding to the received 3D skeleton model DS, the health parameters (HP) and the plurality of reference coordinates thereof from the backup database 41. The processor of the workstation 40 may compare the received 3D skeleton model DS, the health parameters (HP) and the plurality of coordinates BC thereof with the corresponding reference 3D skeleton model and the plurality of reference coordinates thereof by the long short-term memory model or the transformer machine learning model to acquire the difference. The processor of the workstation 40 classifies the difference obtained from the long short-term memory model or the transformer machine learning model as one of the plurality of groups. The processor of the workstation 40 may store the received 3D skeleton model DS, the health parameters (HP) and the plurality of coordinates BC thereof into the corresponding group.
In another embodiment, the workstation 40 may be provided with a history database. The history database may store a history 3D skeleton model with a plurality of history coordinates or the health parameters (HP) related to abnormal symptom classes from the electric medical record ER. The history 3D skeleton model may be a 3D skeleton model generated and stored when the patient used the present invention in the past. The workstation 40 may select the history 3D skeleton model corresponding to the received 3D skeleton model DS, the health parameters (HP) and the plurality of history coordinates thereof from the history database 41. The workstation 40 may then transmit the history 3D skeleton model corresponding to the received 3D skeleton model DS, the health parameters (HP) and the plurality of history coordinates thereof to the graphic processor 21. The graphic processor 21 would calculate a similarity score between the plurality of received coordinates BC, the health parameters (HP) and the plurality of history coordinates by a transformer machine learning model or a long short-term memory model and the graphic processor 21 may then compare the similarity score with a threshold value. If the graphic processor 21 determines that the similarity score is greater than the threshold value, such similarity would indicate that the feet situation of the patient substantially unchanged prior to the treatment, and the doctor may modify the treatment of the patient based on the similarity score. If the graphic processor 21 determines that the similarity score is less than the threshold value, which would indicate that feet situation of the patient is in the process of recovery, and the doctor may further evaluate recovery situation of the patient based on the stride, the pace, the humpbacked value, the risk of falling, the time length from the last sitting on the chair then to next standing up and the similarity score. The doctor may determine whether the patient has a hunchback posture due to illness or not based on the height variation. The dynamic graph convolution neural network may then perform machine learning and more instantly calculate the similar score.
In other embodiment, the graphic processor 21 may extract the features from the first video F1 and the second video F2 to obtain the time length from the last sitting on the chair C then to next standing up, the humpbacked value, the risk of falling. The above features may combine with the stride and the pace to calculate the health score by performing the normalizing (with max height and z-score) and the weight function process. In some embodiments, the hand grip strength or the foot pressure point value can be added to determine the health score.
Finally, the workstation 40 may transmit the received 3D skeleton model DS, the health parameters (HP), the plurality of history coordinates thereof and the corresponding difference to the cloud platform, and the electronic device 20 may also transmit the health score HS to the cloud platform. The cloud platform may then transmit the received 3D skeleton model DS, the health parameters (HP), the plurality of history coordinates thereof, the corresponding variation and the health score HS to the screen DS. The screen SC may consequently display the received 3D skeleton model DS, the health parameters (HP), the plurality of history coordinates thereof, the corresponding variation and the health score HS to the patient.
Besides, the present invention may not only measure the 3D skeleton model DS and analyzes the gait, but also serve as a health score and variation automatic marking system for the other medical purposes, such as an evaluation system of rehabilitation medical assistive device, an evaluation system of sarcopenia behavior, an evaluation system of clinical trial of IPS and research of the other health score analysis or evaluation related to the body motion.
Please refer to
Please refer to
It is worthy to be mentioned that the graphic processor 21 illustrates curves related to the maximum stride per day, the maximum pace per day, the maximum risk of falling per day, the maximum humpbacked value per day and the shortest time length from the last sitting on the chair then to next standing up per day after the patient has used the present invention within a period of one month. It is beneficial for the doctor to observe the recovery situation of the patient. By using the long short-term memory model or the transformer machine learning model, the prediction of the behavior of the stroke patients may have an accuracy rate of about 90%.
Please refer to
Please refer to
Please refer to
The second camera 50 is electrically connected to the electronic device 20 and captures patient images of the patient. The medical care platform 60 is in networked communication with the electronic device 20 to receive the identification number and the health score HS. The medical care platform 60 transmits the electronic medical record ER to the electronic device 20 based on the identification number.
As shown in
If the patient is the male with pigeon chest, the graphic processor 21 obtains a chest part (i.e. chest skeleton) from the point cloud diagram CM and calculates the projection distance to ground (i.e. the pathway H) between a midpoint of the chest part and the midpoint of a rising edge of the chest part (i.e. a joint point between chest and neck). The front foot of the patient is regarded as the boundary point. The boundary point and the projection distance are input to the algorithm of modified gravity center. The graphic processor 21 calculates the modified gravity center ICG based on the moving path of the plurality of coordinates BC of the 3D skeleton model DS in the moving diagram MD, the projection distance and the boundary point and marks the modified gravity center ICG on the 3D skeleton model DS. If the patient is the female with pigeon chest, the calculation of the modified gravity center ICG about the female with pigeon chest is similar to the calculation of the modified gravity center ICG about the male with pigeon chest except the projection distance and the boundary point of the female are different these of the male.
If the patient is the male with protuberant abdomen, the graphic processor 21 obtains an abdomen part (i.e. abdomen skeleton) from the point cloud diagram CM and calculates the projection distance to ground (i.e. the pathway H) between a midpoint of the abdomen part and the midpoint of a rising edge of the chest part (i.e. a joint point between chest and neck). The front foot of the patient is regarded as the boundary point. The boundary point and the projection distance are input to the algorithm of modified gravity center. The graphic processor 21 calculates the modified gravity center ICG based on the moving path of the plurality of coordinates BC of the 3D skeleton model DS in the moving diagram MD, the projection distance and the boundary point and marks the modified gravity center ICG on the 3D skeleton model DS. If the patient is the female with pigeon chest, the calculation of the modified gravity center ICG about the female with pigeon chest is similar to the calculation of the modified gravity center ICG about the male with pigeon chest except the projection distance and the boundary point of the female are different these of the male.
Please refer to
In accordance with the above description, the intelligent gait analyzing apparatus of the present invention utilizes a first camera to capture a video in which the patient walks, sits down and stands up. Subsequently, the graphic processor acquires the stride, the pace, the humpbacked value, the risk of falling and the time length from the last sitting on the chair then to next standing up based on the video and utilizes the algorithm of gravity center to acquire an original gravity center. At last, the graphic processor calculates the health score by performing a weight function on the stride, the pace, the humpbacked value, the risk of falling and the time length from the last sitting on the chair then to next standing up. Doctor estimates recovery situation and muscle endurance of the patient.
The present invention disclosed herein has been described by means of specific embodiments. However, numerous modifications, variations and enhancements can be made thereto by those skilled in the art without departing from the spirit and scope of the disclosure set forth in the claims.
