This application claims the benefit of Taiwan application Serial No. 109138241, filed Nov. 3, 2020, the subject matter of which is incorporated herein by reference.
The invention relates in general to a guidance system and method, and more particularly to a guidance system and method for action postures.
In recent years, the demand for fitness coaches gradually increases due to the increase in sports population. Furthermore, along with the development in technology, many exercise-related digital audio/video contents and applications are provided so that people can do exercise and training at home.
When it comes to exercise and training, the most important issue is whether the action postures are correct. However, the exercise-related digital audio/video contents and applications currently available in the market only provide learning videos for the users to view but cannot provide any feedback with respect to the users' action postures. Therefore, the users can only view the videos for self-learning and cannot determine whether their action postures are correct or not and thus have no idea about how to make improvement.
Some applications can recognize the users' action postures but require building and training a model based on many action postures beforehand, and the cost and efforts for building such an action recognition model is huge. Most fitness coaches or persons who play exercise-related digital audio/video contents on the Internet cannot afford it. Besides, digital audio/video contents of different exercise may require different action recognition models. However, the fitness coaches or persons who play the digital audio/video contents on the Internet seek to diversify the exercise contents to attract people, and thus generate a large amount of digital audio/video contents. It is extremely difficult to build individual action recognition model for each digital audio/video content.
Therefore, it has become a prominent task for the industry to provide a convenient system and tool allowing fitness coaches or persons who play exercise-related digital audio/video contents on the Internet to edit the action posture guidance information with respect to the users' actions, which often require guidance, according to the standard exercise video. When the user is doing exercise and training according to the exercise video, the system and tool can provide corresponding action posture guidance information to the user whose action posture is determined as incorrect for helping the user to achieve a desired effect of exercise and training and to avoid having sport injury due to incorrect action postures.
The invention is directed to a guidance system and method for action postures capable of determining whether the user's action posture is correct according to the position, the angle and the speed of the action posture when the user is doing exercise or training, and providing the user with action posture guidance information which helps the user to avoid having incorrect action postures during the exercise or training process or ending up with an undesired effect or sports injury.
According to one embodiment of the present invention, a guidance system for action postures is provided. The guidance system includes a first skeleton recognition model, an editing device and a storage device. The first skeleton recognition model is used for generating a standard skeleton video according to a standard action posture video. The standard skeleton video includes a standard specific action posture of a person, and the standard skeleton video includes several skeleton images for standard specific action posture corresponding to the standard specific action posture. The editing device is used for receiving a command of an editor to generate at least one action posture guidance information corresponding to the skeleton images for standard specific action posture. The storage device is used for storing the skeleton images for standard specific action posture and the at least one action posture guidance information corresponding to the skeleton images for standard specific action posture.
According to another embodiment of the present invention, a guidance method for action postures is provided. The guidance method includes the following steps. A standard skeleton video is generated according to a standard action posture video, wherein the standard skeleton video includes a standard specific action posture of a person, and the standard skeleton video includes several skeleton images for standard specific action posture corresponding to the standard specific action posture. A command of an editor is received to generate at least one action posture guidance information corresponding to the skeleton images for standard specific action posture. The skeleton images for standard specific action posture and the at least one action posture guidance information corresponding to the skeleton images for standard specific action posture are stored.
The above and other aspects of the invention will become better understood with regard to the following detailed description of the preferred but non-limiting embodiment(s). The following description is made with reference to the accompanying drawings.
Referring to
In step S110, a standard skeleton video SSV is generated by the first skeleton recognition model 110 according to a standard action posture video SV. The standard action posture video SV includes a standard specific action posture of a person, and the standard skeleton video SSV includes several skeleton images for standard specific action posture SSP corresponding to the standard specific action posture. Referring to
In step S120, a command CMD of an editor is received by the editing device 120 to generate at least one action posture guidance information GI corresponding to the skeleton image for standard specific action posture SSP. Each action posture guidance information GI includes at least one of a position guidance information, an angle guidance information, a time guidance information and a speed guidance information, or any combination of the above guidance information. Referring to
In step S130, the skeleton images for standard specific action posture SSP and the action posture guidance information GI corresponding to the skeleton images for standard specific action posture are stored to the storage device 130. The storage device 130 includes a guidance knowledge base 131 storing the skeleton images for standard specific action posture SSP and the action posture guidance information GI corresponding to the skeleton images for standard specific action posture. For example, the guidance knowledge base 131 stores the skeleton image for standard specific action posture SSP3 and the action posture guidance information GI corresponding to the skeleton image for standard specific action posture SSP3. In an embodiment, the guidance knowledge base 131 may store several skeleton images for standard specific action posture (such as SSP1, SSP2 and SSP3) and the action posture guidance information GI corresponding to the skeleton images for standard specific action posture.
Referring to
In step S210, a user action posture of a user U is captured by a capturing device 140 to generate a user live streaming UV.
In step S220, a user action posture skeleton video USV is generated by the second skeleton recognition model 150 according to the user live streaming UV. The user live streaming UV includes a user specific action posture of the user U; the user action posture skeleton video USV includes several skeleton pictures for user specific action posture USP corresponding to the user specific action posture. Referring to
In step S230, the skeleton picture for user specific action posture USP is compared with the skeleton image for standard specific action posture SSP by the skeleton comparison module 160 to generate a comparison result CR. The comparison result CR includes a position comparison result, an angle comparison result, a time comparison result or a speed comparison result. The comparison result CR can also be any combination of the above comparison results. Furthermore, the skeleton comparison module 160 compares the position and angle between the skeleton USK in the user specific action posture skeleton picture USP and the skeleton SK in the skeleton image for standard specific action posture SSP to obtain a position comparison result and an angle comparison result, respectively. The skeleton comparison module 160 can obtain the start and the ending skeleton pictures of the user specific action posture of the user from several skeleton pictures for user specific action posture. For example, the user specific action posture is a user holding the dumb bells vertically and then extending his/her arms to be parallel to the ground. The user specific action posture skeleton picture USP1 is the starting skeleton picture; the user specific action posture skeleton picture USP, is the ending skeleton picture. The duration for which the user performs the specific action can be calculated according to the user live streaming UV. The distance by which the user extends his/her hands can be calculated according to the positions of the user's palms at the skeletons in the skeleton pictures for user specific action posture USP1 and USPn, respectively. The user's speed can be calculated according to the above time and distance. The user's time and speed can be compared with the time guidance information and the speed guidance information to obtain a time comparison result and a speed comparison result.
Referring to
Referring to
In step S240, one of the action posture guidance information GI is selected from the guidance knowledge base 131 by the action guidance module 170 according to the comparison result CR. When the action guidance module 170 determines that the position comparison result (60 pixels) exceeds a position threshold (such as 50 pixels), the action guidance module 170 selects the position guidance information “The arms should be parallel to the ground” corresponding to the skeleton image for standard specific action posture SSP3 from the guidance knowledge base 131. When the action guidance module 170 determines that the angle comparison result (30°) exceeds an angle threshold (such as 5°), the action guidance module 170 selects the angle guidance information “Each shoulder joint needs to form an angle of 90° with the torso” corresponding to the skeleton image for standard specific action posture SSP3 from the guidance knowledge base 131. When the action guidance module 170 determines that the time comparison result (such as 1 second) exceeds a time threshold (such as 0.5 seconds), the action guidance module 170 selects a time guidance information “Keep the arms parallel for 2 seconds” corresponding to the skeleton image for standard specific action posture SSP3 from the guidance knowledge base 131. When the action guidance module 170 determines that the speed comparison result (such as 0.5 m/s) exceeds a speed threshold (such as 0.4 m/s), the action guidance module 170 selects the speed guidance information “Control the arm action to a speed of 0.4 m/s” corresponding to the skeleton image for standard specific action posture SSP3 from the guidance knowledge base 131. Furthermore, corresponding feedback information “Please slow down” or “Please speed up” in response to the speed being too fast or too slow can be created. When the action guidance module 170 determines that the position comparison result does not exceed the position threshold, the angle comparison result does not exceed the angle threshold, the time comparison result does not exceed the time threshold, and the speed comparison result does not exceed the speed threshold, the action guidance module 170 does not select the action posture guidance information GI. It should be noted that the position threshold, the angle threshold, the time threshold and the speed threshold of the present invention are not limited to the above exemplifications and can be adjusted according to the action posture.
In an embodiment, the guidance system for action postures 100 further includes an exercise repetition calculation module 190. The exercise repetition calculation module 190 can be a processing circuit or a processing chip. When the position comparison result does not exceed the position threshold and the angle comparison result does not exceed the angle threshold, the exercise repetition calculation module 190 records an exercise repetition.
In step S250, the selected action posture guidance information GI is shown on the display 180. In step S240, if the action posture guidance information GI selected from the guidance knowledge base 131 by the action guidance module 170 is the position guidance information “The arms should be parallel to the ground”, the display 180 shows “the arms should be parallel to the ground”. If the action posture guidance information GI selected from the guidance knowledge base 131 by the action guidance module 170 is the angle guidance information “Each shoulder joint needs to form an angle of 90° with the torso”, the display 180 shows “each shoulder joint needs to form an angle of 90° with the torso”. If the action posture guidance information GI selected from the guidance knowledge base 131 by the action guidance module 170 is the speed guidance information “The arm action needs to be lasted for 2 seconds”, the display 180 shows “The arm action needs to be lasted for 2 seconds”. If the action guidance module 170 does not select the action posture guidance information GI, the display 180 shows a prompt information “Perfect action” to indicate that the user's action posture is correct.
Thus, the guidance system and method for action postures of the present invention are capable of determining whether the user's action posture is correct according to the position, the angle and the speed of the action posture when the user is doing exercise or training, and providing the user with action posture guidance information which helps the user to avoid having incorrect action posture during the exercise or training process or ending up with an undesired effect or sports injury.
While the invention has been described by way of example and in terms of the preferred embodiment(s), it is to be understood that the invention is not limited thereto. On the contrary, it is intended to cover various modifications and similar arrangements and procedures, and the scope of the appended claims therefore should be accorded the broadest interpretation so as to encompass all such modifications and similar arrangements and procedures.
Number | Date | Country | Kind |
---|---|---|---|
109138241 | Nov 2020 | TW | national |