SCREEN CLIMBING SYSTEM BASED ON MOMENTUM MEASUREMENT

Information

  • Patent Application
  • 20250177832
  • Publication Number
    20250177832
  • Date Filed
    March 03, 2023
    2 years ago
  • Date Published
    June 05, 2025
    a month ago
  • Inventors
    • SUNG; KWANG JE
  • Original Assignees
    • ASPORTZ CO., LTD.
Abstract
A screen climbing system includes a screen in which holds are coupled to an inclined surface formed vertically or at a predetermined slope and on which a climbing image is projected and displayed, an image output device that enlarges the climbing image and projects the enlarged climbing image on the screen, a motion recognition device that recognizes an exerciser climbing the screen according to the climbing image to calculate motion information including position, posture and movement, and a smart climbing analysis device that tracks displacement of each body joint of the exerciser based on the motion information, calculates energy metabolism for the displacement as momentum of each body joint, reflects momentum information in the climbing image based on the calculated momentum, and transmits the momentum information to the image output device.
Description
BACKGROUND
1. Technical Field

The present disclosure relates to a screen climbing system based on momentum measurement, and more particularly, to a technology capable of tracking displacement of a body joint of an exerciser, calculating momentum, and providing momentum information and a customized exercise program to a climbing image (climbing game) based on the momentum.


2. Background Art

As interest in health increases, climbing, which is only performed on natural rock walls, is rapidly gaining attention as a popular sport that enables people to improve their physical strength by climbing walls using not only their hands and feet but also their entire body while holding onto holds attached to various artificial structures indoors. The climbing may help reduce body fat due to high energy consumption, along with increasing physical strength such as cardiovascular function, flexibility, strength, and agility, and may also be effective in improving cardiopulmonary endurance and muscular endurance due to the nature of climbing exercise that relies on physical strength and skills to climb against gravity.


In particular, in recent years, a screen-based climbing system is being developed that minimizes the risks of existing climbing and allows for safer enjoyment by adding sports and game elements. A representative example of the screen-based climbing system may include an augmented reality (AR) screen climbing capable of implementing a motion-based game through the AR screen so that not only adults but also children may participate in the exercise. This AR screen climbing may be implemented by detecting the movement of the exerciser's body using various sensors or devices and reflecting the movement of the exerciser's body in the games using the body directly instead of tools such as a keyboard or a mouse.


However, the conventional screen climbing system uses a position tracking technology that applies image tracking, pressure sensors for each hold, optical sensors, etc. Here, the image tracking has the limitation that it is difficult to recognize accurate positions, and the position tracking by the sensors for each hold and the optical sensors requires a large number of relatively expensive sensors and sensor circuits, so there is a problem in that the initial installation cost is high.


Therefore, there is a demand for the development of a new screen climbing technology capable of producing exerciser's exercise information with a detailed step-by-step algorithm based on general motion information without expensive and complex devices.


SUMMARY

The present disclosure is intended to solve various problems of the above-mentioned conventional technology, and is to measure momentum of each body joint by tracking displacement of a body joint of an exerciser during screen climbing.


In addition, the present disclosure provides a climbing image (climbing game) that reflects momentum information and customized exercise program information.


According to an aspect of the present disclosure, a screen climbing system includes: a screen in which a plurality of holds are coupled to an inclined surface formed vertically or at a predetermined slope and on which a climbing image is projected and displayed; an image output device that enlarges the climbing image and projects the enlarged climbing image on the screen; a motion recognition device that recognizes an exerciser climbing the screen according to the climbing image to calculate motion information including position, posture and movement; and a smart climbing analysis device that tracks displacement of each body joint of the exerciser based on the motion information, calculates energy metabolism for the displacement as momentum of each body joint, reflects momentum information in the climbing image based on the calculated momentum, and transmits the momentum information to the image output device.


The smart climbing analysis device may include: a joint


analysis unit that generates virtual body points for each body joint of the exerciser based on the motion information to track and calculate the displacement; a momentum analysis unit that matches the energy metabolism of the exerciser to the displacement to calculate the momentum of each body joint of the exerciser; a customized climbing unit that reflects momentum of a specific body joint in the climbing image to increase the momentum of the specific body joint according to exercise achievement and an exercise level according to the momentum or reflects the momentum of the specific body joint in the climbing image to increase the momentum of the specific body joint according to a selection of the exerciser, among the body joints of the exerciser; and an exerciser management unit that manages personal information of the exerciser, an exercise time, the number of exercise sessions, and exercise result information.


The joint analysis unit may configure a skeleton divided into the virtual body points matching each body joint of the exerciser to obtain displacement amount of each body joint being tracked by the skeleton of the exerciser, obtain a current position of the body joint being tracked by the skeleton of the exerciser, obtain an absolute value of the displacement amount of the body joint being tracked by the skeleton of the exerciser, and round off and organizes the absolute value of the displacement amount of the body joint being tracked by the skeleton of the exerciser.


The momentum analysis unit may reflect a value obtained by rounding off the absolute value of the displacement amount of the body joint being tracked by the skeleton of the exerciser in a numerical value of the body joint, convert the numerical value to match a color of the image so as to express the momentum of the body joint being tracked by the skeleton of the exerciser, and reflect the numerical value in color property of the momentum image of the body joint being tracked by the skeleton of the exerciser.


The momentum analysis unit may further perform a process of obtaining an average by dividing a sum of the displacement amounts of the body joints being tracked by the skeleton by a specific numerical value to obtain an average momentum of each body joint, and the specific numerical value may be a natural number obtained by dividing a sum of total momentum of the exerciser to be the same as or close to a result of calculating the momentum of the exerciser as momentum per hour during the climbing image time.


The screen climbing system may further include a respiratory gas analysis device that is worn by the exerciser to measure respiratory gas and analyze the energy metabolism. The respiratory gas analysis device may measure at least one of peak oxygen uptake (VO2peak) and respiratory exchange rate (RER) of the exerciser, analyze the energy metabolism including any one of exercise intensity (Mets) and energy consumption per hour (Kcal/h), and additionally measure and analyze maximal heart rate (HRmax).


The climbing image may be a motion-based augmented reality (AR) game in which the exerciser climbs the inclined surface of the screen using a plurality of holds.


In addition, the present disclosure is also achieved by other systems, methods, and computer-readable recording media for recording a computer program for executing the method.


According to the present disclosure, by tracking the displacement of the body joint of the exerciser during the screen climbing and analyzing the energy metabolism through the respiratory gas analysis, it is possible to accurately measure the momentum of each body joint.


In addition, according to the present disclosure, by providing the climbing image (climbing game) that reflects the momentum information and the customized exercise program, it is possible to improve the interest of not only adults but also children, promote health, and expect the advantages of both exercise and game.





BRIEF DESCRIPTION OF THE DRAWINGS


FIG. 1 is a diagram illustrating a configuration of a screen climbing system based on momentum measurement according to an embodiment of the present disclosure.



FIG. 2 is a diagram illustrating in detail an internal configuration of a smart climbing analysis device according to an embodiment of the present disclosure.



FIG. 3 is a flowchart of the overall process of screen climbing based on momentum measurement according to an embodiment of the present disclosure.



FIGS. 4A to 4E are screens illustrating an example of screen climbing according to an embodiment of the present disclosure.





DETAILED DESCRIPTION

Embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. It should be appreciated that various embodiments of the present disclosure are different from each other, but do not have to be exclusive. Combinations of each block of the accompanying configuration diagram (block diagram) and each step of the accompanying flow chart herein may also be performed by computer program instructions. Each block or each step may indicate some of modules, segments, or codes including one or more executable instructions for executing a specific logical function(s). Further, it is to be noted that functions mentioned in the blocks or the steps occur regardless of a sequence in some alternative embodiments. For example, two blocks or steps that are shown continuously or in order may be simultaneously performed in fact or be performed in a reverse sequence depending on corresponding functions.


In addition, it should be understood that positions or an arrangement of individual components in each disclosed embodiment may be changed without departing from the spirit and the scope of the present disclosure. Therefore, a detailed description to be described below should not be interpreted as a restrictive meaning. In addition, the scope of the present disclosure should be interpreted to encompass the full scope equivalent to what the claims assert. In addition, similar reference symbols in the drawings refer to the same or similar functions across various aspects, and their shapes may be exaggerated for convenience.


Terms used herein are used in order to appropriately represent preferred embodiments of the present disclosure and may be construed in different ways according to the intention of users or operators, customary practice in the art to which the present disclosure pertains. Therefore, the definitions of terms used in the present description should be construed based on the contents throughout the specification. Throughout the specification, unless otherwise specified, “including” any component means that other components may be further included rather than excluding other components. In addition, terms “ . . . unit,” “module,”, and the like, described in the specification refer to a processing unit of at least one function or operation and may be implemented by hardware or software or a combination of hardware and software.


Hereinafter, preferred embodiments of the present disclosure will be described in detail with reference to the accompanying drawings so that those skilled in the art may easily practice the present disclosure.


Configuration of System


FIG. 1 is a diagram illustrating a configuration of a screen climbing system based on momentum measurement according to an embodiment of the present disclosure.


Referring to FIG. 1, the entire system according to an embodiment of the present disclosure may be configured to include a communication network 100, a screen 200, an image output device 300, a motion recognition device 400, a respiratory gas analysis device 500, and a smart climbing analysis device 600.


First, the communication network 100 according to an embodiment of the present disclosure may be configured as a network capable of performing a series of data transmission and reception operations for data transmission and information exchange between the screen 200, the image output device 300, the motion recognition device 400, the respiratory gas analysis device 500, and the smart climbing analysis device 600.


More specifically, the communication network 100 may be any one of a local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), an integrated services digital network (ISDN), a wireless LAN, a mobile communication network, wireless fidelity, Bluetooth, ZigBee, etc., or may be a variety of forms of wired and wireless communication networks that may be connected through a combination of these, but this is only an example, and the present disclosure is not limited thereto, and any known communication technology may be adopted and used without limitation.


Next, the screen 200 according to an embodiment of the present disclosure may be a device that is configured by attaching a plurality of holds 220 to an inclined surface 210 formed vertically or at a predetermined slope, and on which a climbing image is projected and displayed.


More specifically, the screen 200 may be an artificial rock wall structure formed by coupling the plurality of holds 220 with the inclined surface 210 formed vertically or at a predetermined slope so that exercise functions and game functions may be combined, and may also perform a screen function of projecting and displaying a climbing image including exercise information (e.g., momentum information, customized exercise program information, etc.) and game information (e.g., character, item, level, score information, etc.) projected from the image output device 300 described below.


Next, the image output device 300 according to an embodiment of the present disclosure may be a digital device including a function of enlarging and projecting the climbing image on the screen 200.


More specifically, the image output device 300 may be installed on an upper portion or a ceiling of a climbing space facing the screen 200 and may project the climbing image by mapping the climbing image to the screen 200. For example, the image output device 300 may perform a beam projector function that may guide a path by projecting a change position of the hold 220 that exerciser's hands and feet touch or grasp sequentially as text or images to enlarge and project the climbing image on the screen 200.


Next, the motion recognition device 300 according to an embodiment of the present disclosure may be a digital device that includes a function that may recognize an exerciser climbing the screen 200 according to the climbing image and calculate motion information including position, posture, and movement.


More specifically, the motion recognition device 300 may be a device that may recognize the position, posture, and movement of the exerciser on the screen 200 and may use, for example, an optical sensor method using an infrared sensor, a pressure sensor method, an image method using a camera, etc. In the present disclosure, describing an example of the image method, a distance from screen 200 is measured in advance and input, and then a distance from an exerciser is measured and a difference in depth is calculated to separately recognize the screen 200 and the exerciser. In this case, the motion recognition device 300 may be programmed to detect the operation of the exerciser in real time, and may include an infrared illuminator (IR), a three-dimensional (3D) camera, and an RGB camera that are used to capture a depth image.


For example, the inclined surface 210 and the entire hold 220 of the screen 200 may be captured, and the captured image may be used as position detection information to detect the exerciser's hands and feet, and the captured image in which the hands and feet are detected may match the pre-stored reference image of the inclined surface 210 of the corresponding screen 200 to detect the hold 220 of a part where the hands and feet of the corresponding exerciser are positioned.


In addition, the motion recognition device 300 may check the position of the hold 320 that the hand is holding and the foot is touching while the detected exerciser is moving. The motion information including the position and movement path of the exerciser detected by the motion recognition device 300 and the position of the hold 220 that is in contact with the hand and foot may be transmitted to the climbing analysis device 600 described later. Meanwhile, since the optical sensor method and the pressure sensor method may apply known technologies without limitation, a detailed description thereof will be omitted in the present disclosure.


Next, the respiratory gas analysis device 500 according to an embodiment of the present disclosure may be a digital device that includes a function of measuring respiratory gas by being worn by the exerciser and analyzing energy metabolism of the exerciser.


More specifically, the respiratory gas analysis device 500 may be the known respiratory measurement device, and in order to measure the exercise intensity, the exerciser may wear a portable respiratory gas analyzer in the form of a mask during climbing to measure maximal oxygen uptake (VO2peak), respiratory exchange rate (RER), exercise intensity (Mets), and energy consumption per hour (Kcal/h), and additionally, the known heart rate measurement function may be added to further measure maximal heart rate (HRmax). That is, by measuring any one of the peak oxygen uptake (VO2peak) and respiratory exchange rate (RER) of the exerciser, it is possible to analyze the energy metabolism including any one of the exercise intensity (Mets) and the energy consumption per hour (Kcal/h), and additionally measure and analyze the maximal heart rate (HRmax).


In this case, the maximal heart rate (HRmax) may refer to the maximal heart rate that the exerciser may achieve for 1 minute, and the maximal oxygen uptake (VO2peak) may refer to the maximal oxygen uptake that the exerciser may achieve by increasing exercise intensity. In addition, the respiratory exchange ratio (RER) may refer to the ratio of the amount of oxygen inhaled and the amount of carbon dioxide exhaled for 1 minute, and the exercise intensity (Mets) may refer to the amount of oxygen consumed (oxygen uptake) per minute as momentum in unit time. In addition, the energy consumption (Kcal/h) per hour may refer to calorie per hour consumed by the exerciser.


Finally, the smart climbing analysis device 600 according to an embodiment of the present disclosure may be a digital device that includes functions of tracking displacement of each body joint of the exerciser based on the motion information and calculating energy metabolism for the displacement as momentum of each body joint, reflecting momentum information in the climbing image based on the calculated momentum, and transmitting the momentum information to the image output device.


More specifically, the smart climbing analysis device 600 may track the displacement of the body joint of the exerciser during the screen climbing and analyze the energy metabolism through the respiratory gas analysis, thereby accurately measuring the momentum of each body joint. In addition, by providing the climbing image (climbing game) that reflects the momentum information and the customized exercise program information, it is possible to improve the interest of not only adults but also children, promote health, and expect the advantages of both exercise and game. The smart climbing analysis device 600 according to an embodiment of the present disclosure may be understood more specifically by the following detailed description with reference to FIG. 2.


Meanwhile, the smart climbing analysis device 600 is a digital device that includes memory means and a microprocessor, and has computational capabilities. As long as the smart climbing analysis device 600 includes a communication function, the smart climbing analysis device 600 may be adopted as a terminal device according to the present disclosure.


In addition, although the motion recognition device 400, the respiratory gas analysis device 500, and the smart climbing analysis device 600 are illustrated as being configured separately in FIG. 1 that illustrates an embodiment of the present disclosure, it will be obvious to those skilled in the art that some or all of them may be integrated, if necessary, when implementing the present disclosure.


Configuration of Smart Climbing Analysis Device 600 In the following detailed description, the internal configuration of the smart climbing analysis device 600, which performs important functions for implementing the present disclosure, and the functions of each component will be examined.



FIG. 2 is a diagram illustrating in detail the internal configuration of the smart climbing analysis device 600 according to an embodiment of the present disclosure.


Referring to FIG. 2, the smart climbing analysis device 600 according to an embodiment of the present disclosure may be configured to include a joint analysis unit 610, a momentum analysis unit 620, a customized climbing unit 630, an exerciser management unit 640, a database 650, a communication unit 660, and a control unit 670.


According to an embodiment of the present disclosure, at least some of the joint analysis unit 610, the momentum analysis unit 620, the customized climbing unit 630, the exerciser management unit 640, the database 650, the communication unit 660, and the control unit 670 may be program modules that transmit and receive data with the screen 200, the image output device 300, the motion recognition device 400, and the respiratory gas analysis device 500. These program modules may be included in the smart climbing analysis device 600 in the form of an operating system, application program modules, and other program modules, may be physically stored on various known memory devices, and may be implemented as hardware components (e.g., general-purpose processors, dedicated processors) and/or software components (e.g., firmware, applications, program modules) and combinations thereof. In addition, these program modules may be stored in a remote memory device that can communicate with the smart climbing analysis device 600. Meanwhile, these program modules include, but are not limited to, a routine, a sub-routine, a program, an object, a component, a data structure, or the like, performing a specific task or executing a specific abstract data type to be described below according to the present disclosure.


First, the joint analysis unit 610 according to an embodiment of the present disclosure may perform a function of generating virtual body points for each body joint of exerciser based on motion information provided (transmitted) by the motion recognition device 400 and tracking and calculating displacement.


More specifically, the joint analysis unit 610 may recognize an exerciser who climbs using the holds 220 coupled to the inclined surface 210 of the screen 200 according to the climbing image, generates virtual body points for each detected body joint (e.g., head, wrist, ankle, etc.) of the exerciser based on the motion information including position, posture, and movement, and tracks the position of the body point while the exerciser moves to calculate the displacement thereof.


For example, the body of the exerciser may be composed of a skeleton that divides the body into a plurality of bones (frames), each body joint may be divided into the matching body point, and the displacement amount of each body joint (body point) may be tracked and calculated.


The process of tracking and calculating the displacement amount may include obtaining the displacement amount of the body joint (body point) being tracked by the skeleton of the exerciser, and then obtaining the current position of the body joint (body point) being tracked by the skeleton of the exerciser. Next, the absolute value of the displacement amount of the body joint (body point) being tracked by the skeleton of the exerciser may be obtained, and the absolute value of the displacement amount of the body joint (body point) being tracked by the skeleton of the exerciser may be rounded off to be calculated. The specific description of the joint analysis unit 610 according to an embodiment of the present disclosure may be understood more specifically by the detailed description below with reference to FIG. 3. Next, the momentum analysis unit 620 according to an embodiment of the present disclosure may perform the function of matching the energy metabolism of the exerciser analyzed by the respiratory gas analysis device 500 to the displacement of each body joint of the exerciser calculated by the joint analysis unit 610 to calculate the momentum of each body joint of the exerciser.


More specifically, the momentum analysis unit 620 may analyze the energy metabolism including any one of the peak oxygen uptake (VO2peak), respiratory exchange rate (RER), exercise intensity (Mets), and energy consumption per hour (Kcal/h) of the exerciser measured by the respiratory gas analysis device 500, additionally measure and analyze the maximal heart rate (HRmax), and may match the analyzed energy metabolism of the exerciser to the displacement of each body joint of the exerciser to calculate the momentum of each body joint.


In addition, the process of calculating the momentum in the momentum analysis unit 620 may include reflecting a value obtained by rounding off the absolute value of the displacement amount of the body joint (body point) being tracked by the skeleton of the exerciser in the numerical value (momentum numerical value) of the body joint, and converting the numerical value to match the color of the image in order to express the momentum of the body joint (body point) being tracked by the skeleton of the exerciser as an image. Next, the momentum may be calculated by performing the process of reflecting the numerical value in the color property of the momentum image of the body joint (body point) being tracked by the skeleton of the exerciser.


In this case, the momentum analysis unit 620 may further perform a process of obtaining an average by dividing a sum of the displacement amounts of the body joints (body points) being tracked by the skeleton by the number of body joints (body points) and obtaining the average momentum for each body joint. In addition, the momentum analysis unit may further perform a process of obtaining an average by dividing a sum of the displacement amounts of the body joints (body points) being tracked by the skeleton by a specific numerical value to obtain an average momentum of each body joint, and the specific numerical value may be a natural number obtained by dividing a sum of total momentum of the exerciser to be the same as or close to a result of calculating the momentum of the exerciser as momentum per hour during the climbing image time.


The description of the momentum analysis unit 620 according to an embodiment of the present disclosure may be understood more specifically by the following detailed description with reference to FIG. 3.


Next, the customized climbing unit 630 according to an embodiment of the present disclosure may perform a function of reflecting momentum of a specific body joint in the climbing image to increase the momentum of the specific body joint according to exercise achievement and an exercise level according to the momentum or reflecting the momentum of the specific body joint in the climbing image to increase the momentum of the specific body joint according to a selection of the exerciser, among the body joints of the exerciser.


More specifically, this customized climbing unit 630 may visually display the achievement according to momentum among the body joints of the exerciser, and also check and display the part that lacks momentum. In addition, as the exercise level (STAGE) increases, in order to strengthen the specific body joint where the momentum lacks, the customized climbing unit 630 may be programmed to increase the momentum of the corresponding joint and provide the climbing image (climbing game) that reflects the momentum of the corresponding joint.


In addition, the customized climbing unit 630 may be programmed to increase the momentum of the specific body joint according to the selection of the exerciser by allowing the exerciser to designate (select) the body joint part that he or she wants to strengthen in advance, and provide the climbing image (climbing game) that reflects the momentum of the specific body joint. The customized climbing device 630 according to an embodiment of the present disclosure may be understood more specifically by the following detailed description with reference to FIGS. 4A to 4E.


Next, the exerciser management unit 640 according to an embodiment of the present disclosure may perform a function of managing personal information, an exercise time, the number of exercise sessions, and exercise result information of the exerciser.


More specifically, the exerciser management unit 640 may manage the personal information (e.g., subscription information, login information, health information, etc.) and the exercise time (daily/monthly exercise time management), the number of exercise sessions (exercise achievement management), and exercise result information (energy metabolism, momentum management) of the exerciser during the screen climbing according to the climbing image (climbing game), thereby providing an exercise course and program suitable for the corresponding user.


Next, the database 650 according to an embodiment of the present disclosure may be a storage device capable of storing climbing image information, exerciser information, exercise records, exercise information (e.g., momentum information, customized exercise program information, etc.), game information (e.g., character, item, level, score information, etc.), etc. In addition, the database 650 may further store access (login) information, information transmission and reception history, etc., of the screen 200, the image output device 300, the motion recognition device 400, and/or the respiratory gas analysis device 500. This database 650 is a concept that includes a computer-readable recording medium, may refer not only to a database in a narrow sense, but also to a database in a broad sense including data records based on a file system, and may be included within the scope of the database referred to in the present disclosure as long as a simple set of logs may be searched and data may be extracted.


Next, the communication unit 660 according to an embodiment of the present disclosure may perform a function that allows the internal components of the smart climbing analysis device 600 to transmit and receive data (information) with external devices such as the screen 200, the image output device 300, the motion recognition device 400, and the respiratory gas analysis device 500. The known wired/wireless communication method may be adopted without limitation. Finally, the control unit 670 according to an embodiment of the present disclosure may perform a function of controlling the flow of data between the joint analysis unit 610, the momentum analysis unit 620, the customized climbing unit 630, the exerciser management unit 640, the database 650, and the communication unit 660. That is, the control unit 670 according to the present disclosure may control the flow of data from the outside or between each component of the smart climbing analysis device 600 so that each of the joint analysis unit 610, the momentum analysis unit 620, the customized climbing unit 630, the exerciser management unit 640, the database 650, and the communication unit 660 may perform their own functions.


In order to help understand the smart climbing analysis device 600 according to the present disclosure described above, the following detailed description will describe the screen climbing process (method) by way of an example.


Screen Climbing Process


FIG. 3 is a flowchart of the overall process of screen climbing based on momentum measurement according to an embodiment of the present disclosure.


Referring to FIG. 3, first, the process of tracking and calculating the displacement amount may perform a process (S10) of obtaining the displacement amount of the body joint (body point) being tracked by the skeleton of the exerciser.


Next, a process (S20) of obtaining the current position of the body joint (body point) being tracked by the skeleton of the exerciser may be performed.


Next, a process (S30) of obtaining the absolute value of the displacement amount of the body joint (body point) being tracked by the skeleton of the exerciser may be performed.


Next, a process (S40) of rounding off the absolute value of the displacement amount of the body joint (body point) being tracked by the skeleton of the exerciser may be performed.


Next, as a process for calculating the momentum, a process (S50) of reflecting a value obtained by rounding off the absolute value of the displacement amount of the body joint (body point) being tracked by the skeleton of the exerciser in the numerical value (momentum numerical value) of the body joint may be performed.


Next, a process (S60) of converting the numerical value to match the color of the image in order to express the momentum of the body joint (body point) being tracked by the skeleton of the exerciser as an image may be performed.


Next, a process (S70) of reflecting the numerical value in the color property of the momentum image of the body joint (body point) being tracked by the skeleton of the exerciser may be performed to calculate the momentum.


In this case, a process of obtaining the average by dividing the sum of the displacement amounts of the body joints (body points) being tracked by the skeleton by the number of body joints (body points) and obtaining the average momentum for each body joint may be further performed. In addition, the momentum analysis unit may further perform a process of obtaining an average by dividing a sum of the displacement amounts of the body joints (body points) being tracked by the skeleton by a specific numerical value to obtain an average momentum of each body joint. In this case, the specific numerical value may be a natural number obtained by dividing a sum of total momentum of the exerciser to be the same as or close to a result of calculating the momentum of the exerciser as momentum per hour during the climbing image time.


As an example of the program of the process of calculating the displacement amount and the momentum, the process S10 may be coded by being programmed with “currDeltal =current0. transform.position.x-currPos1;”, the process S20 may be coded by being programmed with “currPos1=current0.transform.position.x;”, the process S30 may be coded by being programmed with “displacement1+=Mathf.Abs(currDelta1);”, the process S40 may be coded by being programmed with “displacement1=Mathf.Round (displacement0* 10)*0.1f;”, the process S50 may be coded by being programmed with “Camera.main. GetComponent<testMeasureController>( ).fBody01=displacement1;”, the process S60 may be coded by being programmed with “fColor1 =0.1f+(fBody01*10/255);”, and the process S70 may be coded by being programmed with “body01. color=new Color (1, 0, 0, fColor1);”, and the process of calculating the average momentum for each body joint may be coded by being programmed with “float ftotal=(fBody01+fBody01 1_fBody01_2+fBody01_3+fBody02+fBody02_1 +fBody02_2+fBody02_3+fBody03+fBody03_1+fBody03_2+fBody03_330 fBody04+fBody04_1+fBody04_2+fBody04_3) /17 (the number of body joints or specific numerical value);”.


Example of Screen Climbing

In the detailed description below, in order to implement the screen climbing according to the present disclosure, a case is described as an example in which a ‘Kinect V2’ module is applied to the motion recognition device 400 to recognize 25 body joints and track 16 joints among these 25 body joints. However, this is described as the most representative process for convenience of description, and the present disclosure is not limited thereto.


In this case, 16 body joints may be HandLeft (left hand), WristLeft (left wrist), ElbowLeft (left elbow), ShoulderLeft (left shoulder), HandRight (right hand), WristRight (right wrist), (right ElbowRight elbow), ShoulderRight (right shoulder), FootLeft (left foot), AnkleLeft (left ankle), KneeLeft (left knee), HipLeft (left hip), FootRight (right foot), AnkleRight (right ankle), KneeRight (right knee), and HipRight (right hip).



FIGS. 4A to 4E are screens illustrating an example of screen climbing according to an embodiment of the present disclosure.


First, FIG. 4A is a screen displaying a skeleton body in a climbing image (climbing game) using AR screen climbing according to an embodiment of the present disclosure, FIG. 4B is a screen in which the skeleton body is removed from the climbing image, and FIG. 4C is a screen displaying a skeleton body currently being tracked in the climbing image, where a white circle W may represent a joint currently being tracked, and a gray circle B may represent a joint that is recognized but not tracked. In this case, the movement displacement amount of the joint being tracked may be accumulated and recorded every 1/60 second (however, there may be differences depending on the performance of the measuring device).Next, FIG. 4D displays the joint part and name of the skeleton currently being tracked in the climbing image, and FIG. 4E illustrates an exemplary calorie (momentum) measurement result (adult playing a game for 99 seconds) applied to the ‘Everest game’ among the climbing images (climbing games). Therefore, according to the present disclosure, by tracking the displacement of the body joint of the exerciser during the screen climbing and analyzing the energy metabolism through the respiratory gas analysis, it is possible to accurately measure the momentum of each body joint.


In addition, by providing the climbing image (climbing game) that reflects the momentum information and the customized exercise program, it is possible to improve the interest of not only adults but also children, promote health, and expect the advantages of both exercise and game.


The embodiments according to the present disclosure described above are implemented in a form of program commands capable of being performed through various computer components to thereby be recordable in a computer-readable recording medium. The computer-readable recording medium may include a program command, a data file, a data structure, or the like, alone or a combination thereof. The program commands recorded in the computer-readable recording medium may be especially designed and constituted for the present disclosure or be known to those skilled in a field of computer software. Example of the computer-readable recording medium may include a magnetic medium such as a hard disk, a floppy disk, and a magnetic tape, an optical recording medium such as a CD-ROM, a DVD; a magneto-optical medium such as a floptical disk, and a hardware device specially configured to store and perform program commands such as a ROM, a RAM, a flash memory, or the like. Examples of the program instructions include a high-level language code capable of being executed by a computer using an interpreter, or the like, as well as a machine language code created by a compiler. The hardware device may be constituted to be operated as one or more software modules to perform processing according to the present disclosure, and vice versa.


In addition, the detailed description of the present disclosure has been described with reference to specific matters such as specific components and limited embodiments and drawings, but these are only provided to help a more general understanding of the present disclosure, and the present disclosure is not limited to the above embodiments, and those of ordinary skill in the art to which the invention pertains can make various modifications and changes from these descriptions. Therefore, the sprit of the present disclosure should not be limited to the above-described embodiments, and the following claims as well as all modified equally or equivalently to the claims are intended to fall within the scope and spirit of the invention.


The present disclosure relates to a screen climbing system that can track the displacement of the exerciser's body joint, calculate the momentum, and provide the momentum information and the customized exercise program to the climbing image (climbing game) based on the momentum, and has industrial applicability.

Claims
  • 1. A screen climbing system, comprising: a screen on which a climbing image is configured to be projected and displayed, the screen comprising an inclined surface and holds coupled to the inclined surface;an image output device configured to enlarge a climbing image and project the enlarged climbing image on the screen;a motion recognition device configured to recognize an exerciser climbing the screen according to the climbing image to calculate motion information including a position, a posture and a movement; anda smart climbing analysis device configured to: track displacement of each body joint of the exerciser based on the motion information;calculate energy metabolism for the displacement as momentum of each body joint;reflect momentum information in the climbing image based on the calculated momentum; andtransmits transmit the momentum information to the image output device.
  • 2. The screen climbing system of claim 1, wherein the smart climbing analysis device includes: a joint analysis unit configured to generate virtual body points for each body joint of the exerciser based on the motion information to track and calculate the displacement;a momentum analysis unit configured to match the energy metabolism of the exerciser to the displacement to calculate the momentum of each body joint of the exerciser;a customized climbing unit configured to reflect momentum of a specific body joint in the climbing image to increase the momentum of the specific body joint according to exercise achievement and an exercise level according to the momentum or reflects the momentum of the specific body joint in the climbing image to increase the momentum of the specific body joint according to a selection of the exerciser, among the body joints of the exerciser; andan exerciser management unit configured to manage personal information of the exerciser, an exercise time, the number of exercise sessions, and exercise result information.
  • 3. The screen climbing system of claim 2, wherein the joint analysis unit is configured to: configure a skeleton divided into the virtual body points matching each body joint of the exerciser to obtain displacement amount of each body joint being tracked by the skeleton of the exerciser;obtain a current position of the body joint being tracked by the skeleton of the exerciser;obtain an absolute value of the displacement amount of the body joint being tracked by the skeleton of the exerciser, andround off and organizes the absolute value of the displacement amount of the body joint being tracked by the skeleton of the exerciser, andthe momentum analysis unit is configured to:reflect a value obtained by rounding off the absolute value of the displacement amount of the body joint being tracked by the skeleton of the exerciser in a numerical value of the body joint;convert the numerical value to match a color of the image so as to express the momentum of the body joint being tracked by the skeleton of the exerciser; andreflect the numerical value in color property of the momentum image of the body joint being tracked by the skeleton of the exerciser.
  • 4. The screen climbing system of claim 3, wherein the momentum analysis unit is configured to obtain an average by dividing a sum of the displacement amounts of the body joints being tracked by the skeleton by the number of body joints to obtain an average momentum of each body joint.
  • 5. The screen climbing system of claim 3, wherein the momentum analysis unit is configured to obtain an average momentum of each body joint by obtaining an average by dividing a sum of the displacement amounts of the body joints being tracked by the skeleton by a specific numerical value, and the specific numerical value is a natural number obtained by dividing a sum of total momentum of the exerciser to be the same as or close to a result of calculating the momentum of the exerciser as momentum per hour during the climbing image time.
  • 6. The screen climbing system of claim 1, further comprising a respiratory gas analysis device configure to be worn by the exerciser to measure respiratory gas and analyze the energy metabolism, wherein the respiratory gas analysis device is configured to measure at least one of peak oxygen uptake (VO2peak) and respiratory exchange rate (RER) of the exerciser, analyze the energy metabolism including any one of exercise intensity (Mets) and energy consumption per hour (Kcal/h), and additionally measure and analyze maximal heart rate (HRmax).
  • 7. The screen climbing system of claim 1, wherein the climbing image is a motion-based augmented reality (AR) game in which the exerciser is to climb the inclined surface of the screen using the holds.
Priority Claims (1)
Number Date Country Kind
10-2022-0027534 Mar 2022 KR national
CROSS REFERENCE TO RELATED APPLICATIONS AND CLAIM OF PRIORITY

This application claims benefit under 35 U.S.C. 119, 120, 121, or 365 (c), and is a National Stage entry from International Application No. PCT/KR2023/002962, filed Mar. 3, 2023, which claims priority to the benefit of Korean Patent Application No. 10-2022-0027534 filed in the Korean Intellectual Property office on Mar. 3, 2022, the entire contents of which are incorporated herein by reference.

PCT Information
Filing Document Filing Date Country Kind
PCT/KR2023/002962 3/3/2023 WO