Patent Application
20240382806
The present disclosure relates generally to a multi-function method and system for training an athletic motion.
Many different techniques have been implemented in order to teach the proper mechanics of various athletic motions, including swinging a baseball bat, a golf club, a baseball bat, or other sports instrument. Many instructors use a video analysis system to teach a student how to properly swing a sports instrument. Using a typical video analysis system, the student's swing is captured by a video-recording device. The instructor replays the recorded video information to illustrate the student's swing while providing feedback regarding the swing. Instructional feedback may be comments relative to problems associated with the student's swing, compliments regarding improvement in the student's swing, suggestions on correcting the user's swing, or any other verbal instructional comments in context with the student's swing. Visualizing one's personal swing in this manner has been recognized as a valuable tool in identifying problems as well as correcting those problems in order to improve the overall swing.
However, video analysis systems have drawbacks. One drawback relates to having the video information subjectively analyzed. Not only is such analysis open to interpretation and subject to inaccuracies, but it is also exacerbated by the fact that many problems associated with a body movement are typically not captured by the video recording system given different camera angles, too few cameras, or loose clothing.
In order to overcome the drawbacks associated with typical video analysis systems, instructors have adopted motion or position analysis systems as an aid to analysis and instruction. Many conventional motion analysis systems require that the user (e.g., student/athlete) wear sensor elements on their body and the sensor elements transmit positional data of isolated body parts, such as hands, hips, shoulders and head. The isolated points on the body are measured during a swing in accordance with an absolute reference system, e.g., a Cartesian coordinate system wherein the center point is a fixed point in the room. By using motion analysis, exact measurements are provided from which an instructor can more accurately determine problems in a student's swing.
Some existing approaches that employ motion analysis can calculate various speed metrics of the student's swinging motion. The speed metrics can for example include pelvis speed, torso speed, arm speed, hand speed and exit velocity of a ball being struck by a sports instrument such as a baseball bat. These speed metrics are relevant to the student's swinging motion. The exit velocity is closely watched metric and considered to be important by many. However, the existing approaches leave much to be desired, in part because the speed metrics that they produce may do little to actually help the student learn how to improve upon the swinging motion.
It is desirable to improve upon the conventional approaches to address or mitigate some or all of the aforementioned shortcomings.
Disclosed is a method of analyzing an athletic motion by an individual. The method is executed by a motion monitoring system having at least one computer and one or more sensors for sensing movement of the individual. The method involves the at least one computer receiving sensor data captured from the one or more sensors during execution of the athletic motion by the individual, and processing the sensor data to automatically generate at least one speed metric for the individual based on the sensor data.
In accordance with an embodiment of the disclosure, the method also involves the at least one computer categorizing the individual into a category of a plurality of possible categories based on a physical attribute of the individual, and generating, for each speed metric of the at least one speed metric, an indication of relative performance of the speed metric in relation to only other individuals who also belong to the category of the individual. The method also involves the at least one computer outputting the indication of relative performance for each speed metric.
By conveying each speed metric in terms of an indication of relative performance, the individual can be provided with an intuitive view of their performance in executing the athletic motion. For example, if the relative performance is a percentile ranking, the individual may readily know that 90th percentile is excellent while 10th percentile is poor. This can be easier for the individual to perceive and understand than if each speed metric were to be conveyed in terms of raw numbers expressed in meters per second for example.
Moreover, by comparing the individual to only other individuals who also belong to the category of the individual, the relative performance of each speed metric can provide a solid foundation for identifying which speed metric could use improvement. For example, when categorizing individuals according to body mass, the individual would be compared to only other individuals who have comparable body mass as the individual. It has been observed that such comparison based on body mass can improve upon an identification of which speed metric should be targeted for improvement through one or more exercises. Comparisons to other individuals based on age or other criteria that are not related to a physical attribute generally do not provide the same benefit.
Therefore, a combination of (1) conveying each speed metric in terms of an indication of relative performance (e.g. percentile ranking) and (2) comparing the individual to only other individuals who also belong to the category of the individual (e.g. comparable body mass) provides for benefits that can help the individual identify which speed metric should be targeted for improvement through one or more exercises, with a goal of improving the athletic motion as a whole.
Also disclosed is a motion monitoring system. The motion monitoring system has one or more sensors for sensing movement of the individual. The motion monitoring system also has at least one computer having motion monitoring circuitry configured to carry out functionality similar to the steps of the method summarized above.
Other and various aspects, goals and objectives of the invention will be apparent from the examples and illustrations that follow. Pronouns should be interpreted in all cases to include both genders.
The accompanying drawings constitute a part of this specification and illustrate an embodiment of the invention and together with the specification, explain the invention.
The patent or application file contains at least one drawing executed in color. Copies of this patent or patent application publication with color drawing(s) will be provided by the Office upon request and payment of the necessary fee.
An athletic motion analysis system and method for improving performance according to various aspects of the present invention consists of equipment and methods, including cameras, inertial sensors, computers, computer networks, and software, means for providing real time visual feedback in unique formats and prescriptions for practice exercises, all as described in the following paragraphs. The invention comprises many embodiments and variations of which the following examples are illustrative and not limiting.
Referring to
Referring to
System 70 and its related components may be operated at times on a stand-alone basis, but may always or at times be connected or connectable to a remote, knowledge-based enterprise system and database 98 via a browser-based internet access point or other high speed data connection for conducting data transfer and enterprise related activities between the host and local systems.
For example, a website for the enterprise system and host database 98 may provide access for registered user systems 70 to the host company's information, motion analysis products and services information, management information, company news, user access via a log-in screen for product and service FAQs, newsletters, and database 700 libraries of past performance and benchmark data and exercises, and updates thereof.
The website may be configured to provide such global functionalities to registered users as general prescriptions and exercise instructions, explanations, and illustrations-text and/or audio/video, clubhouse events and news, discussion forums, special links for members, global FAQs. an on-line store link, special newsletters, and access to relevant documents and training tips. The website may be divided by categories of registered users pages as between student users and instructor users and provide such particular functionalities as either group might need, such as for instructors the history of instruction sessions by student portfolio, the history of student analysis by portfolio, with sessions organized or stored in respective student “locker rooms” by portfolio, and scheduling for student sessions. Student pages may provide such functionalities as the individual's own personal data, history of his sessions and analysis, his training calendar, instructor contact info, and his golf scores and stats logbook.
There may be a third class of user, an organization user such as a golf school or academy, where a subset of the enterprise system is treated as an OEM client or model, with its own branding, hosting multiple students and instructors as described above.
Individual systems of the invention work in stand-alone configurations as individual test and evaluation systems for collecting student performance data, analyzing and comparing student data to a library of performance data including expert performance data, reporting the results, and prescribing corrective exercises. New test results are added to the database, and may be delivered to or accessed by coaches and/or students via on-line access to internet services. Individual systems may share access to a host database of test results of other users and related practice drills for study or comparative purposes.
Alternate embodiments of the invention may be directed to other athletic, occupational, or rehabilitation motion analysis and training of animals or humans, at either an enterprise level or a local system level as described below.
Referring to
In one example, for a golf swing analysis system, there are a series of three appliances for mounting inertial sensors to the user's body. There is a vest appliance 40 (
Referring to
The slots or sockets for receiving the sensor loops may be characterized as mounting structure, and may be further configured with latch mechanisms that secure the sensor loops 12 within the receiving slots or sockets of the sensor pocket with a mechanical interlock. Variations of the sensor loop structure as a mounting clip or stud and of the pocket slot as a keyed receiver structure, with a latching mechanism such as twist or click fit mechanism incorporated on either or both the appliance and the sensor are within the scope of the invention. The sensor pocket may be reduced in this instance to a mere location on the appliance rather than a full or partial enclosure for the sensor.
Shoulder straps 43 extending from the top corners of back panel 41 attach to strap ends 43A extending from the lower corners of the back panel via buckles. Chest belt sections 44 and 44a extend from the lower corners of the back panel for buckling on the front side of the wearer at about the level of the bottom of the rib cage or kidneys. All straps are adjustable in length for proper fitment to the wearer. The elongated back panel provides stability to the sensor from rotational displacement. The relatively high waist level of the chest strap provides security from vertical displacement of the sensor, and avoids interference with the waist belt appliance 50.
Referring to
Referring to
Referring to
Each sensor has a unique identifier that is encoded within the output data stream, for unambiguous identity during multi-sensor operation. While not strictly necessary, in typical systems sensors are mounted in their appliances on the body with a consistent, pre-determined orientation or “up” end direction, simplifying the calibration and data processing.
Referring to
As described above, referring to
Referring to
Referring to
Control module 30 is configured with a battery pack, hip sensor input, shoulder sensor input, microcomputer, keypad, LCD display, USB connection, remote sensor and system transceiver capability, and optionally with a video game interface.
Referring to
Referring now to
The camera positions and direction with respect to the golfer's position, size and posture are carefully aligned with respect to the test site from one or the other or both of at least two positions: a first camera position at a specific down line angle, height, and lateral position or offset, and another camera position for face on angle, including height and offset. Correct camera positioning enables placement of an overlay in the video display that includes vertical and horizontal alignment lines representing center of alignment and center of balance. There may be multiple cameras on additional stands oriented to capture the motion from different directions and different heights and offsets, and some or all may be positioned carefully to support the further use of overlays of alignment lines relating to the golfer's position, size, posture, and expected motions, so as to make motions and deviations in alignment very apparent in subsequent video presentations of the swing motion.
Stated more generally, prior to testing, it may be required to select and define a test site to have at least one motion reference point; to then position the video camera to be directed at the test site at a pre-defined angle of rotation around the point or test site, a specific height relative to the reference point, with a specific angle of elevation and lateral offset with respect to the reference point. Thereafter a video test signal of the test site and reference point is sent to the computer-driven display screen and an overlay is inserted onto the computer-driven display screen corresponding to the reference point, from which specific motions are more easily observed.
The processing computer or PC of system 70 performs relational calculations on the parameters received from the various sensors, thereby allowing computation of various golf-related parameters of interest. As an example, the PC can calculate club-face angle or the angle through which the golfer turns his or her shoulders while swinging the golf club. Such parameters are referred to here as performance or alternatively diagnostic parameters, to distinguish them from the rate or position parameters transmitted by the sensors to the PC.
In a golf swing motion analysis system in particular, rate and position motion data are typically processed by the application software into performance or diagnostic parameters relating to the golfer's body segment performance, including: hip velocity (degrees per second); hip rotation (degrees negative and positive); shoulder velocity (degrees per second); shoulder rotation (degrees negative and positive); club release (degrees per second); club speed (miles per hour); club face rotation (degrees open/closed); club path (degrees inside or outside of club's address position); hip linear movement (centimeters left or right of neutral address); hip and shoulder separation (time difference between maximum hip, shoulder, and club velocity); flexion/extension of hip segment (centimeters traveled along z-axis); and kinetic link. These parameters are further extrapolated to yield a predicted resulting “ball in flight” performance of parameters: spin (degrees per second); launch angle (degrees); carry distance; roll distance (yards); total distance (yards); distance traveled off line (yards right or left); ball flight character (fade, draw, hook, slice, push, pull, straight); and PTI or power transfer index.
This processed information is reported to the golfer in a unique, synchronized, multi-format presentation of the swing motion that is available in real time and/or playback mode for optimal user and instructor assimilation.
A multi-color animation of the golfer, generated from the inertial sensor motion data, is presented in the upper right portion of the display, although it may be positioned elsewhere in the display. The animation may be color coded to distinguish major body segments, e.g. the shoulders segment versus the hips segment. The animation may be oriented to view the swing motion from any useful angle, depending on what aspect or component of the swing motion is being scrutinized at the time.
In the lower portion of the display a motion data time line graph traces hip, shoulder and hand motions in a multi-colored trace, although it may be positioned elsewhere in the display. The graph may present simply the component motion data from the instant swing motion, and demonstrate graphically the coordination between hips, shoulders and hand motion; or it may present a comparative trace of the present motion or component of motion compared to a prior motion or an expert motion in order to illustrate the degree of deviation and required improvement to achieve a desired performance level.
Referring to
The stepped frame animation is a useful device for illustrating the plane, path or arc of a motion or component of motion, and is a further enhancement of the presentation. Selected positions of a point or object or portion of the video screen are retained as the video progresses so as to show the path leading up to the present position. The stepped aspect of the presentation can be done as function of time, or of linear or angular displacement of the object or point of interest, whichever better serves to illustrate the path of motion best for the viewer.
Stated more generally, the multi-color, three dimensional animation representing the motion of at least one color-coded body segment created from motion data may include or be in some embodiments a stepped frame animation where selected positions of an object in motion are retained in subsequent frames of the animation such that a motion track of the object is apparent to a viewer. The retained positions may be programmed to be selected on the basis of time, position, speed, or acceleration of the object in motion.
The orientation on the screen of these multiple forms of simultaneous presentation may be varied. There may be additional information as well, space permitting. A composite presentation of video, animation, and motion data graphs enhances the user's ability to quickly assimilate and appreciate the subtle differences at the component level of the swing motion, between his current performance and the desired performance. A multi-dimensional presentation of the swing performance can be watched in real time, in an instant replay mode, or in a later review.
The system 70 also offers alternative and supplemental forms of presentation or “report” of the swing performance. Expanded graphs, for example, help clarify the timing of components of motion, as well as the amplitude. For example
The animation capability of the system, driven by the inertial sensor inputs, offers additional opportunities for presenting more detailed illustrations of the swing motion in real time or playback mode. For example,
The animation capability provides yet a further training tool in the form of animated “cages” or scalable limits of selected parameters that cage the animated figure and illustrate the golfer's movement within the three dimensional frame.
Further examples of the power of motion data animation as part or all of the presentation or “report” part of the methodology follow.
The animation capability of the system can also be used to present an enhanced version of the time line traces or graphs.
It should be noted that although
It is a goal of the invention to provide an objective, consistent analysis of each performance. The methodology of the invention depends on capturing motion data, processing it into the described parameters relating to body segments and components of the motion, providing a quantitative analysis of each component of motion, and then summing the scores for each component of motion so as to produce a unitary number or “kinetic index” for the performance as a whole. One embodiment of a system 70 for golf swing motion analysis processes motion data against benchmark values to produce a value on a uniform index scale of 0-50 for each of the following primary performance parameters: sequence, speed, stability, mobility, transfer, timing, club performance, and club accuracy. These values are summed in a pre-determined order to arrive at a unitary number representing the kinetic index for the total performance on a scale of 0-100, as described further below.
Objectivity and repeatability of the system for motion analysis depends on a consistent process that examines and gives weighted consideration of all relevant aspects of the motion in calculating a final performance factor or kinetic index.
Referring now to
The
The primary parameter scores 1-8 are reduced in a first step by a simple summing of related parameters as follows:
9. Sequence & Speed: the sum of the individual indexes of sequence 1 and speed 2 above, having a maximum index of 100.
These second order parameters are further reduced to a body performance factor 13 and a club performance factor 14 as follows:
The body and club performance factors 13 and 14 are summed and divided by 2 to yield the:
It will be appreciated that the pre-selected benchmark values of the individual parameters are somewhat arbitrary, selected to provide a performance challenge to the anticipated range of skills of a target pool of users. The use of other or alternative benchmark values and scoring formulas is within the scope of the invention. Also, the selection and ratio or weight giving to each performance parameter in the reduction process is somewhat arbitrary, the requirement being that each parameter is given a weight or degree of consideration recognized to be relevant to the overall performance.
The reduction process of primary performance parameters into a final kinetic index in the context of a golf swing analysis reflects the kinetic chain philosophy, that the performance value of the total motion is the sum of the performance value of the component parts of the motion executed in an optimal sequence, in order to transfer maximum energy and accuracy from feet to hips to shoulders to arms to the club and ultimately to the ball.
While this description of motion analysis and performance measurement has been cast in the context of a golf swing; the apparatus and methodology is equally applicable to other athletic motions involving, for example, running and kicking leg motions and swinging or chopping hand and arm motions.
Having evaluated individual performance parameters, which may also be referred to as “diagnostic” parameters, the system is able to compare the performance results to a catalog of exercises appropriate to the respective parameters and their test result, and provide an automated recommendation or prescription of exercises. The system may be further preprogrammed with the user's available training schedule and hence able to tailor the prescription to the training time available, with emphasis on the parameters most in need of improvement. In other words, referring back to
In this regard, performance parameters are also characterized as diagnostic parameters. In the golf swing context, they may relate to subsets, body segments or components of the motion including: feet, hip; and shoulder performance. For example, diagnostic parameters of CBL (center balance line) extension and flexion, and of CAL (center alignment line) left and right lateral bending, relate to feet performance. Exercises appropriate to CBL extension problems are scaled according to a pre-determined scheme to the severity or priority of the problem, on a scale of 0 (acceptable performance) to −20 degrees (significantly below acceptable performance). A rating of −5 degrees may generate a prescribed exercise called “posture stick”, using particular training tools; a relatively lower rating of −10 may call for the same exercise but with a different training tool; and so on. The “posture stick” exercise, for example, requires manipulation of a club in a prescribed manner while standing on a base platform, to acquire and practice attaining a stance with the correct alignment of the major joint centers of the body for creating an optimal muscle length tension relationship to enhance the body's postural equilibrium. Other exercises are similarly focused on particular body segments and components of the golf swing.
The initial selection of exercises and tools and the pre-determined scheme for allocation of particular exercises for improving particular performance parameters is somewhat arbitrary, but calculated to induce improvements in performance of components of motion and hence to the total motion performance if practiced as prescribed. The following table 1 lists one embodiment of diagnostic parameters and appropriate exercises by priority by which prescriptions would be issued by the system to a user.
Explanations and detailed instructions for the user's prescribed exercises are available on the local system 70, or may be accessed directly or remotely via an internet access to a host enterprise (
Referring to
Participant devices 1701 may monitor and capture sensor data received from sensors 1702, and to communicate various types of data and instructions to and from devices of the system 1700, such as servers 1709 and observer devices 1703. A participant device 1701 may be any computing device comprising hardware and software components capable of performing the various tasks and processes described herein. Non-limiting examples of a participant device 1701 may include: laptop computers, desktop computers, smartphones, tablets, wearable devices (e.g., smart watches smart glasses, AR headsets, VR headsets, etc.), and the like.
A participant device 1701 may comprise a communications component configured to facilitate wired or wireless data communications between a set of one or more sensors 1702 and the participant device 1701. The communications component may comprise one or more circuits, such as processors and antennas, for communicating sensor data via a communications signal using an associated wired or wireless communications protocol. For example, the communications component of the participant device 1701 may include, for instance, a Bluetooth® or ZigBee® chip that may be configured to monitor and receive sensor data from the set of one or more sensors 1702 associated with the participant device 1701, via the requisite Bluetooth® or ZigBee® protocols. Other non-limiting examples of the communications component and associated protocols may include: a Network Interface Card (NIC) for LAN or Wi-Fi communications, a Near Field Communications (NFC) chip, and the like.
A participant device 1701 may comprise another communications component configured to communicate data and instructions with other devices of the system 1700, such as servers 1709 and observer devices 1703, over one or more networks 1711. For example, the communications component of the participant device 1701 may include, for instance, a wireless NIC allowing the participant device 1701 to communicate data and instructions with servers 1709 and/or observer devices 1703, over one or more networks 1711, using Wi-Fi, TCP/IP, and other, related protocols.
As mentioned, the communications component of a participant device 1701 may be configured to receive sensor data from a set of one or more sensors 1702 configured to capture motion and posture data of a participant, which may then be transmitted to the participant device 1701 as the sensor data. Sensors 1702 may include one or more types of sensors that may be configured to capture the motion and posture data of the participant. Non-limiting examples sensor types may include inertial or movement sensors having a gyroscope, an accelerometer and/or a magnetometer, heat sensors, image sensors (i.e., cameras) capturing still images and/or video images, optical body motion sensors, and the like. In some implementations, the sensors 1702 may be mixed-and-matched and the various types of sensor data may be synchronized, such that the participant device 1701 may receive, and, in some cases, process, the various types of sensor data. Portions of the sensor data may comprise performance parameters and/or diagnostic parameters. Parameters may correspond to fields of data models used by a computing device, such as servers 1709 or observer devices 1703, to model an expected motion or posture data for a particular motion or posture, category of activities, or exercises.
As an example, a factory employee instructional application executed by a participant device 1701 of a factory employee may be configured to teach the factory employee to perform a predetermined set of motions, and then monitor the employee's performance of the motions. While teaching the employee the predetermined motions, the participant device 1701 may receive sensor data from sensors 1701, and may then establish a baseline competency for the employee to perform the motions. This may be done using diagnostic parameters captured in the sensor data. The sensor data may then be transmitted to a server 1709 and/or an observer device 1703. A data library or database located on the participant device 1701, a server 1709, or an observer device 1703, may store data models for each of the predetermined motions. These data models may indicate which data fields or portions of the sensor data are part of the diagnostic parameters for each of the motions.
An observer device 1703 may be operated by an observer (e.g., coach, therapist, doctor, researcher, employer, instructor) and/or system administrator to monitor sensor data from, and communicate instructions with, any number of participant devices 1701a-c. Such monitoring and instructions can also be done autonomously through the use of a trained machine learning module (discussed in more detail below). The observer device 1703 may be any computing device comprising hardware and software components configured to perform the various tasks and processes described herein. Non-limiting examples of the observer device 1703 may include: a laptop computer, a desktop computer, a smartphone, and a tablet. The observer device 1703 may comprise communications components allowing the observer device 1703 to communicate with participant devices 1701a-c simultaneously or near-simultaneously, such that an observer operating the observer device 1703 may review sensor data received from and transmit instructions to, each of the participant devices 1701a-c.
A server 1709 may provide services for monitoring, storing, processing, and communicating sensor data and instructions between devices of the system 1700, such as participant devices 1701 and an observer device 1703. Such services may be cloud based. The server 1709 may be any computing device comprising hardware and software components configured to perform various tasks and processes described herein. Non-limiting examples of the server 1709 may include: a laptop computer, a desktop computer, a smartphone, and a tablet. The server 1709 may comprise communications components configured to allow the server 1709 to communicate with participant devices 1701a-c and/or the observer device 1703 simultaneously or near-simultaneously. For example, the server 1709 may receive sensor data from a plurality of participant devices 1701a-c, and may then covert the sensor data into a file format viewable, sometimes in real-time, from the observer device 1703 (and/or participant devices 1701a-c). As such, an observer device 1703 may access the server 1709 to review or receive real-time sensor data from the server 1709 while the server 1709 receives a data stream of sensor data from the participant devices 1701a-c.
A system 1700 may comprise one or more servers configured to host one or more databases, such as an exercise database 1705 and a participant database 1707. The servers hosting the databases may be any computing devices comprising a processor and non-transitory machine-readable storage media allowing the databases to perform the various tasks and processes described herein. In some embodiments, the databases may be hosted on the same device or on distinct devices. In addition, in some embodiments, a database may be hosted on a computing device that may be used for other purposes. For instance, an exercise database 1705 may be hosted on a server 1709, an observer device 1703, or a participant device 1701, while a participant database 1707 may be hosted on a server 1709.
An exercise database 1705 may store a plurality of exercise records containing data fields associated with exercises. The data fields of a particular exercise may include indicators of the activity categories (e.g., motions, postures, actions) that may benefit from the exercise. The exercise record may include a data model that models the sensor data inputs and parameters that may be used to measure how well the participant is performing the exercise.
A participant database 1707 may store a plurality of participant records containing data field associated with participants. The data fields of a particular participant may include data about the participant, such as vital information about the participant (e.g., name, participant identifier, height, weight), a history of sensor data and parameters, threshold values determined for the participant, and the like.
In some implementations, an observer device 1703 and/or a server 1709 may be configured to automatically generate a set of exercises for participants based the sensor data received from the participant devices 1701a-c. For example, the set of exercises may be based on diagnostic and/or performance parameters of the sensor data. Additionally or alternatively, the software application executed by the observer device 1703 and/or the server 1709 may generate a user interface allowing the observer to input parameter values and/or the set of exercise. For example, for implementations where the system 1700 automatically generates a set of exercises, the diagnostic parameters may be identified in the sensor data and then applied to a data model for a particular motion, or other activity category, to determine a participant's initial skill level, or diagnostic score, for a targeted motion. Based on a diagnostic score calculated for the activity category using the data model, the server 1709 and/or observer device 1703 may identify a set of exercises in an exercise database 1705 determined to be appropriate for the participant's capabilities for the activity category. The set of exercises may be updated and revised as the participant improves a diagnostic score that was calculated for a particular activity category, which may correspond to a particular motion, posture, collection of muscles, or other movement skill (e.g., throwing a baseball, swinging a golf club, a predetermined labor-related motion). The targeted motion may be defined by a data model comprising a set of parameters for motions or postures captured in the sensor data of particular motions or postures; an activity category may be used to identify exercises or other data points and data structures associated with improving upon the targeted motion. For example, the targeted motion and activity category may be associated with improving a runner's stride. In this example, diagnostic and/or performance parameters for this activity category may capture sensor data for aspects of a runner's stride (e.g., upright posture, length of leg extension, arm swing), and the exercises for this activity category may include exercises for improving upon speed and posture (e.g., squats, wall sits, leg extensions, sprints).
The observer device 1703 or server 1709 may generate a regime file, after selecting the set of exercises for an exercise regime to improve a participant's diagnostic score for an activity category or to improve a participant's performance for a given exercise. The regime file may contain data that may be used by an application executed by a participant device 1701 to identify the selected exercises, display the appropriate exercises on the user interface of the participant device 1701, and to capture and send the appropriate sensor data from the sensors 1702. The server 1709 or observer device 1703 may utilize data from the exercise, participant, and/or motion databases to generate each exercise in the regime file. For example, the server may query the exercise database from the latest performed range of motion exercise performed by a given participant and use this information to generate exercises in the regime file with appropriate ranges.
It should be appreciated that the regime file may be one or more machine-readable data files of nearly any file type that may be used as a binary or library of the application. Non-limiting examples of the regime file may include: a database file or database records (e.g., SQL code), a text document, an XML file, an HTML file, an executable file (.exe), a code script (e.g., python, java, C, C++, perl), and the like. The application may be configured to receive and read the data fields of the regime file, which may instruct the participant device 1701 to generate user interfaces displaying still images or multimedia examples of particular postures, motions, or exercises. In some cases, the application may have a set of APIs that correspond to the inputs and outputs of the regime file, allowing the regime file to pass data and instructions to the application. The regime file may contain data associated with the selected exercises; the server or observer device 1703 may query the exercise database 1705 to extract the data of the regime file from the data fields of the exercise records. In some implementations, the regime file may be transmitted directly from the observer device 1703 to participant devices 1701, using a communications protocol and application (e.g., email, FTP, communication protocol native to exercise application). In some implementations, a server 1709 may store a regime file in a participant database 1707 or other storage location, accessible to participant devices 1701 and an observer device 1703.
A system and method for analyzing and improving the performance of an athletic motion such as a golf swing may require: instrumenting a user with sensors (e.g., inertial or movement sensors) and optionally with video cameras, time of flight cameras, or radar-based systems capable of capturing 2D or 3D scene information at regular time intervals; monitoring a golf swing or other motion (athletic or otherwise) of interest; drawing upon and contributing to a vast library of performance data for analysis of the test results; scoring the motion; providing an information rich, graphic display of the results in multiple formats including video, color coded and stepped frame animations from motion data, and synchronized data/time graphs; and based on the results prescribing a user-specific training regime with exercises selected from a library of exercises. As discussed above. scoring the motion may involve scoring pre-defined parameters relating to component parts of the motion and combining the parameter scores to yield a single, kinetic index score for the motion.
One or more embodiments of the invention may include an auto capture system in which data capture from the sensors (e.g., inertial sensors having a gyroscope, an accelerometer and/or a magnetometer, heat sensors, image sensors (e.g., cameras) capturing still images and/or video images, optical body motion sensors, and/or the like) is triggered by a specific input (e.g., a motion or gesture).
In such embodiments, streaming data may be processed in real time, or near real time, and when a specific input (e.g., gesture) is recognized (e.g., a golf swing), a time window of sensor data is automatically recorded. The time window being taken from a predetermined time period around the moment in time in which the specific input was recognized (e.g., when the gesture occurred). For example, the predetermined time period may include 2 seconds before the moment in time when the specific input was recognized and 3 seconds after the moment in time when the specific input was recognized.
Exemplary embodiments of the auto capture system are illustrated in
According to the embodiment illustrated in
Alternatively, as shown in the embodiment illustrated in
The local data buffers 1725 may exist in one or more of the sensors. The sensors may be considered separate from each other or be ganged or networked together in some relationship configuration. For example, sensor data may be transmitted from one or more sensors to a local data buffer existing in another sensor. The aggregate sensor data from the sensors may then be transmitted from that local data buffer 1725 to the CPU transceiver.
For example, as shown in
For example, in a golf scenario, an impact that occurs when a golfer strikes a golf ball may result in a particular signature of at least one of acceleration data, body segment orientation data, and rotational velocity data. This impact can be measured by a handset sensor, wrist sensor, and/or a club mounted sensor. The signature may be used by the system to automatically identify a particular motion gesture (e.g., golf swing). Then, as discussed above, a predetermined time window of the sensor data may be analyzed by the system.
Similarly, in a baseball scenario, an impact that occurs when a batter strikes a baseball may result in a particular signature of at least one of acceleration data, body segment orientation data, and rotational velocity data. This impact can be measured by a handset sensor, wrist sensor, and/or a club mounted sensor. The signature may be used by the system to automatically identify a particular motion gesture (e.g., baseball bat swing). Then, as discussed above, a predetermined time window of the sensor data may be analyzed by the system.
An alternative embodiment of an autonomous training system for a motion instruction system 1900 is illustrated in
The customizable regime files may be of particular use with (although not limited to) large groups of participants wherein each participant is at a different level of proficiency. The customizable regime files allow all participants to be together but some to work completely alone (without coaching or training) while others receive coaching; and where all of the participants receive unique versions of the same program based on the individual participant profiles.
As previously described with regard to the embodiment illustrated in
The participant database 1907 may store user or participant related information. The information stored therein may consist of data fields such as Participant ID, Participant Name, Participant Height, Participant Weight, etc.
The observer database 1913 may store observer (e.g., coach, trainer, etc.) related information. The information stored therein may consist of data fields such as Observer ID, Observer Name, Associated Participants (e.g., participants associated with the observer, such as a class of 50 trainees), Generated Regime Files, etc.
The exercise database 1905 may store exercise related information. For purposes herein, it is understood that “exercise” may include a training exercise (e.g., bend at address) as well as movements such as a golf swing (previously referred to herein as an Activity Category). Each exercise may include one or more component motions. The information stored therein may consist of data fields such as Exercise ID, Exercise Name, Scoring Function, Attribute Tags, Tolerance Variables, etc.
The motion database 1915 may store captured motion data for an exercise. The information stored therein may consist of data fields such as Sensor Data (e.g., inertial, video, etc.), Outcome Data (e.g., launch monitor, etc.), Participant ID, Exercise ID, a Timestamp, etc.
In a non-limiting example, such as shown in
In a non-limiting example, generating the regime file using the content-based filtering approach may involve having a library or exercise database 1905 of N different training exercises for which an exercise vector of length N can be initialized and updated as follows:
Initialization: For a new user, initialize the exercise vector to the zero vector. For example, for a library of 5 exercises consisting of “Rotation at Impact”, “Bend at Address”, “Rotation at Impact”, “Hip Twister”, and “Duck Walks”, the exercise vector would be initialized to [0, 0, 0, 0, 0].
Exercise Recommendation Algorithm: For each user, the exercise vector provides a means of ranking training exercises based on how much they improve the output score of interest. For example, an exercise recommendation algorithm could be to recommend M exercises with the highest values in the exercise vector, where M<=N. However, this approach may be prone to converge on a local optimum as soon as M exercises achieve values greater than 0 in the exercise vector. Another exercise recommendation algorithm could be to recommend M+L exercises (where M+L<=N), consisting of M exercises with the highest values in the exercise vector and L exercises chosen at random from the remaining N-M exercises. The invention is of course not limited to these two exemplary content-based filtering exercise recommendation algorithms
In a non-limiting example, the collaborative filtering approach for generating the regime file may involve implementing a collaborative filtering algorithm by extending the content-based filtering approach described above. In order to do this, for example, an augmented exercise vector of length N may be defined. The elements of a user's augmented exercise vector corresponding to exercises that have been performed at least once by the user are assigned the same values as the corresponding elements in the user's exercise vector. The elements of a user's augmented exercise vector corresponding to exercises that have never been performed before by the user are assigned the same values as the corresponding elements in the exercise vector of the user in the participant database who is most similar to the user of interest. Similarity between two users can be determined by the inner product of their normalized exercise vectors (higher values indicate greater similarity). With the foregoing technique, an exercise recommendation algorithm could be that for each user, recommend M+L exercises (where M+L<=N), consisting of M exercises with the highest values in the augmented exercise vector and L exercises chosen at random from the remaining N-M exercises. The invention is of course not limited to the foregoing collaborative filtering exercise recommendation algorithm.
According to another embodiment of the invention, the motion instruction system 1900 may operate in a dynamic biofeedback mode. In this mode, the processing computer performs a dynamic motion scoring process and trains a dynamic motion as opposed to one or more static postures. A separate description of the portions of the system having the same structure and function as those of the previous embodiment are omitted for purposes of convenience to the reader, and the differences from the previous embodiment are herein described.
In the dynamic biofeedback mode, the motion instruction system 1900 may compare biomechanical parameters computed for a captured motion (discussed above) to a previously generated motion template stored in a database. The motion instruction system 1900 may then compute a similarity score. For example, a similarity score of 0 may be used to represent a perfect similarity match (i.e., the derived biomechanical parameters are identical to the motion template), and a similarity score of positive values (e.g., 1-100) may be used to represent degree of mismatch. The similarity score may then be displayed on the participant device 1701, or another display or recipient device that is configured to convey feedback to the user.
Based on the similarity score, the motion instruction system 1900 can then generate an auditory, visual, and/or haptic biofeedback signal. The biofeedback signals may be different depending on the similarity score. For example, the similarity score may range from 0 to 100, with zero being ideal and 100 representing a high divergence from the ideal. In this example, a red light might follow an exercise in which a derived biomechanical parameter badly diverged from ideal (e.g., score of 50-100), a yellow light might follow an exercise in which a derived biomechanical parameter only somewhat diverged from ideal (10-49), and a green light might follow an exercise in which a derived biomechanical parameter is ideal or diverged from ideal by less than the pre-assigned margin of error (0-9). The signal light may be the background color of an animation or avatar displayed on the participant device 1901 and/or observer device 1903, or another display or recipient device that is configured to convey feedback to the user. Similar differences in biofeedback signals could be done using audio or haptic signals.
Furthermore, unlike static posture training, where only differences in posture are considered, the dynamic biofeedback similarity score may also capture differences in timing.
An exemplary algorithm for generating a similarity score is described below:
Else, if τ<0 or τ>K−N, stop here and raise an error indicating that the captured motion does not contain data matching the entire template.
where each value wi is a scalar weight applied to the NRMSE for row i.
According to one or more of the foregoing embodiments, the biofeedback mode of the motion instruction system 1900 allows a user to “train to his or her best motion.” According to a non-limiting example, such training may be accomplished by:
According to one or more of the foregoing embodiments, the motion instruction system 1900 may generate comprehensive user health, fitness, and skill scores based on many component diagnostic scores. Although some prior embodiments describe “activity categories,” it is understood that such activity categories may relate to exercises, which can include training exercises (e.g. bend at address) and specific motions (e.g. golf swing) and “diagnostic scores,” and the concept of “scoring functions” can be assigned to each exercise. The outputs of these scoring functions may be used to represent performance on specific training exercises, quality of specific motions, and even things like range of motion for particular body segments. Furthermore, group health, fitness, and skill scores can be generated for groups of users (e.g. teams or organizations) based on the individual user health, fitness, and skill scores of their members. This may be beneficial for group competitions where one group of users competes against another group of users, such as in a group training class.
According to one or more of the foregoing embodiments, the motion instruction system 1900 may be configured to continually monitor user compliance with a training regime. For example, in the factory worker example discussed above, user compliance with an exercise regime (e.g., assigned exercise score is above a predetermined threshold), or lack thereof (e.g., assigned exercise score is below a predetermined threshold, or the assigned exercise is not being performed), may be transmitted to an observer (via text message, e-mail message, alert on web portal, etc.). Preferably, such alert is transmitted to the coach or observer in real time so that the exercise regime may be revised or changed accordingly.
For example, such continual monitoring may be performed so that employers can ensure that their employees are complying with a particular training regime while in the workplace. Pre-determined movements of employees may be measured as they perform their regular day-to-day work tasks, such as, for example, lifting or walking. Such continual monitoring can be important to prevent injuries for employees performing repetitive tasks, such as those in hospitality (e.g., making beds), in a warehouse (e.g., lifting, pick and place movements), etc. Using the captured motion data, the motion instruction system 1900 assigns an exercise score for a particular exercise or movement being performed. When the exercise score is below a predetermined threshold or the assigned exercise or movement is not being performed, then the motion instruction system 1900 may transmit an alert to an observer (via text message, e-mail message, alert on web portal, etc.) to inform the employer that the employee is moving incorrectly (which puts them at risk of injury). Preferably, such alert is transmitted to the observer (employer) in real time so that the exercise regime may be revised or changed accordingly. Additionally, when the exercise score is below a predetermined threshold or the assigned exercise or movement is not being performed, then the motion instruction system 1900 may transmit an alert to the participant device so that the employee may have an opportunity to self-correct. If the employee fails to self-correct (i.e., the exercise score remains below the predetermined threshold), then the motion instruction system 1900 may send an alert to the participant device 1901 (as well as the observer device 1903) ordering the employee to stop and then guide the user through a protocol to activate muscles and remind them of the correct movement pattern via instructions (graphical, video, and/or textual) displayed on the participant device 1901 and/or the observer device 1903, or another display or recipient device configured to convey feedback to the employee.
According to an embodiment of the invention, the system 1900 may be configured to provide real-time alerts to a user, such as a coach/observer, to prevent injury. For example, a coach, organization, or general user (such as a participant) can set a custom alert trigger based on sensor data for a specific user or group of users. For example, a coach may set a trigger such that whenever a player with a back injury risk exceeds 60 degrees of forward bend of the torso, an alert is sent to him or her in the form of an email, text message, phone call, etc.
According to an embodiment of the invention, the system 1900 may be configured to provide real-time athlete monitoring for group training. In this embodiment, a group of users repeatedly train the same biofeedback exercise or perform swing motions at the same time. The motion data for each user is captured locally and immediately sent to the cloud, where it is processed to determine how well each user is performing each biofeedback exercise or swing motion. This data is then used to render a web dashboard to be viewed on an observer device 1903 by a coach. The rendering represents each user as a simple tile, which turns red if the user if performing poorly and green if the user is performing well (not limited to any particular color or look). This allows the coach to identify users in the group that are struggling or excelling during a live training session.
According to one or more of the foregoing embodiments, motion instruction system 1900 may be configured so that during exercise routines, real-time feedback or analysis may be provided to the user based on sensed data, including image data, about the user. In this manner, the system 1900 may function as a “virtual coach” to the user to help make exercising more interactive and help achieve results and goals of the user faster. In other words, such real time feedback may be based on any of a number of data inputs, such as personal data of the user, real-time exercise parameters of a current exercise session, and/or archived exercise parameters of past exercise sessions. The feedback may be transmitted from an observer device 1903 and/or server 1909 to the participant device 1901, or another display or recipient device that is configured to convey feedback to the user.
The virtual coach feature may operate by automatically generating useful tips and exercise lessons based on motion data from the sensors. For example, with a sport such as golf, such virtual coach feedback may be based on motion data from inertial sensors, club and ball data from a launch monitor apparatus (e.g., club speed, ball speed, launch angle, spin, and azimuth), and virtual course data from a golf simulator (e.g., ball position, course difficulty, weather conditions, terrain, etc.). These tips and lessons can be communicated to the user in real-time through text, audio, vibration, an animated coach avatar in the golf simulator, or any combination thereof. The same concept can be altered for other sports and activities, such as baseball, tennis, exercising, etc. Moreover, this concept may be extended to all forms of motion monitoring.
Training Motion Scoring Models with Supervised Machine Learning Algorithms
According to one or more of the foregoing embodiments, the system 1900 may apply machine learning techniques to learn relationships, functions, and categories associated with various analysis procedures, which may include modeling or scoring a particular motion gesture (e.g., golf swing) or exercise based on the sensor data. A supervised machine learning algorithm offers flexibility as it trains motion scoring models based on data, such as data contained in exercise database 1905, participant database 1907, observer database 1913, a motion database 1915, and/or subsets thereof.
An exemplary embodiment of a machine learning technique that may be used with one or more embodiments of the motion instruction system 1900 described herein is illustrated in
The machine learning algorithm may generally be configured to train two model categories: classification and regression. The classification model may output discrete class categories, e.g., classifying an input motion as “expert”, “novice”, or “beginner”. The classification model may include, but is limited to, logistic regression, decision trees, decision forests, support vector machines, naïve bayes, k-nearest neighbors, and convolutional neural networks. The regression model may output continuous values, e.g., assigning a numerical score to an input motion. The regression model may include, but is not limited to, linear regression, polynomial regression, k-nearest neighbors, and convolutional neural networks. Trained classification and regression models can then be used to score input motions.
As shown in exemplary
Alternatively, as shown in exemplary
In one or more of the foregoing embodiments, the motion instruction system 1900 may be used for training a user (e.g., golfer) in a set of exercises based on continuously captured data (e.g., capture or auto capture motion data, measure the data, assess the data, coach the user, and prescribe training regime). The training may be based on pre-captured data (e.g., load and train a prebuilt program). The training may be for on a single motion parameter, or for more than one motion parameter. The user and or observer (coach) may select which motion parameter to target and/or which body segment to be trained.
In one or more of the foregoing embodiments, the motion instruction system 1900 may be used to train a user based on captured data. For example, instrumenting a user with sensors (e.g., inertial or movement sensors) and optionally with video cameras, time of flight cameras, or radar-based systems capable of capturing 2D or 3D scene information at regular time intervals; monitoring a golf swing or other motion (athletic or otherwise) of interest; capturing or auto capturing motion data of interest; drawing upon and contributing to a library of performance data for analysis of the test results; scoring the motion; providing an information rich, graphic display of the results in multiple formats including video, color coded and stepped frame animations from motion data, and synchronized data/time graphs; and based on the results prescribing a user-specific training regime. As discussed above, the system 1900 may apply machine learning techniques to learn relationships, functions, and categories associated with various analysis procedures, which may include modeling or scoring a particular motion gesture (e.g., golf swing) or exercise based on the sensor data.
In one or more of the foregoing embodiments, a user may train to his or her best swing (or any swing, any shot, or any ball flight) using captured or auto captured motion data. For example, a golfer may swing the club ten times and then select one of those swings (e.g., their best swing or a swing based on their desired body/ball performance) as a model swing. The motion instruction system can then automatically develop and prescribe a user-specific training regime based on the model swing. In this manner, a user-specific training regime can be prescribed for any motion that a user desires to repeat (e.g., their best swing). Thus, a dynamic training program can be generated for the selected swing so that a user can be coached and trained to perform the exact captured movement with video, audio, and/or haptic cues according to one or more of the above described embodiments.
In one or more of the foregoing embodiments, a user may be instrumented with a wrist sensor 2300 (e.g., inertial sensor) that is attached or worn on his or her wrist, such as shown in exemplary
According to an embodiment of the invention, performance indicators and real-time feedback or analysis may be provided to the user based on the wrist sensor motion data and/or wrist sensor motion data in conjunction with club and ball data obtained from the launch monitor apparatus. The feedback may be transmitted from the observer device 1903 and/or server 1909 to a user interface presented in the participant device 1901, or another display or recipient device that is configured to convey feedback to the user.
For example, the server 1909 (or observer device 1903) may be configured to generate different graphical user interfaces and display them on different computing devices described herein. As previously discussed, the server 1909 hosting the databases may comprise a processor and non-transitory machine-readable storage media comprising a set of instructions allowing the various databases to perform various tasks and processes described herein, such as to display various graphical user interfaces. Each instruction within the set of instructions may command and cause a different module of the server 1909 or processors to display a particular section or container of the graphical user interfaces described below. For example, a first instruction may instruct (e.g., command or cause) a first module of the server 1909 to query pertinent data from the exercise database 1905, participant database 1907, observer database 1913, or motion database 1915 and display a first section of a graphical user interface; and a second instruction may instruct a second module of the server 1909 to query pertinent data from a different database and display a second section of the graphical user interface. Although described herein as separate modules, it is intended that these modules can be configured as at least one module. Moreover, the server 1909 may be a database server comprising a processor capable of performing the various tasks and processes described herein. Non-limiting examples may include a server, desktop, laptop, tablet, and the like. The server 1709 may host an online service, such as cloud-computing application service, or any other service that provide web-based applications that collect data through we-based client interactions over one or more networks such as network 1911. Accordingly, the server 1909 may generate and display different graphical user interfaces on different computing devices described herein.
According to another embodiment, for example, the one or more servers 1909 include an analytics engine that further includes a data extraction module and data processing module. The analytics engine can be a software component stored on a computer readable medium and executed by a processor, e.g., as specially-programmed software on a server (referred to and used interchangeably as an analytics engine server). The analytics engine can be configured to receive user input from one or more participant devices 1901 and/or one or more observer devices 1903, receive data from a database (e.g., exercise database 1905, participant database 1907, observer database 1913, motion database 1915, etc.), produce solution data from the received user input and data, and provide the produced solution data to one or more participant devices 1901 and/or one or more observer devices 1903. Thus, for example, a user may request a report, such as an Evaluation Report, regarding the status of a particular training program, and the analytics engine may generate and present the report on different computing devices described herein.
In some embodiments, the analytics engine is implemented as a set of computer instructions executed by one or more servers 1909 that run computer executable program instructions or related algorithms.
Referring to
The multi-color animation of the animated
In one or more of the foregoing embodiments, the motion instruction system may be configured to provide/prescribe the user with an exercise or workout of the day that is based in part on the user's prior performance of various motions as assessed by the system based on diagnostic parameters from the sensor data. Such prescription can also be done autonomously through the use of a trained machine learning module or manually by an observer/coach based on the sensor data. The information may be delivered to the user via a user interface presented in the participant device 1901, or another display or recipient device that is configured to convey feedback to the user.
It is understood, for example, that the observer/coach may prescribe a workout of the day to one or more users (unlimited number of users). The prescribed workout (e.g., regime file) may be pushed to each user's participant device or provided on a website accessible by a web browser. The prescribed workout may be identical for each user, or individually customized to each user based on performance data associated with each user. In other words, each user may be prescribed the same exercise at the same time (e.g., squats for a one minute time period); however, the prescribed workout for each of the users may be customized based on performance data associated with that particular user (e.g., advanced user may be prescribed 15 squats in the one minute time period, while a novice user may be prescribed 10 squats—in this way all users in the workout are performing the same exercise at the same time).
The workout may be generated in conjunction with an auto capture system; an autonomous training system; a dynamic motion scoring and training system; and/or a training motion scoring models with machine learning algorithms system, such as described herein, as well as biofeedback. Motion data may be transmitted to the observer/coach in real time or at the conclusion of the prescribed workout so that that the trainer/coach can provide feedback or additional coaching to the user. Furthermore, performance data from one or more users can be used to generate a leaderboard, points, competitions, etc. in conjunction with the prescribed workout of the day. The system may further include a database of observers/coaches such that the user may select an observer/coach from the database based on the user's preference (e.g., gender, age, intensity of workouts, music playlists, personality, etc.). The foregoing embodiments are advantageous in that they provide for a cloud-based student monitoring platform with biofeedback learning loop embedded software for analyzing and improving the performance of an athletic motion such as a golf swing. The cloud-based student monitoring program shows the observer/coach every repetition of every player's training.
As discussed, the motion instruction system 1900 links a coach to one or more users of the system. The system 1900 is configured to automatically generate a training program based on user data (personal, biometrics, motion data, etc.) and transmit the training program to a user interface. The user can then follow the training program on site or remotely, and motion data for the prescribed exercises are sent to the coach or observer in real-time. The system 1900 provides the coach or observer with every repetition of every user's training.
Accordingly, the system 2700 may be configured to perform: 1) a local capture process 2710 in which the CPU captures motion data, such as described above (e.g., captured motion data from wearable inertial sensors 1702a-c, a wrist sensor 2300, a launch monitor, video camera, radar system, etc.); and 2) cloud-processing techniques 2720 in which the captured motion data is received, analyzed, and processed by the CPU to generate one or more exercises for the participant to perform based on the sensor data, such as described above. Based on the cloud-processing techniques 2720, the CPU can generate, among other things: (a) an evaluation report 2730 based on the captured motion data to provide an objective record of the type and degree of changes in performance that the user has experienced; (b) a training program 2740 for the selected movement (e.g., swing) so that a user can be coached and trained to perform the exact captured movement with video, audio, and/or haptic cues according to one or more of the above described embodiments; and (c) personalized content marketing to deliver content or messages to the user based on the motion data and/or information provided by the user. Additionally, the cloud-based system 2700 may process information provided by the user to target advertising to users in real-time or at the conclusion of a prescribed workout across any platforms. Such advertising can be targeted based on personal data, performance characteristics, or any other data gathered by the system 2700.
In various exemplary embodiments the user (e.g., participant, observer/coach) can use a graphical user interface generated by the CPU and displayed on the participant device 1903 and/or the observer device 1901 to view and/or select a range of different information on the display. The graphical user interface can provide a wide range of control and informational windows that can be accessed by a click, touch, or gesture. Such windows may provide information about the user's own performance and/or the performance of other participants in the same who are performing the same or different activity-both past and present.
The graphical user interface may be used to access user information, login and logout of the system 2700, as well as access live training instruction and archived content. Such user information may be displayed in a variety of formats and may include past and present performance and account information, social networking links, achievements, etc. The user interface may also be used to access the system to update user profile information, manage account settings, and control participant device 1903, observer device 1901, and/or server 1909 settings.
Referring to
The graphical user interface generated for the Equipment Manager screen 2900 may further include a “(RE)-DETECT SENSORS” button 2910 that the user can press or touch to direct the system 2700 to reestablish a connection to the sensors in the event that any of the sensors are not connected to the system 2700.
The graphical user interface generated for the Equipment Manager screen 2900 may further include a “Usage Tracking Level” button 2915 that may be toggled by the user to allow the system 2700 to continually track the amount of usage of the various sensors connected thereto. As shown, the user has the option to turn off the tracking so that such sensor usage is anonymous and not tracked by the system 2700.
The graphical user interface generated for the Equipment Manager 2900 screen may further include a section for monitoring and detecting peripheral monitoring device, such as a launch monitor manager 2920 and a camera manager 2925. Similar to the “(RE)-DETECT SENSORS” button 2910 described above, the screen may include a “FIND MONITOR” button that the user can press or touch to direct the system 2700 to establish (or reestablish) a connection to with a launch monitor device. Here, the launch monitor is not connected to the system 2700.
Referring to
As discussed above, the system 2700 may be configured to continually monitor participant compliance or progress with a training regime and transmit information related thereto to a user, such as a coach/observer (or the participant them self), in real time. This may be done via a web portal, a mobile app, or other electronic means.
In this exemplary embodiment,
The Speed Creation Score 3705 is a measurement of how well the participant can create club speed with his or her body relative to a database of thousands (or more) male and female golfers of all ages. Because club speed is generated from the ground up, greater weight may be applied to the speed of body segments that are closer to the club (pelvis<torso<upper arm<lower arm<hand). In this example, the Speed Creation Score 3710 is 68.
The Consistency Score 3710 is a measurement of how consistent the participant's golf swing is in terms of both body angles and timing relative to a database of thousands (or more) male and female golfers of all ages. The body angles component measures variability of pelvis and torso angles at address, top, and impact, while the timing component measures variability of backswing and downswing times. The overall body angles component and timing component may be weighted equally by the CPU. In this example, the Consistency Score is 74.
The Visualization 3715 shows how the participant's score ranks against the range of scores for other players in his or her peer group (e.g., same gender and age group). The Text Field 3720 is a text field where a user (e.g., coach) can enter his or her comments on the Evaluation results. The Next Steps Field 3725 is a text field where a user (e.g., coach) may enter recommended next exercises for the participant to perform based on the evaluation results. The comments entered into the Text Field 3720 and/or Next Steps Field 3725 may be processed by the CPU and stored in one or more databases of the system 2700.
As shown in
For example, the CPU may compute average Peak Speeds 5105 (degrees/second) for various body segments of interest (e.g., pelvis, torso, upper arm, hand, etc.) across all Evaluation swings (shown as a black dot) and a corresponding range for professional baseball players (shown in green) from related data stored in a database of professional baseball player data and present them on the display as shown in accordance with one or more of the foregoing embodiments. In other words, the CPU compares the participant's body segment speed against that of an average body segment speed for professional baseball players and presents the comparison on the display. The CPU may be configured to generate and present an automatically generated comment 5110 based on a determined relationship between the participant's speed segments versus that of the professional baseball players. Here, because all of the participant's measured peak speeds are within the range of the professional baseball players, the CPU is programmed to present an auto-generated comment 5110(a) that reads “Your peak speeds are all within Pro Range” (not limited thereto). However, in the event that all of the participant's measured average peak speeds falls below the range of professional baseball players, the CPU is programmed to present an auto-generated comment that reads, “Your peak speeds are below Pro Range”. If one or more, but not all, of the participant's measured average peak speeds fall within the range of the professional baseball players, the CPU is programmed to present an auto-generated comment that reads, “Your peak speeds are partially within Pro Range.”
An exemplary algorithm for generating an auto-generated comment 5110 is described below:
For example, the CPU may compute Speed Gain 5115 across all Evaluation swings (shown as a black dot) and a corresponding range for professional baseball players (shown in green) from related data stored in the database of professional baseball player data and present the comparison on the display as shown in accordance with one or more of the foregoing embodiments. Speed gain is the ratio between the peak speeds of adjacent segments, such as the torso/pelvis peak speed ratio. As discussed above, the CPU may generate and present an automatically generated comment 5110 based on a determined relationship between the participant's speed gain versus that of the professional baseball players. Here, because the participant's speed gain (1.38) falls below that of the range of the professional baseball players, the CPU is programmed to present an auto-generated comment 5110(b) that reads “Your torso is slightly low resulting in a speed gain below Pro Average” (not limited thereto).
For example, the CPU may compute Sequence 5120 across all Evaluation swings. Sequence is the order in which the participant's body parts reached peak speed. Here, the respective body parts are displayed as different color drawings representative of each body part (not limited thereto) for each recognition by the user. In this example, the order in which the participant's body parts reached peak speed was pelvis then upper arm then torso then hand. The professional baseball player sequence is displayed in the order pelvis then torso then upper arm then hand, which the CPU determines from related data stored in the database of professional baseball player data. Thus, the CPU compared the participant's sequence with that of the average professional baseball player and determined that the participant's order was not consistent with that of the professional baseball player because the participant's torso speed peaked too late. Here, because the participant's torso speed peaked too late as compared with the average professional baseball player, the CPU is programmed to present an auto-generated comment 5110(c) that reads “In your swing, the torso peaked too late” (not limited thereto).
For example, the CPU may compute Timing 5125 across all Evaluation swings. Timing is the calculated time between when the heel contacts the ground and the bat contacts the ball. The CPU automatically calculates this time for each swing based on the captured motion sensor data from at least the hand sensor and the pelvis and torso sensors. Timing is typically measured in seconds. Here, the participant's measured time to contact is 0.225 seconds, which is much faster than that of the average professional baseball player which measures 0.284 seconds. Although not shown, like above, the CPU may be programmed to automatically generate and present a comment related to timing.
It is understood that the CPU may generate the Report Summary 5100 without comparison to a professional baseball players or as compared to a different category of players, such as a peer group.
According to an embodiment of the invention, as illustrated in exemplary
For example, in Auto Capture mode (discussed above), a user may simply take a swing and the motion is detected, automatically recorded, and processed by the CPU, and then each of the tiles in the Tile Display 5605 is updated with the appropriate movement data by the CPU. To facilitate faster training feedback loops, the Tiles Display 5605 is displayed next to the 3D avatar in the AutoCapture screen, such as shown in
According to the foregoing, the invention in one aspect provides a method and system for analyzing and improving the performance of an athletic motion (e.g., golf swing, baseball swing, yoga, dance, etc.) or body movement (lifting, walking, etc.) to monitor a user's accountability, which involves: 1) capturing motion sensor data for a user (e.g. through body worn sensors (e.g., inertial or movement sensors) and optionally with video cameras, time of flight cameras, or radar-based systems capable of capturing 2D or 3D scene information at regular time intervals); 2) transmitting the captured motion sensor data (all or select portions of the data) to the cloud for processing by a CPU or server that provide services for monitoring (continually or at predetermined time periods of interest), storing, processing, and communicating sensor data and instructions between devices of the system, such as participant devices and/or observer devices; 3) generating by the CPU one or more user interfaces with dashboards that present snapshots, progress reports, comparisons to development path, etc. to be displayed on the participant devices and/or observer devices; 4) automatically creating by the CPU an exercise program with biofeedback (or only video/animation) for the user based on machine learning techniques that measure, analyze, and process the motion data; 5) transmitting the training program to the user via a network; 6) monitoring by the CPU the user's compliance with the training program against a baseline threshold competency; and 7) alerting the user and/or a coach/observer in real time if the user's compliance falls below the baseline threshold competency by sending a message to a web portal, via text message, e-mail message, etc., which may order the user to stop and/or guide the user through a protocol to remind them of the correct movement pattern via instructions (graphical, video, and/or textual) displayed on any display or recipient device configured to convey feedback to the user.
Categorizing & Comparing Individuals based on Physical Attribute
Referring now to
In the description that follows, the athletic motion pertains to swinging a baseball bat to strike a baseball, and the one or more sensors include at least one of a handset sensor, a wrist sensor, and a bat mounted sensor for capturing an impact between the baseball bat with the baseball. However, other athletic motions are possible. For example, the athletic motion can pertain to swinging a golf club to strike a golf ball, and the one or more sensors include at least one of a handset sensor, a wrist sensor, and a club mounted sensor for capturing an impact between the golf club with the golf ball. More generally, the method is applicable to any athletic motion by an individual that can be captured by one or more motion sensors.
At step 58-1, the observer device 1903 or server 1909 receives sensor data captured from one or more sensors during execution of an athletic motion by the individual, such as swinging a baseball bat to strike a baseball. The sensor data can for example include inertial data (i.e. from inertial sensors), video data (i.e. from one or more video cameras), or other sensor data. In some implementations, the observer device 1903 or server 1909 also receives outcome data (e.g. launch monitor data, or other outcome data), a participant ID of the individual, an exercise ID of the athletic motion, and a timestamp of when the athletic motion is being executed. In some implementations, additional information about the individual, such as name, height and weight for example, is accessed from a participant database 1907.
The observer device 1903 or server 1909 processes the sensor data to automatically generate at least one speed metric for the individual based on the sensor data. In some implementations, the at least one speed metric includes pelvis speed, torso speed, arm speed, hand speed and/or exit velocity (Exit Velo or EV) of a ball being struck by a sports instrument such as a baseball bat. Other speed metrics are possible. The speed metrics may not be easy for the individual to perceive or understand if they are conveyed in terms of raw numbers expressed in meters per second for example. Therefore, rather than conveying the speed metrics in this way, at step 58-2 the observer device 1903 or server 1909 determines speed percentiles for each speed metric. The individual may readily know that, for a given speed metric, 90th percentile is excellent while 10th percentile is poor, for example, and hence such speed percentiles can be easier for the individual to perceive and understand. While the examples described herein focus on speed percentiles, other possibilities exist such as quantile ranking for example, and more generally any suitable indication of relative performance can be employed.
In some implementations, in order to determine the speed percentiles, the observer device 1903 or server 1909 accesses a participant database 1907 and/or a motion database 1915, so that the speed metrics of the individual can be compared with other individuals to determine the speed percentiles. However, in accordance with an embodiment of the disclosure, the observer device 1903 or server 1909 compares the individual to only other individuals who belong in a same body mass category as the individual. Therefore, the individual is compared to only other individuals who have comparable body mass as the individual. It has been observed that such comparison based on body mass can improve upon an identification of which speed metric of the individual should be targeted for improvement through one or more exercises. Comparisons to other individuals based on age, sex, experience level, or other criteria that are not related to a physical attribute generally do not provide the same benefit.
While the examples described herein focus on categorizing and comparing individuals based on body mass, other possibilities exist such as physical height and/or wingspan for example, and more generally any suitable set of one or more physical attributes can be used for categorization and comparison. A database can be provided with, for each category of the possible categories, speed metrics for individuals belonging to the category. The observer device 1903 or server 1909 can then access that database to make appropriate comparisons when generating the speed percentiles.
The speed percentiles can be output for the individual to view. A couple of specific examples are provided below. Note that the speed metrics shown below are very specific for example purposes only.
Pelvis Speed
21
Percentile:
Arm Speed Percentile:
The combination of (1) conveying each speed metric in terms of an indication of relative performance (e.g. percentile ranking as shown above) and (2) comparing the individual to only other individuals who also belong to the category of the individual (e.g. comparable body mass) provides for benefits that can help the individual identify which speed metric should be targeted for improvement through one or more exercises, with a goal of improving the athletic motion as a whole.
In some implementations, at step 58-3, the observer device 1903 or server 1909 identifies which speed metric should be targeted for improvement through one or more exercises. In the example above shown in Table 2, those speed metrics are shown as underlined for the First Individual and the Second Individual, although other possibilities exist such as highlighting for example, and more generally any suitable way can be utilized to convey which speed metric is to be improved. In some implementations, the observer device 1903 or server 1909 determines the speed metric based on a combination of poor relative performance (e.g. low percentile) and affected body parts being lowest to ground. For the First Individual, the arm speed is chosen primarily because it has poor relative performance at only 5th percentile. For the Second Individual, the pelvis speed is chosen primarily because the pelvis is lowest to ground, even though torso speed has a lower percentile. The precise manner in which speed metric is chosen is implementation-specific. Also, while only one speed metric is identified for improvement, it is noted that other implementations are possible in which more than one speed metric can be identified for improvement.
According to one embodiment, the speed metric having the lowest percentile may be chosen and targeted for improvement through one or more exercises regardless of its position relative to the ground. In other words, if the speed metric having the lowest percentile is the hand speed, then the hand speed will be chosen first for improvement. According to another embodiment, a speed metric having a relative performance less than a predetermined percentile, such as the 30th percentile, will be selected first regardless of its position relative to the ground, and if there is more than one speed metric at less than the predetermined percentile, then the speed metric that is lowest to the ground is selected first for improvement. According to another embodiment: (i) first, the measured body part that is lowest to ground (e.g., pelvis) is targeted for improvement until the speed metric for that body part is above a predetermine percentile, (ii) second, the measured body part that is located above the first measured body part (e.g., torso) is targeted for improvement until the speed metric for that body part is above a predetermined percentile, (iii) third, the measured body part that is located above the second measured body part (e.g., arm) is targeted for improvement until the speed metric for that body part is above a predetermined percentile, and (iv) fourth, the measured body part that is located above the third measured body part (e.g., hand) is targeted for improvement until the speed metric for that body part is above a predetermined percentile.
In some implementations, at step 58-4, the observer device 1903 or server 1909 generates a regime file as similarly described in previous sections. More generally, the observer device 1903 or server 1909 can determine at least one exercise for improving the speed metric, and convey the at least one exercise for the individual to practice. With such focused practice or exercise, which aim to specifically address shortcomings of the athletic motion by individual, the individual may experience improvement. The steps described above can be repeated through several iterations, and the observer device 1903 or server 1909 can determine and conveying an indication of improvement or change in performance. Carrying on with the example described above for the First Individual and the Second Individual, improved speed metrics are provided below.
Pelvis Speed
78
Percentile:
Arm Speed Percentile:
85
As shown above in Table 3, the arm speed for the First Individual has improved significantly, resulting in an improved exit velocity. As also shown above, the pelvis speed for the Second Individual has improved significantly, resulting in an improved exit velocity. The magnitude of improvement is large in both cases. In some implementations, the magnitude of improvement is displayed, for example as a numeric increase in the percentile ranking or other indication on improvement.
In some implementations, the observer device 1903 or server 1909 determines and conveys an overall body speed metric based on a combination of the body speed metrics. In the examples shown above, a “body speed percentile” is calculated as an average of pelvis speed percentile, torso speed percentile, arm speed percentile, and hand speed percentile. However, other functions can be used, such as median or mode or weighted average for example, and more generally any suitable mathematical function can be used to provide an indication of overall body speed. Determining and improving overall body speed is important given, for example, in baseball that fastball velocities continue to increase, along with breaking ball usage.
In some implementations, the observer device 1903 or server 1909 compares exit velocity percentile against the body speed percentile. Increasing the body speed percentile tends to increase the exit velocity percentile. In baseball statistics, exit velocity (EV) is the estimated speed at which a batted ball is travelling as it is coming off the player's bat. Exit velocity is generally measured and presented in miles per hour. Batters generally aim for a higher exit velocity in order to give opposing fielders less time to react and attempt a defensive play. Hitting a ball with the proper force, bat speed and contact is critical to hitting the ball well. Indeed, exit velocity is one of the most important measurements tracked stat in Major League Baseball (MLB) right now. MLB teams use the exit velocity stat to gauge a batter's abilities. Transversely, exit velocity can be analyzed to improve a pitcher's results, especially those prone to giving up hard contact.
In some implementations, the one or more sensors each have an inertial sensor, a local processor, a local data buffer, and a transceiver. The sensor data is initially buffered in the local data buffer, whereby upon recognition that the athletic motion has occurred, the local processor extracts from the local data buffer sensor data in a predetermined time window before and after a moment in which the athletic motion occurred and only the sensor data that is extracted is transmitted to a transceiver of the at least one computer. Example implementation details have been provided in previous sections and are not repeated here.
In some implementations, the one or more sensors include a plurality of sensors networked together so that sensor data can be transmitted from one or more of the sensors to the local data buffer existing in another one of the sensors and an aggregate of the sensor data from the sensors is transmitted from that local data buffer to the transceiver of the at least one computer. Example implementation details have been provided in previous sections and are not repeated here.
In some implementations, the one or more sensors include one or more video cameras (e.g., high speed tracking cameras). For such implementations, pose estimation can be used to determine positioning and/or speed of the individual based on image data from one or more video cameras. In some implementations, use of video cameras and pose estimation is implemented to supplement use of other sensors such as inertial sensors for example. Notably, sensor fusion using a video camera and one or more inertial sensors could improve accuracy of the speed metrics that are calculated for the individual. In other implementations, use of video cameras and pose estimation is implemented instead of other sensors such as inertial sensors. In this manner, it may be possible to determine the speed metrics for the individual solely based on video data from one or more video cameras. It is understood that video may be obtained from a camera coupled with a mobile device, or any camera that is separate from or otherwise remove from the mobile device.
According to another embodiment of the disclosure, there is provided a non-transitory computer readable medium having recorded thereon statements and instructions that, when executed by a processor, implement a method as described herein. The non-transitory computer readable medium can for example include an SSD (Solid State Drive), a hard disk drive, a CD (Compact Disc), a DVD (Digital Video Disc), a BD (Blu-ray Disc), a memory stick, or any appropriate combination thereof.
The illustrated examples described herein focus on software implementations in which software can be executed by a processor of a computer. However, other implementations are possible and are within the scope of this disclosure. It is noted that other implementations can include additional or alternative hardware components, such as any appropriately configured FPGA (Field-Programmable Gate Array), ASIC (Application-Specific Integrated Circuit), and/or microcontroller, for example. More generally, there is provided motion monitoring circuitry configured for implementing any method described herein, which can be implemented with any suitable combination of hardware, software and/or firmware.
The foregoing method descriptions and the process flow diagrams are provided merely as illustrative examples and are not intended to require or imply that the steps of the various embodiments must be performed in the order presented. The steps in the foregoing embodiments may be performed in any order. Words such as “then,” “next,” etc. are not intended to limit the order of the steps; these words are simply used to guide the reader through the description of the methods. Although process flow diagrams may describe the operations as a sequential process, many of the operations can be performed in parallel or concurrently. In addition, the order of the operations may be re-arranged. A process may correspond to a method, a function, a procedure, a subroutine, a subprogram, etc. When a process corresponds to a function, its termination may correspond to a return of the function to the calling function or the main function.
The various illustrative logical blocks, modules, circuits, and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the overall system. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
Embodiments implemented in computer software may be implemented in software, firmware, middleware, microcode, hardware description languages, or any combination thereof. A code segment or machine-executable instructions may represent a procedure, a function, a subprogram, a program, a routine, a subroutine, a module, a software package, a class, or any combination of instructions, data structures, or program statements. A code segment may be coupled to another code segment or a hardware circuit by passing and/or receiving information, data, arguments, parameters, or memory contents. Information, arguments, parameters, data, etc. may be passed, forwarded, or transmitted via any suitable means including memory sharing, message passing, token passing, network transmission, etc.
The actual software code or specialized control hardware used to implement these systems and methods is not limiting of the invention. Thus, the operation and behavior of the systems and methods were described without reference to the specific software code being understood that software and control hardware can be designed to implement the systems and methods based on the description herein.
When implemented in software, the functions may be stored as one or more instructions or code on a non-transitory computer-readable or processor-readable storage medium. The steps of a method or algorithm disclosed herein may be embodied in a processor-executable software module which may reside on a computer-readable or processor-readable storage medium. A non-transitory computer-readable or processor-readable media includes both computer storage media and tangible storage media that facilitate transfer of a computer program from one place to another. A non-transitory processor-readable storage media may be any available media that may be accessed by a computer. By way of example, and not limitation, such non-transitory processor-readable media may comprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other tangible storage medium that may be used to store desired program code in the form of instructions or data structures and that may be accessed by a computer or processor. Disk and disc, as used herein, include compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk, and Blu-ray disc where disks usually reproduce data magnetically, while discs reproduce data optically with lasers. Combinations of the above should also be included within the scope of computer-readable media. Additionally, the operations of a method or algorithm may reside as one or any combination or set of codes and/or instructions on a non-transitory processor-readable medium and/or computer-readable medium, which may be incorporated into a computer program product.
The preceding description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the following claims and the principles and novel features disclosed herein.
While various aspects and embodiments have been disclosed, other aspects and embodiments are contemplated. The various aspects and embodiments disclosed are for purposes of illustration and are not intended to be limiting, with the true scope and spirit being indicated by the following claims.
This patent application which claims priority to U.S. Provisional Application No. 63/242,853, filed on Sep. 10, 2022, which is incorporated by reference herein in its entirety.
| Filing Document | Filing Date | Country | Kind |
|---|---|---|---|
| PCT/US2022/043099 | 9/9/2022 | WO |
| Number | Date | Country | |
|---|---|---|---|
| 63242853 | Sep 2021 | US |
