Patent Application
20230218974
A player of a sport can improve their skills by playing balls launched from a ball machine. For example, a tennis player can use a ball machine to launch tennis balls aimed at exercising their forehand, backhand, etc. In another example, a baseball player can use a ball machine to launch baseballs aimed at exercising various aspects of their batting skills. A player can evaluate their own performance by using their own vision to tell whether or not a desired training goal is achieved, e.g., whether or not a tennis stroke lands in an intended area of a tennis court or whether or not a baseball when hit lands in a desired area of the outfield.
In general, in one aspect, the invention relates to a portable drill instructor for a sport. A portable drill instructor according to the invention can include: a camera that captures a live video of a player of the sport in a venue for the sport playing a ball launched from a ball machine during a training drill for the player conducted in the venue; a computing mechanism that in response to the live video determines whether or not the player satisfies a training goal of the training drill when playing the ball; and a feedback mechanism that provides a real-time feedback to the player during the training drill that indicates to the player whether or not the training goal is satisfied.
In general, in another aspect, the invention relates to a method for portable drill instruction for a sport. The method can include: capturing a live video of a player of the sport in a venue for the sport playing a ball launched from a ball machine during a training drill for the player conducted in the venue; determining in response to the live video whether or not the player satisfies a training goal of the training drill when playing the ball; and providing a real-time feedback to the player during the training drill that indicates to the player whether or not the training goal is satisfied.
Other aspects of the invention will be apparent from the following description and the appended claims.
Embodiments of the present invention are illustrated by way of example, and not by way of limitation, in the figures of the accompanying drawings and in which like reference numerals refer to similar elements.
Reference will now be made in detail to the various embodiments of the present disclosure, examples of which are illustrated in the accompanying drawings. Like elements in the various figures are denoted by like reference numerals for consistency. While described in conjunction with these embodiments, it will be understood that they are not intended to limit the disclosure to these embodiments. On the contrary, the disclosure is intended to cover alternatives, modifications and equivalents, which may be included within the spirit and scope of the disclosure as defined by the appended claims. Furthermore, in the following detailed description of the present disclosure, numerous specific details are set forth in order to provide a thorough understanding of the present disclosure. However, it will be understood that the present disclosure may be practiced without these specific details. In other instances, well-known methods, procedures, components, have not been described in detail so as not to unnecessarily obscure aspects of the present disclosure.
Examples of the real-time feedback 174 include audio feedback, visual feedback, haptic feedback, etc. In one or more embodiments, a particular sound indicates to the player 120 that the training goal 156 is satisfied while a different particular sound indicates that the training goal 156 is not satisfied.
In one or more embodiments, the feedback mechanism 172 generates synthesized speech indicating whether or not the training goal 156 is satisfied. In some embodiments, the synthesized speech includes a motivational aspect, e.g., “good shot”, “missed”, “third straight miss”, “you're on a roll”, etc.
In one or more embodiments, the feedback mechanism 172 employs an audio subsystem in the portable drill instructor 100, e.g., internal speaker and amplifier. In other embodiments, the feedback mechanism 172 employs an external speaker, earphones, or other device worn by the player 120 using, e.g., via Bluetooth communication.
In one or more embodiments, the real-time feedback 174 informs the player 120 of a recommended corrective action. Examples of a corrective action using synthesized speech include “hold your racquet higher”, “keep your head steady”, “you're hands are too close to your body”, etc.
The portable drill instructor 100 in one or more embodiments is implemented as a mobile app adapted for a particular sport, e.g., tennis, table tennis, baseball, football, soccer, etc., for a mobile device, e.g., a smartphone, tablet, etc. A user, e.g., the player 120, mounts the mobile device in a suitable position with respect to the venue 130, e.g., behind a baseline of a regulation tennis court, behind a baseball batting cage, etc., so that a camera in the mobile device captures the flight of the ball 150 and the movements of the player 120.
The portable drill instructor 100 conducts the training drill 154 to train the player 120 in any one or more aspects of the particular sport. In a tennis embodiment, for example, the training drill 154 can train the player 120 to hit forehands, backhands, cross-court shots, volleys, service returns, overheads, etc. In a baseball embodiment, the training drill 154 can train the player 120 to hit fastballs, curveballs, pitches in various locations, pulling the ball, hitting to the opposite field, etc.
The training goal 156 can pertain to any aspect of how the player 120 plays the ball 150 launched from the ball machine 140, e.g., where the ball 150 lands after being hit by the player 120, the flight of the ball 150 after being hit by the player 120, the positions and movements of a racquet, bat, etc., used by the player 120 to play the ball 150, positions and movements pertaining to the body of the player 120, e.g., hand and arm positions and movements, footwork, position and movement, etc. In the following, for purposes of illustration, the positions and movements pertaining to the player 120 and the ball 150 and the relevant objects involved in the particular sport and layout and features of the venue 130 are represented in a three-dimensional (3D) space 132 above a surface of the venue 130.
For example, if the training goal 156 for a cross-court forehand is satisfied, the computing mechanism 160 can switch the new training goal 456 to a cross-court backhand and adjust the ball machine 140 to launch the next ball for a cross-court backhand. On the other hand, if the training goal 156 for a cross-court forehand is not satisfied, the computing mechanism 160 can cause the ball machine 140 to launch the next ball for another cross-court forehand. In one or more embodiments, the computing mechanism 160 causes the ball machine 140 to repeatedly launch balls for, e.g., cross-court forehands until a threshold percentage of the return trajectories satisfy the training goal 156 before moving on to other skills.
In one or more embodiments, the ball machine 140 is controllable by the portable drill instructor 100, e.g., using wireless communication, e.g., Bluetooth, Wi-Fi, etc., so that the computing mechanism 160 controls how balls are launched, e.g., direction and speed of launch.
In one or more embodiments, the computing mechanism 160 increases or decreases the intensity of the training drill 154 in response to the health data 460 depending on whether the player 120 is working too hard or not working hard enough or is missing too many of the training goals, etc., by adjusting how the ball machine 140 launches balls at the player 120 during the training drill 154. For example, the computing mechanism 160 can adapt a set of training parameters 470 for the training drill 154 that control the duty cycle of the ball machine 140, how the player 120 is moved around the venue 130 during the training drill 154, e.g., from side to side along the baseline in tennis, etc.
The mobile device 700 in one or more embodiments uses the feedback mechanism 172 to relay the real-time viewer feedback 790 to the player 120. For example, the viewer 782 may be a coach or a friend who views the live video 112 via the cloud and verbally coaches or encourages the player 120 remotely.
The mobile device 700 in one or more embodiments adapts the training drill 154 in response to the real-time viewer feedback 790. For example, the viewer 782 may be a coach who views the live video 112 via the cloud and adjusts the training drill 154 to the training needs of the player 120, e.g., by increasing or decreasing shot tempo or emphasizing particular drills, e.g., forehand cross-courts, etc.
In one or more embodiments, the neural network 860 is trained to correlate particular arrangements of pixels in the image frames of the live video 112 to the 3D property 870. For example, the neural network 860 is trained to correlate one particular arrangement of pixels to one particular 3D position of the ball 150 and to correlate another particular arrangement of pixels to another particular 3D position of the ball 150.
A series of 3D positions of the ball 150 extracted from the live video 112 by the neural network 860 indicate a flight of the ball 150 in the 3D space 132. Likewise, a series of extracted 3D positions/orientations of the racquet held by the player 120 indicate how the player 120 strokes the ball 150 through the 3D space 132
The neural network 860 is trained using a set of acquired training data 850. The training data 850 includes sets of 2D training videos and corresponding 3D measurements of relevant objects captured during a set of prior training events. The acquired timestamped 2D training images and timestamped 3D sensor measurements are used to iteratively train a set of model parameters of the neural network 860. The model parameters can include, e.g., node structure parameters and weights of connections among the nodes.
At step 910, a live video is captured of a player of a sport in a venue for the sport playing a ball launched from a ball machine during a training drill for the player conducted in the venue. At step 920, whether or not the player satisfies a training goal of the training drill when playing the ball is determined in response to the live video. At step 930, a real-time feedback is provided to the player during the training drill that indicates to the player whether or not the training goal is satisfied.
While the foregoing disclosure sets forth various embodiments using specific diagrams, flowcharts, and examples, each diagram component, flowchart step, operation, and/or component described and/or illustrated herein may be implemented, individually and/or collectively, using a range of processes and components.
The process parameters and sequence of steps described and/or illustrated herein are given by way of example only. For example, while the steps illustrated and/or described herein may be shown or discussed in a particular order, these steps do not necessarily need to be performed in the order illustrated or discussed. The various example methods described and/or illustrated herein may also omit one or more of the steps described or illustrated herein or include additional steps in addition to those disclosed.
While the invention has been described with respect to a limited number of embodiments, those skilled in the art, having benefit of this disclosure, will appreciate that other embodiments may be devised which do not depart from the scope of the invention as disclosed herein.
