Patent Application
20230264092
Trademarks used in the disclosure of the invention and the applicants make no claim to any trademarks referenced.
The present invention relates to sporting equipment and includes a combination of a base unit, including a mobile device or terminal, a mobile application within a mobile device or terminal having a graphical user interface (GUI) for displaying formatted data, a data collection and storage component, and an analysis engine for capturing, aggregating, synthesizing, analyzing and processing data.
“Putting” in golf refers to the practice of hitting a golf ball along the ground or “green” into a hole, an activity requiring practice to develop the feel for line, speed, and accuracy. Putting is the most utilized aspect of the game of golf, considering it must be done on every hole to complete the hole. As such, about 45% of an average golfer's scores are putting strokes. Current products aim to assist golfers in alignment, best in practice stroke techniques, building muscle memory, distance control, reading greens, among other areas, all of which are primarily involved in the measure of and influence on the swing of the putter.
There are a variety of passive products aimed at enhancing a user's golf putting practice, among them floor mats and ramp constructions with actual or simulated holes. While ramp constructions with actual holes can provide a semi-realistic experience, they generally require a large space to set up and use. Floor mats which are flush with the ground can be more compact but rely on flat or shallow simulated target holes, making it sometimes difficult to tell if a shot was made or missed. In addition, because these are passive apparatuses, a user desiring some record of their putting practice must observe and record each shot manually. At best, there are products that simply count the number of strokes you make, but not in relation to the total number of attempts. Further, such conventional products do not have a means to collect, process and analyze putting statistical records so as to provide the user with actionable feedback for improving the user's performance. There exists no platform or solution in this domain that uses multivariate putting performance data or analysis which is based directly on data-based proficiency of putting a ball to a target; nor is there computer-driven modeling that correlates this performance data and analysis to produce instructional guidance specific to each user so as to inform a user to their specific low-performance areas to focus on improving their putting.
An active device could make use of electrical sensors to accurately and automatically determine the position and/or velocity of a putted golf ball with reference to a virtual hole. There are a variety of mechanical sensors which can be used to sense impact force and/or location, among them accelerometers which detect movement, vibration, or impact; and piezoelectric sensors which produce an electric charge when deformed by a force. In addition, a variety of remote sensors exist which are able to determine the position and velocity of an object over a distance. These are broadly referred to as motion detectors. They include passive infrared sensors, which measure infrared light radiating from objects, microwave and ultrasonic motion detectors which emit microwave radiation and high-frequency sound waves, respectively, and then detect resulting reflections, and digital video cameras combined with a computer program which applies motion analysis to the generated videos, allowing for object tracking. This type of analysis is stateless in that once the data about the immediate putt has been analyzed and displayed, it is not retained in memory for later analysis with additional data.
Mobile devices or terminals such as smart phones, which are now widely-used, combine a microprocessor and computer memory with several methods of remote communication (e.g. a network connection, Z-Wave®, 6LoWPAN®, Thread®, NFC®, Dash7®, RFID®, WIFI, Bluetooth®, Zigbee®, cellular networks or other wireless communication channel), a camera, and a user-interface which accepts and displays information. This makes them an excellent platform for interfacing with a base unit and objects. In addition, they provide storage of and access to information in a local, convenient location for a user.
The invention is an apparatus combining a base unit having a transceiver, the base unit with a camera interfaced via a wireless or wired channel to a standalone mobile device or terminal, the mobile device or terminal operable to (i) receive video data from the base unit, and (ii) store, aggregate, process, format and display the processed, formatted data to the user at the mobile device or terminal. The invention is further operable to analyze the data using a variety of analysis techniques and present the analyzed data to the user in a format so as to allow the user to improve putting performance such as, but not limited to assisting golfers in understanding data and analytics of their performance results to identify specific putting strengths, and more importantly putting weaknesses. From this awareness of weaknesses, instruction is provided that addresses specific tendencies related to alignment, stroke techniques, muscle memory, distance control and reading greens which supports skill improvement in these areas.
The camera of the instant invention can be either a user-facing or position facing directly down at the target from the target poll so that it is capable of recording a 360-degree view of the green and surrounding area.
The invention further comprises an apparatus having a base unit with transceiver, mobile application including a graphical user interface (GUI) and analysis engine which utilizes the aggregate data from the base unit for providing various analysis, such as long term putting made/miss performance statistics (stratified by distance, percentage putts made, surface slope or gradient to the target, break (right-to-left slope), proximity (distance) from target (hole), compared to professional users' statistics, etc.) The analysis platform aspect of the invention is operable to collect, aggregate, synthesize, analyze and process data from the base unit and mobile application so as to identify specific low-performance putting statistics and provide the user with actionable feedback for improving these areas of user's performance. The processing and analysis of the data can occur in a processor in the base unit or in a processor in the mobile device or terminal or between both of them.
The invention uses, among other things, video data plus actual ball performance data. A variety of products exist that employ video alone for subjective assessment, meaning the user is only able to view the video to determine by self-diagnosis any abnormalities which are obvious to the unaided eye to correct. Other products exist that use video data in tandem with sensors that capture data about the user physical mechanics or the mechanics of the sporting equipment, such as the golf putter during user's use. Output data can be specific to the user or the equipment such as swing path, angles, swing speed; analytics regarding the proficiency of the user compared to best practices or professional metrics; feedback that contrasts the user to the best mechanics in the sport. The other products target relevant swing mechanics, but not the specific data of the putted ball or analytics of results of the putted ball.
Alternatively, the system can also be used to capture and analysis the data with regards to shots made to the green with full swing clubs. To accomplish this the data capture camera system is position on the flag which is located at the target and the camera using video techniques captures the ball from its initial position to the target.
In the invention, video recording is used to sense motion and cue the video; capture the ball (not swing data) from point of origin to end point; and determine spatial orientation of ball relative to target, including based on coordinates such as Cartesian, polar, or other 2-dimensional plotting system; end point: made/missed; missed: left/right, short/long; speed and velocity of the ball when impacting target compared to an acceptable speed range; spin of the ball; lift of the ball off club face; and a basic visual representation of the putting stroke, without data.
The analytics engine of the invention receives user positioning and motion, ball position, ball velocity, and ball acceleration data, hole location and gradient information, between varying locations, including user location and the hole location and performs a multivariate analysis of all variables captured. The data is, inter alia, conditioned, normalized, optimized, and then processed, formatted and made available for storage and display on a mobile device or terminal through an application program, including via a mobile device application.
The invention further is operable to allow correlation of analytics to learning modules or instructional databases, such that it curates or matches instructional resources to users directly based on the user's ball data, performance, and skill deficiencies.
The invention is further operable as an entertainment game that enables comparison of data for multiple users, such as for a golf team of a plurality of players engaging in a practice game; head-to-head competition either live on one device, or virtually across local or wide area wireless or wired networks and simulated game play against professional golf statistics or against a user's historical stat averages.
The invention includes a base unit, or a base unit comprising, a protective housing, an actual or simulated target golf hole, a means for sensing the position of a golf ball at rest and velocity and location of a golf ball as it moves from the at rest position to the target in proximity to said base unit relative to said target, a microprocessor and associated signal conditioning electronics, a computer program being executed by said microprocessor, means for communicating data to and from a standalone interfacing device if applicable (e.g. a network connection, Z-Wave®, 6LoWPAN®, Thread®, NFC®, Dash7®, RFID®, WIFI, Bluetooth®, Zigbee®, cellular networks or other wired or wireless communication channel), and a power source; and a standalone interfacing device (e.g. a personal computer, tablet, smart phone or user equipment or user terminal) comprising a microprocessor, a computer program comprising instructions for being executed by said microprocessor, a means for accepting user input, a means for storing data, a means for communicating data to and from said base unit, a means for displaying information, and a power source; and an analytics engine comprised of software running on a processor for synthesizing and analyzing data from the base unit and mobile application so as to provide the user with actionable suggestions for improving the user's performance. The collection, storage, processing, and analysis of the data can occur in a processor in the base unit or in a processor in the mobile device or terminal. Alternatively, the data can be sent via the mobile device or terminal to a “cloud” provider or hosted server on a wide area network for storage, processing and analysis, the analyzed data then formatted for display back at the mobile device or terminal.
To those skilled in the art to which this invention relates, many changes in construction and widely differing embodiments and applications of the invention will suggest themselves without departing from the scope of the invention as defined herein. The disclosures and the descriptions herein are purely illustrative and are not intended to be in any sense limiting.
For a better understanding of the present invention including the features, advantages and embodiments, reference is made to the following detailed description along with accompanying Figures, in which:
In the following description, numerous specific details are set forth. However, it is understood that embodiments of the invention may be practiced without these specific details. In other instances, well-known structures and techniques have not been shown in detail in order not to obscure the understanding of this description. Those of ordinary skill in the art, with the included descriptions, will be able to implement appropriate functionality without undue experimentation.
References in the specification to “one embodiment” or “an embodiment,” may indicate that the embodiment described may include a particular feature, structure, or characteristic, but every embodiment may not necessarily include the particular feature, structure, or characteristic. Moreover, such phrases are not necessarily referring to the same embodiment. Further, when a particular feature, structure, or characteristic is described in connection with an embodiment, it is submitted that such feature, structure, or characteristic may be deployed in connection with other embodiments whether or not explicitly described.
Lastly, the terms “or” and “and/or” as used herein are to be interpreted as inclusive or meaning any one or any combination. Therefore, “A, B or C” or “A, B and/or C” mean “any of the following: A; B; C; A and B; A and C; B and C; A, B and C.” An exception to this definition will occur only when a combination of elements, functions, steps, or acts are in some way inherently mutually exclusive.
As this invention is susceptible to embodiments of many different forms, it is intended that the present disclosure be considered as an example of the principles of the invention and not intended to limit the invention to the specific embodiments shown and described.
The terms people, user, golfer, client, customer, and individual are used interchangeably to mean an individual who uses the invention.
The terms Wi-Fi and WIFI are used interchangeably and are intended to mean the family of wireless network protocols, based on the IEEE 802.11 family of standards, which are commonly used for local area networking of devices and Internet access.
The terms wireless protocols are used interchangeably and are intended to mean a wireless network protocols as used in, Z-Wave®, 6LoWPAN®, Thread®, NFC®, Dash7®, RFID®, WIFI, Bluetooth®, Zigbee® and cellular networks.
The term target can be any object or feature that is the target of a putted golf ball.
The term instruction as used in the disclosure means teaching or education performed by a teacher or an automated educational system.
The term ball and golf ball are used interchangeably and are intended to mean golf ball.
The term golf club and club are used interchangeably and are intended to mean a golf club.
The terms golf shot and shot are used interchangeably and are intended to mean a golf shot whereas a user strikes the golf ball with a golf club causing it to move towards a target.
The terms drills, teach and teaches as used in the disclosure are intended to mean to impart knowledge of or skill in; give instruction in a subject. The teacher can be an automated educational system.
The term slope as used in the disclosure are used interchangeably and are intended to mean the slope of the terrain from the golf ball to the target (up-down) and the slope of the terrain left and right of the target or the slope of the terrain towards and away from the target.
The term brake as used in the disclosure are used interchangeably and are intended to mean the slope from left to right or right to left of the terrain of the putting green or the movement of the golf ball from left to right or right to left depending on the slope of the terrain of the putting green.
The term instant invention and system software as used in the disclosure are used interchangeably and are intended to mean the system of the instant invention.
The instant invention provides as one feature an automated educational system for putting.
References in the specification to “one embodiment” or “an embodiment,” may indicate that the embodiment described may include a particular feature, structure, or characteristic, but every embodiment may not necessarily include the particular feature, structure, or characteristic. Moreover, such phrases are not necessarily referring to the same embodiment. Further, when a particular feature, structure, or characteristic is described in connection with an embodiment, it is submitted that such feature, structure, or characteristic may be deployed in connection with other embodiments whether or not explicitly described.
While the making and using of the disclosed embodiments of the present invention is discussed in detail below, it should be appreciated that the present invention provides many applicable inventive concepts which can be embodied in a wide variety of contexts. Some features of the preferred embodiments shown and discussed may be simplified or exaggerated for illustrating the principles of the invention.
The prior art is replete with invention is not a ball launch monitor system such as Nipper et al. U.S. Patent application U.S. 20090036237 which are hereby incorporated by reference herein for all purposes. The instant invention analysis the motion of the ball and not the motion of the athlete or their club. Nipper et al. discloses methods, locations, installations, devices, and systems adapted and arranged for observing, processing, analyzing and communicating data and images of individual or group golf gaming performance with respect to golf balls in play in three-dimensional environments. More specifically the system disclosed in Nipper et al. is a ball launch monitor system at the tee site, and employing software capable of generating a struck ball's flight model, the invention gathers and integrates data regarding one or more parameters that influence the flight path of the struck ball, such as the ball's vertical and horizontal launch angles, launch velocity components, aerodynamic flight properties such as ball spin velocity and direction, measured wind direction and velocity, and current air density to predict the flight path and likely landing location for the ball.
Other prior art identifies, quantify, and analyze motion such as the U.S. Patent application by Tremblay-Munger et al. U.S. 2015/0057111 which is hereby incorporated by reference herein for all purposes. The instant invention analysis the motion of the ball and not the motion of the athlete or their club. Tremblay-Munger discloses a method for identifying and quantification motion, using a sensing unit comprising an inertial sensor for providing acceleration and rotation data in three axis, a microprocessor executing algorithms for identifying and quantifying motion and memory means, located on a user or on a stick, comprising the steps of: automatic activation by a motion of the sensing unit, starting motion detection, wherein said sensor unit is initialized and starts real-time monitoring and analysis of sensing data from the inertial sensor, starting a potential motion event, when the algorithm detects and identifies a motion pattern associated to a backswing motion, starting a potential motion acceleration event, when the algorithm detects and identifies a motion pattern that is greater in value than a predetermined lowest threshold and lower in value than a higher threshold, starting a potential motion continuation event, when the algorithm detects and identifies a motion pattern associated to a potential downswing motion, starting potential impact detection metric, by which the algorithm monitors the variation of the sensing data over an impact detection period, and log the event as a potential impact when the variation reaches a predetermined level, starting impact detection metric, by which the algorithm detects and identifies an impact event following a downswing motion within a predetermined time period, starting shot detection, by which the algorithm confirms an impact event, and quantifies the motion metrics associated with the event. Paragraph [0044]. As disclosed, the system and methods of Tremblay-Munger are specific to the analysis of the motions of the individual and the stick being used by the individual and not the path of the item struck, such as a hockey puck or golf ball.
The invention, as more fully described herein, is a golf instruction apparatus and analytics platform embodied in one of a base unit alone, a base unit having certain functionalities in combination with a mobile device or terminal, a base unit having certain functionalities in combination with modules in a remote server; a base unit having certain functionalities in combination with modules in a mobile device or terminal and a remote server, a mobile device or terminal alone; or a mobile device or terminal having certain functionalities in combination with modules in a remote server.
In certain such combinations above, the base unit housing comprises a form factor, two examples of which are seen in
The base unit 100 comprises a housing unit 102 being made of a rigid material, preferably a hard plastic such as polyvinyl chloride. Within the housing unit is a base unit microprocessor (S-MPC) running camera driver firmware, means for remotely communicating video data to and from a standalone interfacing device, and an infrared or optical sensing means such as a camera or recording video for playback in mobile application, and a power source or sources (e.g. a battery).
The base unit 100, 300 incorporates an optical sensing means 101, 301, a microprocessor and a wired or wireless transceiver. The interfacing device to the base unit 100, 300 is preferably a mobile device or terminal such as a smart phone, running an associated computer program. As seen in
The invention further comprises a mobile application running on a mobile device or terminal, such as a smartphone, the mobile device or terminal comprising a device microprocessor (D-MPC), a device computer program being executed by said D-MPC, a means for accepting user input, a means for storing data, a means for communicating video data to and from said base unit (e.g. a wired or wireless network connection, Bluetooth®, cellular or other wired or wireless communication channel), a means for displaying information including results of the putt and the video playback, and a power source(s). The device computer program comprises instructions being stored on a computer readable medium to be executed by a microprocessor to perform the following functions: querying for and accepting user input (e.g. putt length, planned number of shots in current session), communicating with the D-MPC in said base unit to trigger certain events, manipulating video data, and formatting said video data for output via said means for displaying information.
In an embodiment, said means for sensing positioning and velocity of a putted golf ball in close proximity to user-facing side of said housing comprises one or more remote motion detectors, e.g. an optical sensor such as a passive infrared detector or video camera, or an active radiating sensor such as an ultrasonic motion detector or a microwave motion detector. In one aspect of the invention, the camera on a mobile device or terminal, such as a smart phone, is used to capture the video, the camera being positioned on the base unit and in another aspect of the invention, the camera is integrated into the base unit and the video is communicated from the base unit, along with sensor data, to the mobile device or terminal.
In a further embodiment, said means for sensing positioning and velocity of a putted golf ball in close proximity to user-facing side of said housing comprises three remote motion detectors in the base unit located to the left, right, and rear respectively of a simulated golf hole. The invention further comprises a display for displaying the output results.
The golf instruction apparatus and analytics platform can include a base unit coupled to a mobile device or terminal over a wired or WIFI, Bluetooth®, cellular or similar radio spectrum link. The base unit further comprises a housing being made of a rigid material, preferably a hard plastic such as polyvinyl chloride, a sensing circuit for sensing position and velocity of a putted golf ball in close proximity to a user-facing side of said housing, a microprocessor running base unit firmware for driving camera input/output (I/O), a signal conditioning circuit for conditioning sensor data such that it is readable by said microprocessor, a transceiver for remotely communicating video data to a mobile terminal or device, a recording module for video recording user's putting stroke from the vantage point of simulated target hole; and a power source.
The invention includes an analytics engine as more fully described below. As used herein, “multivariate” generally means the use, analysis, and correlation of two or more different data points in combination. “Multivariate putting performance data and analysis” means the numerical and mathematical measures of a golf ball putted towards a golf hole or simulated target, so that any combination of variables collected may be cross-analyzed with any others to provide more granular feedback on specific skill areas needing improvement. For example, multivariate analysis may determine a user's worst performance skill segment is “when putting from six feet from target with Left to Right slope, user misses right 70% and short 83%.” Contrast this with a single variable that shows a user on average “misses right 70%” over all putts recorded. The more meaningful and accurate feedback is the multivariate analysis because in that example, the Left to Right slope variable imparts an otherwise non-obvious influence on putting performance since more balls on right sloping terrain that are putted short (83%) will fall right of the hole due to gravity. So, the multivariate analysis points to the high miss percentage most likely being a speed/distance issue since 83% were short of the target. But if only the single variable were reviewed, in this case (70% right misses), it can mislead the user about the cause of the missed putts, for example, that perhaps they are pushing the ball right.
The analytics engine of the invention includes application software executed by at least one processor that enables the analysis of both individual data points and multivariate analysis of any number of variables. Such data analysis provides broader and more practical feedback since there are inherent interdependencies and influences across each of the different variables. The breadth of the invention's analysis operations can be best understood this way: there are at least 3 primary categories of variables for “made” putts, and at least 8 for “missed” putts, with over 70 total subcategories. This enables thousands of combinations of multivariate analysis of factual ball-related performance data.
The analytics engine of the invention collects video data either in-device or via customized mobile phone camera/software and integrates it with aggregate user performance data and performs a multivariate analysis thereof so as to provide actionable feedback to a user. More specifically, the invention is a method and apparatus employing video capture to record users putting stroke and determine the state of a putted golf ball relative to a target, and remotely displaying the same comprising a camera(s), base unit, a standalone interfacing device, and an associated computer program.
In an embodiment, the golf instruction apparatus and analytics platform invention herein comprises a base unit or mobile device or terminal operable to receive stimuli related to the state of a golf ball, a processor coupled to the base unit operable to digitize said stimuli into input data; and an analytics engine comprising software in the form of computer instructions stored on a computer readable medium and executed by a processor, the analytics engine operable to:
(i) correlate the input data with at least one external variable,
(ii) using an algorithm, process the correlated input data and at least one external variable to obtain a result and
(iii) output the result.
The invention further includes a data analytics engine that comprises software in the form of instructions to be executed by a processor. The invention, in the form of a system comprising the base unit, including at least one camera and/or sensor, and the mobile application executable on a mobile device or terminal, is operable to capture, aggregate, analyze, and then correlate a user's actual skill proficiency and ball performance during use. The invention is operable to capture video data from the base unit or mobile device or terminal, save the data in a local memory unit therein or in a central server via a network connection, the data segregated by user for individual user access, and analyze user data to provide the user practical information.
Referring now to
A two variable analysis 502 shows percentages based on the number of putts taken and missed left side versus number of putts taken and missed right side. The percentages correlate to cells 607 and 608 in
A two variable analysis 503 shows percentages based on the number of putts taken and missed short versus number of putts taken and missed long. The percentages correlate to cells 610 in
A three plus variable analysis 504 shows percentages based on the number of putts taken and missed short and left. The percentage correlates to cell 611 in
A three plus variable analysis 505 shows percentages based on the number of putts missed left and short for left-to-right sloping attempts. The percentage correlates to cell 604 in
A three plus variable analysis 506 shows percentages based on the number of putts missed left and short for left-to-right sloping attempts at 2-4 feet, for example, from target or hole. The percentage correlates to cell 602 in
A three plus variable analysis 507 shows percentages based on the number of putts missed left and short for left-to-right sloping attempts at 2-4 feet in the most recent 30 days.
The location of a missed putt is determined by orientation to the hole by coordinates wherein the coordinates are Cartesian, polar, or some other coordinate system that plots the ball's location in proximity to the target. Statistical percentages can also be provided based on the user's proficiency with “2-putting”, defined as putting the ball in the hole on the next (or 2nd) putt attempt immediately after a missed putt.
Permutations of all of the above, further modified by the length of the putt, timespan of practice (i.e., last 30 days, last 120 days), gradient of slope between the beginning of the putt and the hole, brake (e.g., level, left-to-right (LR), or right-to-left (RL)). Each set of variables related to a putt is referred to as a putt context. Similar putting contexts can be grouped and analyzed together by the analysis engine.
The analysis engine is operable to calculate a user's lowest performance segments or contexts as focus areas for practice. The analysis engine is further operable to provide trend data for each data class or context. For example, trend data may show that a user's putts made from less than 10 feet improved from 43 percent to 45 percent over last 90 days and 1,373 attempts.
A further embodiment of the invention incorporates a machine learning module. The stimuli (referred to as a “feature” in machine learning as described herein) is at least one selected from the group consisting of location of the golf ball relative to a start position and an actual or virtual target or target hole, speed, velocity, acceleration and deceleration of the golf ball over a selected length of travel, golf ball force at any time prior to and upon impacting or reaching the actual or virtual target or target hole, spin of the golf ball at any time prior to and upon reaching or impacting the actual or virtual target or target hole and lift of the golf ball off a club face. The at least one external variable (also a “feature”) is one selected from the group consisting of user position, user motion, club position, club motion, actual or virtual target location, actual or virtual target hole location, gradient between selected locations, wind velocity, wind acceleration, characteristics of a putting surface, including attributed speed based on the measurement of static and dynamic co-efficient of friction and obstacles. At least one of the at least one external variables are previously stored in a memory accessible by the processor. Alternatively, or in addition, at least one of the at least one external variables are sensed, collected, and stored in a memory accessible by the processor contemporaneously with the receipt of stimuli related to the state of the golf ball. The external variables comprise spatial orientation of ball relative to a target, speed and velocity of the ball, when impacting target compared to an acceptable speed range, spin of the ball, lift of the ball off club face, and a visual representation of the putting stroke.
According to Wikipedia, machine learning uses statistical techniques to give computer systems the ability to “learn” (i.e., progressively improve performance on a specific task) with data, without being explicitly programmed. Machine learning is closely related to (and often overlaps with) computational statistics, which also focuses on prediction-making through the use of computers. It has strong ties to mathematical optimization, which delivers methods, theory, and application domains to the field. Within the field of data analytics, machine learning is a method used to devise complex models and algorithms that lend themselves to prediction; in commercial use, this is known as predictive analytics. These analytical models allow researchers, data scientists, engineers, and analysts to “produce reliable, repeatable decisions and results” and uncover “hidden insights” through learning from historical relationships and trends in the data.
An algorithm which can be used with the invention to improve its performance and user suggestions is TensorFlow, a machine learning open source application. Using a neural network algorithm, over time the invention builds data sets of stimuli related to the state of a golf ball and external variables that act upon the golf ball and maps them to the final result of the golf ball. The data sets are thus training data sets that are improved each time a user uses the invention.
The data sets are stored as tabular data formatted as comma-separated values (CSV). The first line of a CSV is a header containing information about the dataset, including features, which are float numbers to hold information about the stimuli and the external variables and a label, which is a final outcome, result, prediction, or suggestions. These data sets are parsed and combined to provide a single tensor, which is the label, using a model coded in Python. A model is the relationship between “features” and the “label”. Traditional programming techniques (for example, many conditional statements) can be used to create a model. However, this requires an analysis of datasets over a long period of time to determine the relationships between golf ball measurements, external variables, and the final result. The machine learning approach of this invention determines the model. Neural networks can find complex relationships between features and the label. It is a highly structured graph, organized into one or more hidden layers.
Each hidden layer consists of one or more neurons. There are several categories of neural networks and an embodiment of the invention uses a dense, or fully connected neural network: the neurons in one layer receive input connections from every neuron in the previous layer. When the model is trained and then receives stimuli and external variables data, it yields predictions and suggestions to the user.
Training is the stage of machine learning when the model is gradually optimized, or the model learns the dataset. The invention learns enough about the structure of the training dataset to make predictions about unseen data. The invention can implement either supervised machine learning where the model is trained from examples that contain labels or unsupervised machine learning where the examples don't contain labels. Instead, the model typically finds patterns among the features (stimuli and external factors).
The invention uses a training loop to feed dataset examples into the model to help it make better predictions. Using TensorFlow, the invention implements a code block to set up the training steps:
(1) Iterate each epoch. An epoch is one pass through the dataset.
(2) Within an epoch, iterate over each example in the training Dataset grabbing its features (x) and label (y).
(3) Using the example's features, make a prediction and compare it with the label. Measure the inaccuracy of the prediction and use that to calculate the model's loss and gradients.
(4) Use an optimizer to update the model's variables.
(5) Keep track of statistics for visualization.
(6) Repeat for each epoch.
Over time, the invention is trained so that when it receives certain combinations of stimuli related to the state of a golf ball and external variables, it provides a certain output or result comprising user performance.
The output result of the machine learning module comprises a user's putting performance relating to user and club alignment, user and club position, stroke technique, muscle memory and distance control (each being a “label”) as further described herein.
Image Capture. Processing and Analysis Referring to
In one aspect of the invention, the camera on a mobile device or terminal, such as a smart phone, is used to capture the video data 808, the camera being positioned on the base unit and in another aspect of the invention, the camera is integrated into the base unit and the video data 808 is communicated from the base unit, along with sensor data, to the mobile device or terminal.
Another embodiment utilizes a mobile device alone without a separate base unit, and such mobile device is placed in a stand or platform so as to both capture, process and analyze video data with statistical data in the multivariate analysis, or to capture, partially process and communicate data to the cloud (a distributed or remote processor or server networked to the mobile device or terminal), whereupon such data is further processed and made available (pushed or pulled) to the user locally or remotely. In this example embodiment, the video data is used to determine which putts are made 809 and which are missed left 810A, and of those, which are short 811A and long 811B and those which are missed right 810B, and of those, which are short 812A and which are long 812B.
The components of the invention include several video aspects. In one aspect, an unaided video camera records the user's putting stroke for subjective self-assessment by user or sharing externally through any conventional means of communication such as email, text, and social media. Video review is available in real-time and without tethered/coupled motion, capture elements either on the putter or user, sensor data, other inputs, or in-device validations.
A further video aspect of the invention comprises image/motion capture. This aspect measures factual user performance of the putted golf ball in relation to a target during use, records result and metrics such as ball accuracy, path, spin, lift, speed, and spatial orientation. None of the above metrics are necessarily based on calculated or predicted outcomes, nor based on any sensor-based measures of equipment or user motion. Video analysis and/or image processing is used to determine metrics and analyze actual ball data from which to draw meaningful conclusions that a user can use to improve putting skills.
Referring to
As noted in
Further, the invention uses video clips only for subjective self-analysis and “tagged” to allow the user to review strokes based on an understanding of the data. For example, assume a user performs 20 putts of which the user makes 14 and misses 6, 5 of which are left misses. The user may wish to review the 5 left miss video clips to observe issues that caused the misses, so as to inform what aspects are in need of correction. To do this, the videos are simply tagged with the related result, “made” “missed left.” The invention does not formulate comparison data, rather it uniquely captures outcome data that is analyzed as seen in steps 706, 707, 708 and 814 by the analytics engine for immediate performance feedback as seen in
The invention, in operation, is as follows according to an embodiment seen in
The invention is operable to store results of user putts in batches, the quantity chosen by the user before each use. So, all putts taken in a batch are stored, analyzed, and displayed to the user on the mobile device or terminal in near-real time. For example, as seen in
The invention then aggregates factual data and then calculates the user's actual performance. Data is captured locally based on the actual movement of the ball and is not extrapolated by a) utilizing a remote database of archived reference data, or b) predictive calculation based on stroke technique or characteristics other than those related directly to the ball.
The analytics engine of the invention is further operable to correlate performance results to instructional resources so as to provide curated instructional resources to user for purchase and compare user performance data against professional golf putting statistics.
The invention provides actionable feedback to the user to enable self-diagnosis and device-generated guidance on areas of practice by providing actual performance data along with video footage of stroke at ground level to observe set up or stroke flaws.
In addition to being a training system and platform, the invention is also an entertainment platform as it is operable to permit multiple players to compete against each other either in alternating player format or virtually across the internet using network connectivity. The invention is further operable as a standalone, unaided video analysis system.
The invention provides results based on ball movement, regardless of the stroke or swing used. As a result, the invention provides user feedback that is based on performance, whereas conventional training systems provide feedback based on technique. Hence, a user with an incorrect stroke will benefit from the invention's accurate measurement of factual results. In contrast, conventional systems incorporate sensor data and video to provide feedback to the user specific to aspects of their technique even if improper technique may provide better actual performance results for the user. The invention does not provide data related to appropriate or desired techniques for using equipment. The invention is further differentiable from conventional systems in that the input and output to the invention is based on image and data capture of actual ball movement after being engaged by the user whereas conventional systems use calculated or other predictive methods based on sensors attached to equipment or user.
The invention further incorporates an analytics engine operable to perform multivariate analysis of actual performance results for instructional purposes, whereas conventional systems provide predictive analysis of stroke characteristics. In contrast, the invention compares actual putting performance against the actual putting performance of professional golfers' average putting performance, without regard for technical data comparison.
The invention does not require any visual markers, sensors, other aids or objects on the equipment or user in order to recognize and process video to record the ball or user. Video is taken from the vantage point of the base unit or stand-alone mobile device facing toward user.
The recording is transferred from the base unit over a wireless protocol such as a network connection, Z-Wave®, 6LoWPAN®, Thread®, NFC®, Dash7®, RFID®, WIFI, Bluetooth®, Zigbee®, cellular networks to the mobile device or terminal and is accessed 816 at the mobile device or terminal via a mobile application.
The invention is operable to transmit data over a wired or wireless network to the cloud comprising public and/or private server(s), and all historical performance data and video data is accessible from such server(s) in the mobile application via the mobile device or terminal. In an aspect of the invention, the user can log into the mobile application to review their performance statistics, analysis, and compare such statistics against the average statistics of professional golfers. The invention is further operable to communicate data to the mobile application via the wireless protocol such as, Z-Wave®, 6LoWPAN®, Thread®, NFC®, Dash7®, RFID®, WIFI, Bluetooth®, Zigbee® and cellular networks. The communication can be typified by a Wi-Fi protocol or by plugging the mobile device or terminal using a USB cord into a computer that is connected to the network. In operation, a user can review the video on their mobile application to determine issues in set-up or stroke. By using the personal statistics and the video review, users can identify which corrections are needed to improve putting performance.
The invention provides the following advantages. It is operable as a factual assessment tool that reflects putting skill performance based on actual results of putting a ball in relation to a target; it is operable as a game that can be played by multiple players against each other either in alternating player format or virtually across the internet assuming players have connectivity. The video component enables visual self-learning of stroke and ball characteristics. The longitudinal data analysis enables device-driven learning and curated instructional feedback. The use of analytics by the invention is operable to generate and serve alerts to the user of the primary areas where practice is required. The invention enhances the user experience compared to conventional systems that only allow user to see success on a stroke by stroke basis. The application embodied in the invention can include different levels of instruction as the user reaches skill plateau and the invention provides 24/7 access via the mobile application to video and performance data.
A method and apparatus in accordance with the present invention for determining the state of a putted golf ball relative to a target and remotely displaying the same generally comprises a base unit and a standalone interfacing device and an analysis engine operable to provide actionable feedback to the user.
In an embodiment, at least one of the input data, external variables, machine learning algorithm and output are transmitted and stored in a remote or distributed server. In a further embodiment, the input data, external variables, algorithm and output are collected, transmitted and stored in a remote or distributed server and machine learning and artificial intelligence techniques are applied thereto to provide insights into the user's past, current and predicted performance. Thereby the program is an artificial intelligence program.
The data generated and stored by the firmware and device computer program are maintained to be acted upon by analytics programs to enable factual conclusions to be drawn about the users putting skills, both historically as compared to the user's prior putts (all or a subset thereof) and as compared to putts made by others, such as by professional golfers.
In an embodiment, said means for sensing position and velocity of a putted golf ball in close proximity to user-facing side of said base unit housing comprises one or more optical sensors.
The analytics engine of the invention is further operable with a virtual reality headset. In such embodiment, the only physical device may be a sensor handle that mimics a putter or golf club. An extension to the golf head, the golf ball, the target hole, and the putting green, all or a portion thereof, are virtual. The firmware and device computer program are configured to execute algorithms to detect the velocity of the virtual ball to make a determination if the virtual golf ball would have entered a virtual target hole. If it determines that a golf ball had too much velocity, it can further determine how far past the virtual target hole the virtual golf ball progressed. Further, the user can input virtual parameters into the platform, such as the elevation of the hole, obstacles between the user and the hole and wind speed and direction and take such factors into account when determining if the golf ball went into, or to the left or right of the hole.
The data capture device 905 has one or more cameras with 360-degree overhead vantage point facing down and a field of view up to 50 feet in diameter. During use, data capture device 905 and mobile device 960 internal sensors are capable of auto-detect length, slope (up-down) and break (left-right) and pull them into the app as inputs for balls hit from each location. The data capture device is also capable of monitoring a golf ball from the at rest position and velocity and location of a golf ball as it moves from the at rest position to the target over the terrain of the putting green or the path to the target.
Referring to
The system is also capable of analyzing and combining the second putt information that is captured by the instant invention. The instant invention can determine if the second putt is made or the distance/location, if missed, is similar to a first putt. The information on each putt, is kept in one file, so the user can review the information.
Referring to
The capture device 905 is capable of monitoring a golf ball from the at rest position and velocity and location of a golf ball as it moves from the at rest position to the target over the terrain of the putting green or the path to the target as it approaches the putting green.
The instant invention can be retro fit onto a flag stick for golfers to capture live data during play for whatever shot was made to get ball on green. The instant invention uses a capture device 905 with the camera 910 facing the green terrain 1010. The camera 910 can see 360 degrees and a field of view up to 50 feet in diameter B shown in
The camera 910 uses motion detection or in-application user command to detect balls hit onto the green in the field of view. Once a user is in radio frequency communication range, such as when using a Bluetooth® enabled device which typically has a range of approximately 100 feet, they access the instant invention to review balls shown on green, select their ball, then they can select the club used to make the shot and enter the distance it was hit from, so as to enter the data into the instant invention. The instant invention then stores ball proximity which consists of distance and coordinates to the hole into the archive file. The user can then putt the ball, and the instant invention captures the putting data and stores the data which integrates the data into their account for analysis. Over time this data provides their “proximity to the hole” average for each club during play in addition to the putting statistics and analysis.
The instant invention can be used in a similar method to the on-course method, however, the capture device 905 with the camera 910, is replaced with a capture device 905 with higher-resolution cameras 910 such as a High Definition camera positioned higher and/or not on the flag pole itself but on a higher poll located on the course. This application enhances golf ball data captured for shots to the green by professionals in televised golf tournaments. The instant invention picks up close-up images of the balls as they come into field of view on the green. Data points captured include ball spin, axial rotation, angle of descent, and velocity among others.
The cameras of the instant invention can be selected from Standard Definition (SD) or High Definition (HD). The standard resolutions are 640×360, 640×480, 720×480 and 720×576 pixel size. High Definition video usually has a resolution of 1280×720 (720 p) or 1920×1080 (1080 p) pixel size, is also known as Full HD).
The instant invention can also be configured in a multi-camera analysis mode. When using a multi-camera analysis mode there is a second capture device 905 with the camera 910 positioned at the location of the ball on the course or green, such that the multi-camera configuration captures data from a golf shot including club, ball, stroke and user. Then the instant invention overlays the data with the actual shot result data captured by a first capture device 905 with the camera 910 positioned at the target 49. The information is used to analyze the shot to provide correlational insight and instructional feedback to user. The data captured by the instant invention to overlay with shot results include club lie, ball lift, spin, axial rotation, line with regards to camera and the hole, break percentage, break length, and impact speed, among others.
The instant invention capture device 905 with the camera 910 can also be mechanized such that it can mechanically swivel or rotate around pole directing users to putt from different locations. The instant invention capture device 905 with the camera 910 can be equipped with a laser or other visual mechanism which points to where user should putt from. This is incorporated into an Artificial Intelligence-automated (robotic)-coaching session. The instant invention with artificial intelligence can lead users through a set of sequential putts to practice, based on their performance data; or simply provide the proper camera alignment for general use.
Aggregation and analysis of the pool of data from the user database will drive the system for the user's content and drills. It is the assimilated knowledge from both users, and group the users based on similarities in their performance that will be useful to the user to improve their game. The instant invention use of user profiling is tailored for each user improving the user experience, and guiding users to perform better through curated content and drills. Artificial Intelligence provides the curation of the content and for driving golfer's performance analysis.
Alternatively, the data can be synchronized in real-time for user-to-user collaboration putting in the same location or in different locations. The user-to-user collaboration includes, but is not limited to, coach-to-student collaboration and teaching.
The instant invention capture device 905 with the camera 910 and the base unit 100 using microprocessor having a program which is an artificial intelligence program, and the artificial intelligence program is capable of capturing a large number of users. It also provides a data set from which to study golfer's actual performance, with data points which are currently not captured using devices found in the prior art. The ability of the instant invention to capture a large number of user's data, allows the instant invention to mine data which will automate instructional drills, individual instructional programs and provide curated relevant external/internal instructional resources.
Referring to
a. The golf ball 1215 would be detectable by the base unit 100, 300 or capture device 905 using a wireless protocol such as but not limited to Z-Wave®, 6LoWPAN®, Thread®, NFC®, Dash7®, RFID®, WIFI, Bluetooth®, Zigbee®, cellular networks, or other wireless detection means located in a target hole or contained in housing and used as a simulated target of its own as shown in
b. Golf ball 1215 hit with golf club 1210 towards target 49 (putt or other means) are picked up and tracked to endpoint, then coordinates are uploaded and made available in mobile application so that the user can see the information and the track of the golf ball as it moved over the terrain of the green.
Typical means of using the sensor enabled golf ball 1215:
a. The user initiates start of session via base unit, mobile app, audible or other means.
b. The system software triggers detection mechanism to search for the golf ball 1215 with the sensor 1401.
c. The system determines distance and other variables before golf ball is hit.
d. The system software notifies user that the system is ready and user may strike the golf ball 1215.
e. The system software determines distance and other variables before golf ball 1215 is hit.
f. The system software determines if golf ball 1215 made or missed, and when missed distance and location proximity of the golf ball 1215 at endpoint and then determines the coordinate direction of the golf ball 1215 in relationship to North, South, East, West or Cartesian coordinates or other mapping method with respect to the target.
When using a smart golf ball with sensors 1401 in the golf ball 1215 the instant invention is capable of monitoring a golf ball from the at rest position and velocity and location of a golf ball as it moves from the at rest position to the target over the terrain of the putting green or the path to the target.
The system software determines the various statistics and information related to the performance from this data collected from the golf ball 1215 including archiving and analysis of data as well as made and missed information.
The system software can also be used to provide performance and drills for the user based on performance information.
All embodiments of the instant invention are designed to capture, aggregate, archive, process, analyze, and display the actual performance results for a user striking a golf ball 1215 towards a target 49.
The term “performance” means the measured level of success of the user as with regards to the following:
a. User striking the golf ball 1215 at a target 49 and hitting the target 49=shot made.
b. User striking the golf ball 1215 at a target 49 and missing the target 49=shot missed.
The terms machine learning and artificial intelligence include the aggregation and analysis of each user's data in addition to aggregation and analysis of the collective data pool from all users in the system software database are driving the recommendation system for each user's statistics, feedback, suggested practice drills and related instruction and content. Artificial intelligence is used for curation of the personalized output to each user providing the most practical guide for improvement. The system software assimilates knowledge to group (segment by proficiency level) the users based on similarities in their performance. Further, the data is synchronized by the system software in real-time for user-to-user gaming or other collaboration whether in the same location or different locations altogether using Z-Wave®, 6LoWPAN®, Thread®, NFC®, Dash7®, RFID®, WIFI, Bluetooth®, Zigbee®, cellular networks or other mobile app connectivity. The collaboration includes but is not limited to user-to-user, user-to-coach or otherwise.
Proximity to hole is the distance of the golf shot made to the green. The instant invention provides this metric when using the on-course embodiment or interfacing with another shot management system. The system software accomplishes this by:
a. The user making a golf shot to the green or putting surface.
b. Upon coming within connection range with the instant invention base unit 100, 300 or capture device 905, the user opens mobile application to view the golf balls 1215 that were captured by the camera or golf ball 1215 sensing mechanism 1405 which is part of the base unit 100, 300 or capture device 905, on the flag which is used to sense the sensor 1401 on the golf ball 1215.
c. The user chooses their golf ball 1215 from the golf balls 1215 identified in the playing area by the instant invention system software.
d. The user can either enter the distance or the system can determine the distance of the shot and the user then enters the club used for golf shot and associates it with the current golf ball 1215 position.
e. The instant invention system software then determines and records if golf ball 1215 made it in the hole or missed target 49.
f. When missed, the instant invention system software measures and records the distance between ball 1215 and target 49 hole
g. With this data the instant invention system software is able to provide the “proximity to the hole or target 49” by club used and distance of the golf shot so that the data can be displayed and/or archived.
The design of the instant invention allows the user to utilize a mobile device driven with a resident application on the mobile which is capable of providing an easy to use putting instruction and analytics platform. The mobile device-based embodiment can be used as a “golf shot analytics platform” which can capture both non-putt and putt golf balls and provide a comprehensive tracking of the golf ball from the time of being struck by the golf club to the golf ball either hitting the target or missing the target. The system comprises of:
a. Mobile device with custom computer instructions integrated into an Artificial Intelligence (AI) and machine learning platform that provides a robust platform for the various applicable embodiments including:
b. The mobile device application is designed to collect inputs regarding putting terrain such as slope (up, down), break (left, right) and length.
c. The mobile device application can detect, track, process, archive, analyze, display golf putt and/or shot data made onto a putting green surface.
d. The mobile device application provides similar information to the user when used as a stand-alone device or when it is a component of a multi-component system. The mobile device application collects data and analysis the appropriate instructions and drills to provide as output to user.
To those skilled in the art to which this invention relates, many changes in construction and widely differing embodiments and applications of the invention will suggest themselves without departing from the scope of the invention as defined herein. The disclosures and the descriptions herein are purely illustrative and are not intended to be in any sense limiting.
This application claims priority from U.S. patent application Ser. No. 15/969,617, filed May 2, 2018, U.S. Provisional Patent application 62/635,229, filed on Feb. 26, 2018, and U.S. Provisional Patent application 62/501,015, filed on May 3, 2017 all of which are hereby incorporated by reference herein for all purposes.
| Number | Date | Country | |
|---|---|---|---|
| 20200391094 A1 | Dec 2020 | US |
| Number | Date | Country | |
|---|---|---|---|
| 62635229 | Feb 2018 | US | |
| 62501015 | May 2017 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | 15969617 | May 2018 | US |
| Child | 17004419 | US |
