Patent Grant
8926416
1. Field of the Invention
The present invention relates generally to computer based sports simulators, and more particularly to systems for predicting the future trajectory of a sports object. In particular, the invention relates to a golf simulator.
2. Description of the Related Art
Golf is a sport that is continuing to grow in popularity. One of golf's main attractions to enthusiasts is the continual challenge of improving one's game. To become an adept golfer and to maintain golfing proficiency, a significant amount of practice is required. However, few enthusiasts have the available time required to play full rounds of golf or to practice hitting golf balls at outdoor driving ranges. To solve this problem, many have found indoor golf simulators to be a viable alternative.
Golf simulators have been introduced for providing an indoor facility in which a golfer can practice all aspects of the golfing game. One example of such a device is disclosed in U.S. Pat. No. 5,333,874 to Arnold et al., which is incorporated herein by reference. According to the Arnold invention, a golfer can hit a golf ball against a screen, and an image of a golf course that is projected onto the screen displays the projected path of the golf ball. Prior to hitting the screen, the golf ball travels through two arrays that capture the golf ball's position to calculate the translational velocity of the golf ball. After hitting the screen, the golf ball bounces back through the second array. The position of the golf ball on its rebound is compared to its position when it first passed through the second array. This measurement is then used to calculate the rotational velocity of the golf ball.
One drawback of the Arnold invention is in its limited precision when measuring the rotational velocity. The rotational velocity, or spin, of the golf ball is a major component in determining a precise trajectory of the golf ball as well as its movement after hitting the ground. Allowing a more precise measurement of the spin of the golf ball will help improve a golfer's game by giving them more realistic results when displaying the golf ball's predicted future trajectory.
In one embodiment, the invention comprises a method for simulating a sports activity. The method includes accelerating a sports object from a launch area towards a screen, capturing images of the sports object, and determining, based at least in part on the images, one or more components of rotational velocity of the sports object. The method further includes determining translational velocity of the sports object, computing a future trajectory of the sports object based at least in part on the one or more components of rotational velocity and the translational velocity, and displaying the future trajectory of the sports object.
In another embodiment, an apparatus for simulating a sports activity where the future trajectory of a sports object is predicted is provided. The apparatus includes strobe lights, a strobe controller coupled to the strobe lights, a triggering device coupled to the strobe controller to flash the strobe lights, at least one camera that captures images viewed by the strobe lights, a computer that takes the captured images and computes the spin and trajectory of the sports object, and a display that shows the predicted trajectory of the sports object.
Aspects of the invention will now be described with reference to the Figures. Referring first to
The sports object will typically comprise a ball of some kind, and the display 2 will have an image thereon that is appropriate for the sport being simulated. For example, a baseball could be thrown from the launch area 3 to an image of a catcher on the display 2. A soccer ball could be kicked toward an image of a goal. In the exemplary embodiment described herein, the sports object is a golf ball and the display on the screen is a fairway, green, or other part of a golf course. In these embodiments, after the golf ball hits the display, an image of the ball following a predicted trajectory is generated and displayed to simulate a golf shot in the displayed golf course. Although golf simulation is a particularly advantageous application of the inventions described herein it will be appreciated that other sports simulation could be performed in accordance with the principles described.
As mentioned above, displaying an image of the golf ball trajectory on the display screen in an accurate manner requires an evaluation of the spin imparted to the object by the golf club at impact. The spin determines hook and slice, bite on impact, etc. It is one aspect of some embodiments of the invention that the spin is determined with image processing techniques as set forth further below.
Referring now to
In
Once the spin capturing system 10 has acquired images of the object, the object may pass through a translational velocity capturing system 14 which secures translational velocity components of the object as it travels towards the screen 2.
Processing circuitry 15 is configured to compute components of rotational velocity based at least in part on the images captured by the spin capturing system 14. The processing circuitry 15 also computes the translational velocity of the object and then combines it with the computed rotational velocity to compute a future trajectory of the object. When the object reaches the display 2, the future trajectory has been computed and is then displayed on the screen 2.
The images acquired by the camera 11 are processed to produce a measure of the change in angular orientation of the sports object between two or more images. Knowing the time span between strobes, a rotational velocity can be derived. Thus, using multiple strobes on a systematic inter-strobe time period can capture at least two clean images of the object to analyze.
Generally, the first step of image analysis is to define the pixels in the one or more images that correspond to the sports object. This may be done by an edge detection method such as by binarizing the image and detecting the binary large objects (blobs). The blobs can be found by labeling each color characteristic of the object pixel that is connected to another. The appropriately shaped blobs represent the object whereas the other blobs are background artifacts. Another way to perform edge detection is to use the Canny or Sobel methods. Once you find the edges, the image processing algorithm can then pick out the edges for the round shapes which represent the object. Overlapping images of the sports object can be identified by the area and the shape of the blobs, those images can be discarded and used to know which imprint was made by which strobe. This gives a time period between two clean images of the object. Once that is done, the location of the object edges can be refined in order to more accurately pick out the shape and center of gravity of the object.
Once two clean images of the object are identified, the pixel values in each image can be compared to determine how much the object rotated between the two images and around what axes of rotation. Most objects have stamps on the poles and equator as well as identification marks put on the object by the manufactures. These marks move between images, and comparing their change in position allows a spin vector computation to be made. Even without intentionally created markings, sports objects will include texture on the surface that can be used in the image analysis in the same basic way. Although changes in object orientation between images can often be seen easily by eye, it can be complex to analyze automatically. However, methods to compute components of rotational velocity of a variety of objects have been developed using image analysis. Examples of such methods have been described in the articles Tracking the Translational and Rotational Movements of the Ball using High Speed Camera Movies by Hubert Shum et al., City University of Hong Kong, and Measuring Ball Spin by Image Registration by Toru Tamaki et al., Niigata University. Each of these articles is hereby incorporated by reference in its entirety.
In some such methods, the orientation of the object in each image is defined by Euler angles. The object pixel values of the first image are transformed by different Euler angle changes, and the Euler angle changes that best correlate the pixels of the first image to the pixels of the second image are determined to compute an orientation change between strobes. The Euler angle changes correspond to rotations about three orthogonal axes, which are preferably aligned to the frame of reference of the simulator. Generally, spin around a vertical axis through the center of the ball will define hook and slice. Spin around a horizontal axis through the center of the ball and parallel to the club face will determine top and/or back spin. Spin around a horizontal axis through the center of the ball and approximately normal to the club face will typically be negligible, and the computation can be simplified if spin around this axis is ignored. The spin vector may in these embodiments lie in the vertical plane that is approximately parallel to the club face.
Referring now to
Now referring to
One advantageous placement of the spin capturing system 10 is above the launch area 3 so the camera axis is approximately normal to the ground or floor. In these embodiments, the top or back spin as well as spin defining hook and slice are easily visible. Furthermore, it has been found that advantageous shadows can be produced which enhance the edge detection process during image analysis. However, it will be appreciated that the spin capturing system may also be placed to the side of the launch area so the camera axis is at or near parallel to the ground. Other embodiments may have cameras mounted on poles which are not oriented parallel or normal to the ground, although this makes the image analysis a bit more complex.
As illustrated in
The spatial location of the camera 11 can be found by taking an image of an object from a known location, and based on the size of the object, the location of the camera 11 can be found and stored for use in the calculations of an object's spin.
Once the ball has left the launch area and the images used to compute spin have been captured, the ball will travel through a first plane 19 and second plane 20 that function as the translational capturing system 14. These planes may include one or more IR beam sensors that determine when and where within each plane the plane the golf ball passes through. This configuration is one method that has been employed to calculate translational velocity, but other methods may also be used. One such embodiment is described in the Arnold patent mentioned above. Because the IR camera used to calculate spin takes 2-D images, it is difficult to produce a 3-D velocity vector from the images taken of the ball off the tee. Thus, it is advantageous to have separate spin and velocity vector acquisition systems. It would, however, be possible to have multiple orthogonally mounted cameras produce both spin and velocity vectors from image analysis.
A computer houses the processing circuitry 15 and controls the simulation. From the computer, a player can also select various options of game play which may include practice modes and golf course selection. Other configuration settings such as trigger timings, delays, and microphone sensitivity may also be controlled from the computer.
Referring now to
Once the object has left the launch area 3, the object's position will be sensed when it passes through a first plane 33 and once again when is passes through a second plane 34. The position coordinates will map the translational trajectory of the ball, and the time of travel between the two planes is used to calculate the ball's speed. These elements are then combined and the processing circuitry 15 will compute one or more components of translational velocity 35.
Before the object reaches the screen 2, the processing circuitry 15 will predict a future trajectory of the object 36 using the computed components of rotational and translational velocities. Once the object reaches the screen 2, the future trajectory is already computed and the values are sent to a graphics engine to be displayed 37 on the screen 2 by the projector 18.
Accordingly, the present invention provides a sports simulator which can precisely measure components of spin of a sports object using image analysis. Capturing multiple images of a sports object in motion is used to determine one or more components of rotational velocity. Combining those measurements with the translational velocity will result in more precise predictions of a future trajectory of the sports object.
The foregoing description details certain embodiments of the invention. It will be appreciated, however, that no matter how detailed the foregoing appears in text, the invention may be practiced in many ways. It should be noted that the use of particular terminology when describing certain features or aspects of the invention should not be taken to imply that the terminology is being re-defined herein to be restricted to including any specific characteristics of the features or aspects of the invention with which that terminology is associated, and it will be understood that various omissions, substitutions, and changes in the form and details of the device or process illustrated may be made by those skilled in the technology without departing from the spirit of the invention. The scope of the invention is indicated by the appended claims rather than by the foregoing description. All changes which come within the meaning and range of equivalency of the claims are to be embraced within their scope.
| Number | Name | Date | Kind |
|---|---|---|---|
| 4150825 | Wilson | Apr 1979 | A |
| 4160942 | Lynch et al. | Jul 1979 | A |
| 4542906 | Takase et al. | Sep 1985 | A |
| 4563005 | Hand et al. | Jan 1986 | A |
| 5056791 | Poillon et al. | Oct 1991 | A |
| 5221082 | Curshod | Jun 1993 | A |
| 5333874 | Arnold et al. | Aug 1994 | A |
| 5342054 | Chang et al. | Aug 1994 | A |
| 5443260 | Stewart et al. | Aug 1995 | A |
| 5478077 | Miyahara | Dec 1995 | A |
| 5566951 | Dart et al. | Oct 1996 | A |
| 5649706 | Treat et al. | Jul 1997 | A |
| 5768151 | Lowy et al. | Jun 1998 | A |
| 5846139 | Bair et al. | Dec 1998 | A |
| 5882204 | Iannazo et al. | Mar 1999 | A |
| 5906547 | Tynan | May 1999 | A |
| 5926780 | Fox et al. | Jul 1999 | A |
| 6012987 | Nation | Jan 2000 | A |
| 6042483 | Katayama | Mar 2000 | A |
| 6286364 | Aoyama et al. | Sep 2001 | B1 |
| 6458035 | Katayama | Oct 2002 | B1 |
| 6571600 | Bissonnette et al. | Jun 2003 | B2 |
| 6700486 | Banki | Mar 2004 | B1 |
| 6834129 | Asakura | Dec 2004 | B2 |
| 7038764 | Lee | May 2006 | B2 |
| 7098891 | Pryor | Aug 2006 | B1 |
| 7300356 | Nishino et al. | Nov 2007 | B2 |
| 7324663 | Kiraly | Jan 2008 | B2 |
| 7544137 | Richardson | Jun 2009 | B2 |
| 7946960 | Vitolo et al. | May 2011 | B2 |
| 7959517 | Lastowka | Jun 2011 | B2 |
| 8016688 | Gobush | Sep 2011 | B2 |
| 20040030527 | Rankin | Feb 2004 | A1 |
| 20040125115 | Takeshima et al. | Jul 2004 | A1 |
| 20040242292 | Hansen | Dec 2004 | A1 |
| 20040248662 | Gobush et al. | Dec 2004 | A1 |
| 20050023763 | Richardson | Feb 2005 | A1 |
| 20050159231 | Gobush | Jul 2005 | A1 |
| 20050239548 | Ueshima et al. | Oct 2005 | A1 |
| 20060046861 | Lastowka | Mar 2006 | A1 |
| 20060063574 | Richardson et al. | Mar 2006 | A1 |
| 20060082657 | Meier | Apr 2006 | A1 |
| 20060281572 | Gobush et al. | Dec 2006 | A1 |
| 20070060410 | Gobush | Mar 2007 | A1 |
| 20070196800 | Douthit et al. | Aug 2007 | A1 |
| 20070238539 | Dawe et al. | Oct 2007 | A1 |
| Number | Date | Country |
|---|---|---|
| 08-182786 | Jul 1996 | JP |
| 2004135706 | May 2004 | JP |
| 2007038000 | Feb 2007 | JP |
| 1020030044601 | Jun 2003 | KR |
| WO 2007146620 | Dec 2007 | WO |
| Entry |
|---|
| Dead Solid Golf, Golf Simulator Swing Analysis, http://golfindoor.com/golf—simulators—swing—analysis.html (last visited Aug. 25, 2008). |
| Golf Simulator brochure, http://www.trugolf.com. |
| Holiday Golf Simulators brochure, http://www.holidaygolfusa.com/HolidayGolfBrochure2008.pdf. |
| Indoor Golf Simulators by AboutGold, http://www.aboutgolf.com/3trak.html (last visited Aug. 25, 2008). |
| Par T Golf, Advanced Indoor Golf Simulators, http://www.partgolf.com/DE—Tech.htm (last visited Aug. 25, 2008). |
| Brox et al. Three-dimensional shape knowledge for joint image segmentation and pose estimation. In Pattern Recognition, Vienna, Austria, pp. 109-116, Aug. 2005. |
| Els, DNJ. The Hassenpflug matrix tensor notation. Department of Mechanical Engineering, University of Stellenbosch, South Africa, pp. 1-11, 2001. |
| Shum et al. Tracking the translational and rotational movement of the ball using high-speed camera movies. City University of Hong Kong, 4 pages. |
| Tamaki et al. Measuring ball spin by image registration. Niigata University, Japan, 6 pages. |
| U.S. Final Office Action for U.S. Appl. No. 12/688,659, dated Oct. 9, 2012. |
| U.S. Non-final Office Action for U.S. Appl. No. 12/688,659, dated Nov. 21, 2011. |
| Amendment and Response to U.S. Office Action dated Nov. 21, 2011; Filed on May 16, 2012. |
| Number | Date | Country | |
|---|---|---|---|
| 20090042627 A1 | Feb 2009 | US |
