Patent Application
20250050166
This application claims priority to and the benefit of Korean Patent Application No. 10-2023-0102871 filed in the Korean Intellectual Property Office on Aug. 7, 2023, the entire contents of which are incorporated herein by reference.
The present invention relates to a method and a device for providing an image in a virtual golf simulation system. More particularly, the present invention relates to a method and a device for providing an image of a path of a club hitting a golf ball in a virtual golf simulation system.
With the interest in golf and the development of information and communication technology, virtual golf simulation systems, also known as screen golf, have become widely used. These virtual golf simulation systems are not only used to enjoy golf play in a virtual environment, but are also used to practice golf indoors.
When a player hits a golf ball with a golf club, the path of the head of the golf club relative to the golf ball before and after the golf ball is hit is a very important factor in determining the golf ball flight. For this purpose, there are virtual golf simulation systems that provide the player with the path of the head of the golf club before and after the golf ball is hit.
However, it is necessary to have a separate camera to photograph the head path of the golf club in order to provide the head path of the golf club to the user of the virtual golf simulation system. This causes the cost of the virtual golf simulation system to increase. Also, when a separate camera is not provided, the head path of the golf club is not effectively provided to the user.
The present invention has been made in an effort to provide a method and a device for providing a club path image, which are capable of providing an image of a head path of a golf club without having a separate camera for photographing the head path of the golf club before and after a golf ball is hit.
An exemplary embodiment of the present invention provides a device for providing a club path image includes: a camera control unit for controlling a camera sensor including a first camera acquiring image information of at least a portion of a golf mat; an image acquiring unit for acquiring an image photographed by the first camera; and a club path image generating unit for receiving an image of the first camera from the image acquiring unit and generating a club path image of a path of a golf club hitting a golf ball placed on the golf mat.
In the exemplary embodiment, a region of interest of the first camera may be set to a first region of interest before the golf ball is hit by the golf club, and the first region of interest may be changed to a second region of interest after the golf ball is hit by the golf club.
Further, when the golf club is moved from a first side to a second side, the first region of interest is set such that the golf ball is located on the second side, and the second region of interest may be set such that the golf ball is located on the first side.
Further, the camera sensor may further include a second camera, and a region of interest of the second camera may be set to the second region of interest before and after the golf ball is hit by the golf club.
In the exemplary embodiment, the device may further include a main control unit controlling the camera control unit and the image acquiring unit, in which the main control unit may detect hit of the golf ball based on image information from the second camera.
In the exemplary embodiment, the main control unit may control the camera control unit to change the region of interest of the first camera from the first region of interest to the second region of interest when the hit of the golf ball is detected.
In the exemplary embodiment, the club path image generating unit may stitch image information (mat segmentation image) for a partial region of the golf mat to image information for the first region of interest and image information for the second region of interest from the first camera to extend the image along a direction of movement of the golf club.
Further, the mat segmentation image may be extracted from a region of the image of the golf mat included in a photographing region of the first camera, in which the golf ball is not located.
In the exemplary embodiment, the club path image generating unit may include: an image processing unit that receives image information of the first camera from the image acquiring unit and sequentially processes image information for the first region of interest and image information of the second region of interest; an image synthesizing unit for stitching the mat segmentation image to the image information for the first region of interest and the image information for the second region of interest; and an image storing unit for temporarily storing the image information from the image processing unit and the image synthesizing unit.
The club path image generating unit may stitch the mat segmentation image to a first side of image information for the first region of interest, and the mat segmentation image to a second side, opposite to the first side, of image information for the second region of interest.
Another exemplary embodiment of the present invention provides a method of providing a path image of a golf club hitting a golf ball placed on a golf mat by a club path image generating device, based on image information from a camera sensor including a first camera including the golf mat in a photographing region, the method including: (a) setting a region of interest of the first camera as a first region of interest; (b) acquiring a pre-hit image of the golf ball by the golf club in the first region of interest from the first camera; (c) stitching a mat segmentation image of a segmented portion of the golf mat to the pre-hit image to synthesize an extended pre-hit image; (d) changing a region of interest of the first camera to a second region of interest when the hit of the golf ball is detected; (e) acquiring a post-hit image of the golf ball by the golf club in the second region of interest from the first camera; (f) stitching the mat segmentation image to the post-hit image to synthesize an extended post-hit image; and (g) connecting the extended pre-hit image and the extended post-hit image.
In the exemplary embodiment, when the golf club is moved from a first side to a second side, the first region of interest may be set such that the golf ball is located on the second side, and the second region of interest may be set such that the golf ball is located on the first side.
Further, the camera sensor may further include a second camera, and a region of interest of the second camera may be set to the second region of interest before and after the golf ball is hit by the golf club.
Further, in the operation (c), the mat segmentation image may be stitched to a first side of the image information for the first region of interest, and in the operation (f), the mat segmentation image may be stitched to a second side, opposite to the first side, of image information for the second region of interest.
According to the present invention, it is possible to provide an image of a head path of a golf club without having a separate camera for photographing the head path of the golf club before and after a golf ball is hit. Accordingly, it is possible to effectively provide a user with an image of a path of the golf club without significantly increasing the complexity or cost of a virtual golf simulation system.
The foregoing summary is illustrative only and is not intended to be in any way limiting. In addition to the illustrative aspects, exemplary embodiments, and features described above, further aspects, exemplary embodiments, and features will become apparent by reference to the drawings and the following detailed description.
It should be understood that the appended drawings are not necessarily to scale, presenting a somewhat simplified representation of various features illustrative of the basic principles of the invention. The specific design features of the present invention as disclosed herein, including, for example, specific dimensions, orientations, locations, and shapes will be determined in part by the particular intended application and use environment.
In the figures, reference numbers refer to the same or equivalent parts of the present invention throughout the several figures of the drawing.
Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings. First of all, it should be noted that in giving reference numerals to elements of each drawing, like reference numerals refer to like elements even though like elements are illustrated in different drawings. Further, in the following description of the present invention, a detailed description of known configurations or functions incorporated herein will be omitted when it is judged that the detailed description may make the subject matter of the present disclosure unclear. It should be understood that although the exemplary embodiment of the present invention is described hereafter, the spirit of the present invention is not limited thereto and the present invention may be changed and modified in various ways by those skilled in the art.
A virtual golf simulation system 10 includes a club path image providing device 100, a camera sensor 20 for photographing a region including at least a portion of a golf mat, and a ball path calculating unit 160 for calculating a path of a golf ball by using the image acquired from the camera sensor 20. In the exemplary embodiment, the club path image providing device 100 may provide the ball path calculating unit 160 with necessary image information. The virtual golf simulation system 10 may further include a projection device (not illustrated) for displaying a virtual golf course and a golf ball path. The projection device may include a screen and a projector, and various systems are known in the art for the basic virtual golf simulation system 10, so a description of other detailed configurations of the virtual golf simulation system 10 in the present invention will be omitted.
The club path image providing device 100 forms a part of the virtual golf simulation system 10 and provides a club path image including images of a golf club head hitting the golf ball and the golf ball. The club path image provided by the club path image providing device 100 may be provided to a user via the projection device. In the exemplary embodiment, the club path image may be presented via the projection device with an image displaying the path of the golf ball on a virtual golf course, or the club path image may be presented with information about the path of the golf ball (e.g., the speed of the golf ball at the hit, and the amount of spin of the golf ball).
The camera sensor 20 may include at least two cameras including a first camera 22 and a second camera 24. The first camera 22 and the second camera 24 may configure stereo cameras to acquire three-dimensional movement information of the golf ball.
To acquire the three-dimensional movement information of the golf ball in a stereo manner, the first camera 22 and the second camera 24 may simultaneously acquire images of a region including the golf ball after the hit of the golf ball is detected. The images including the golf ball acquired by the first camera 22 and the second camera 24 may be utilized for stereo matching, and the movement information of the golf ball may be calculated. In the exemplary embodiment, the images acquired by the first camera 22 and the second camera 24 may be transmitted to an image acquisition unit 130 of the club path image providing device 100 and then transmitted to a ball path calculating unit 160. The ball path calculating unit 160 may apply stereo matching to calculate the movement information of the golf ball and present the calculated movement information as information about a speed, movement path, and driving distance of the golf ball on the virtual golf course.
The club path image providing device 100 may include a main control unit 110, a camera control unit 120, the image acquiring unit 130, the club path image generating unit 140, and a club path image output unit 150.
The main control unit 110 controls the overall operation of the club path image providing device 100.
The camera control unit 120 controls the operation of the camera sensor 20. In the exemplary embodiment, the camera control unit 120 may control a region of interest (ROI) of the camera sensor 20. Basically, the first camera 22 and second camera 24 configuring the camera sensor 20 are fixedly mounted to a ceiling of a golf booth so as to include the golf mat on which the golf ball is placed in a photographing region. However, in order to accurately calculate the movement information of the golf ball according to the hit of the golf ball, it is necessary for the first camera 22 and the second camera 24 to perform high-speed photographing. Acquiring and processing all of the sensing information from the entire region of the image sensors provided in the first camera 22 and the second camera 24 may not be suitable for high-speed photographing. Accordingly, the camera control unit 120 may set the ROIs of the first camera 22 and the second camera 24 under the control of the main control unit 110.
The image acquiring unit 130 receives image information acquired from the first camera 22 and the second camera 24. In the exemplary embodiment, the image acquiring unit 130 may receive image information from the first camera 22 and the second camera 24 according to respective ROIs set in the camera control unit 120. In the implementation of the present invention, when the first camera 22 and the second camera 24 and the club path image providing device 100 have sufficient processing speed, the process of setting the ROI of each of the first and second cameras 22 and 24 by the camera control unit 120 may be omitted. In this case, it is also possible that the image information of the entire region acquired by the image sensors of the first camera 22 and the second camera 24 is transmitted to the image acquiring unit 130, and the image acquiring unit 130 extracts only the image information of the region corresponding to the region of interest for further processing. However, the ROI of the first and second cameras 22 and 24 is described herein as being set by the camera control unit 120.
In the implementation of the present invention, any one of the first camera 22 and the second camera 24 may be set to have the same ROI before the golf ball is hit by the golf club as the ROI immediately after the golf ball is hit to detect the hit of the golf ball by the golf club, and the other of the first camera 22 and the second camera 24 may be set to have a different ROI before the golf ball is hit from an ROI after the golf ball is hit. For convenience of description, in the following description, the second camera 24 is set to have the same ROI before the golf ball is hit as the ROI after the golf ball is hit, and the first camera 22 is set to have a different ROI before the golf ball is hit from an ROI immediately after the golf ball is hit. Further, the hit of the golf ball is detected from the image information of the second camera 24 in which the ROIs before the hit and the ROI after the hit are the same. However, in the implementation of the present invention, it is of course possible that the detection of the hit of the golf ball is performed from the image information of the first camera 22.
The club path image generating unit 140 generates a ball hit image (club path image) including a club path before and after the golf ball is hit by using the image of the first camera 22 in which the ROI before the golf ball is hit and the ROI immediately after the golf ball is hit are set differently.
The club path image output unit 150 outputs the club path image generated by the club path image generating unit 140 to the outside of the club path image providing device 100.
The club path image generating unit 140 may include an image processing unit 142, an image storing unit 144, and an image synthesizing unit 146.
The image processing unit 142 processes the images received from the camera sensor 20. In the exemplary embodiment of the present invention, the club path image may be generated by using only the image from the first camera 22. Accordingly, the image acquiring unit 130 may receive image information from the first camera 22 and the second camera 24, and then transmit only the image information from the first camera 22 to the image processing unit 142 of the club path image generating unit 140.
The image processing unit 142 processes the image information received from the first camera 22 so that the processed image information is temporarily stored in the image storing unit 144. In the exemplary embodiment, the image storing unit 144 may be configured as a buffer memory.
The image synthesizing unit 146 synthesizes a golf ball hit image including a path of a golf club hitting a golf ball, by using the image stored in the image storing unit 144. Each frame of the image synthesized by the image synthesizing unit 146 may be stored in the image storing unit 144.
In
Part (a) of
Referring to part (a) of
The image information from the first camera 22 is continuously transmitted to the club path image generating unit 140 via the image acquiring unit 130.
Immediately after the main control unit 110 determines that the ready state is complete, the camera control unit 120 enables the first camera 22 to transmit the image information including the image information for the extra region of interest RB to the image acquiring unit 130. The image processing unit 142 extracts the information about the extra region of interest RB from the image information of the first camera 22 received from the image acquiring unit 130 and stores the extracted information in the image storing unit 144. However, in the implementation of the present invention, the image information of the extra region of interest RB is not stored after the ready state is complete, but it is also possible to store the image of the entire region of the golf mat in advance and utilize the stored image as the image information of the extra region of interest RB by dividing the image of the golf mat in consideration of the width of the first region of interest R1 and the width of the second region of interest R2.
The camera control unit 120 sets the ROI of the first camera 22 to the first region of interest R1. When the information about the extra region of interest RB is generated at the time of each hit of the golf ball, the ROI of the first camera 22 may be set to the first region of interest R1 after the information about the extra region of interest RB is stored after the ready state is complete. When the information about the extra region of interest RB is generated by utilizing the already stored image information of the golf mat, the region of interest of the first camera 22 may be set to the first region of interest R1 after the ready state is complete.
The image processing unit 142 sequentially stores the image information for the first region of interest R1 from the first camera 22 in the image storing unit 144. In the exemplary embodiment, the image processing unit 142 may cause the image information for the first region of interest R1 from the first camera 22 to be stored in the image storing unit 144 over time. The storage of the image information for the first region of interest R1 from the first camera 22 may be performed until the hit of the golf ball 34 is detected.
Meanwhile, the image synthesizing unit 146 generates a synthesized image by stitching the image information of the extra region of interest RB to the right side of the image information for the first region of interest R1 from the first camera 22. For example, when the frame rate of the club path image to be finally generated is 30 frames per second (fps), the image information from the first camera 22 for the first region of interest R1 is stored in the image storing unit 144 every 1/30 second, and the image synthesizing unit 146 may also synthesize the frames of the synthesized image by stitching the extra region of interest RB to the image information for the first region of interest R1 at 1/30 second intervals. In the exemplary embodiment, the image synthesizing unit 146 may perform processing on a boundary region such that when stitching the image information for the first region of interest R1 and the image information for the extra region of interest RB, the stitched image boundaries are smooth.
Referring to part (b) of
Meanwhile, the image processing unit 142 receives the image information from the first camera 22 in the second region of interest R2 and stores the received image information in the image storing unit 144.
The image synthesizing unit 146 generates a synthesized image by stitching already stored image information of the vacant golf mat (e.g., the image information of the extra region of interest RB) to the left side of the image information of the second region of interest R2 stored in the image storing unit 144. The synthesized image after the golf ball is hit may also be generated according to the frame rate of the club path image to be finally generated.
The image storing unit 144 stores images of the golf club and the golf ball before and after the golf ball is hit generated by the image synthesizing unit 146.
The club path image output unit 150 extracts and outputs the club path images corresponding to a predetermined time before and after the golf ball is hit based on the time of the hit of the golf ball. In the exemplary embodiment, the club path image output unit 150 may set an output range of the club path image, such as from −1 second to +1 second, or from −2 seconds to +2 seconds, based on the time of the hit of the golf ball. In addition, in the exemplary embodiment, a time when the golf club enters the first region of interest R1 and a time when the golf club leaves the second region of interest R2 are calculated according to the speed of the golf club (i.e., the club head speed), and the club path image output unit 150 may extract and output the club path image according to the time when the golf club enters the first region of interest R1 and leaves the second region of interest R2. The club path image output unit 150 may also be capable of providing the club path image as a slowed-down screen.
Part (a) to part (c) of
Part (d) to part (f) of
Referring to part (a) of
Referring to part (a) to part (c) of
In a golf club path image, a mat segmentation image of at least a portion of a golf mat (e.g., an image of the previously described extra region of interest RB) is stored for utilization as an image of the golf mat on which a golf ball is not located (S10).
The operation S10 may be performed after the main control unit 110 has determined that the golf ball is placed on the golf mat and is in a complete ready state, or may be performed by using previously acquired images of the golf mat independent of the ready state.
When it is determined that the ready state in which the golf ball is placed on the golf mat and the golf ball is completely ready to be hit, the camera control unit 120 sets a region of interest of the first camera 22 to as a first region of interest R1 (S20).
The club path image generating unit 140 acquires an image before hit of the golf ball for the first region of interest R1 from the first camera 22 (S30).
The club path image generating unit 140 stitches the mat segmentation image to the image before the golf ball is hit for the first region of interest R1 to synthesize an extended pre-hit image in which the image before the golf ball is hit for the first region of interest R1 is extended along a movement direction of the golf club (S40).
The extended pre-hit image may be generated at a predetermined time interval, and the generated extended pre-hit image is stored in buffer memory of the image storing unit 144 (S50).
When the hit of the golf ball is detected from the image acquired by the camera sensor 20 (S60), the region of interest of the first camera 22 is set to a second region of interest R2 (S70).
The club path image generating unit 140 acquires an image after the golf ball is hit for the second region of interest R2 from the first camera 22 (S80).
The club path image generating unit 140 stitches the mat segmentation image to the image after the golf ball is hit for the second region of interest R2 to synthesize an extended post-hit image of the golf ball in which the image after the golf ball is hit for the second region of interest R2 is extended along the direction in which the golf club has been moving (S90).
The extended post-hit image may be generated at a predetermined time interval, and the generated extended post-hit image is buffer stored in the image storing unit 144 (S100).
The extended pre-hit image and the extended post-hit image are connected to generate a club path image as an image of the golf club hitting the golf ball (S110). The generated club path image may be provided to a user.
The club path image providing device 100 and the ball path calculating unit 160 of the virtual golf simulation system 10 described above may be implemented in software, or may be implemented in hardware. If the club path image providing device 100 and the ball path calculating unit 160 are implemented in software, the CPU may execute a computer program to perform various functions. The program may be stored in an embedded memory or on a non-transitory computer-readable recording medium. The club path image providing device 100 and the ball path calculating unit 160 may also be realized by so-called cloud computing by reading a program stored on an external recording medium. If the club path image providing device 100 and the ball path calculating unit 160 of the virtual golf simulation system 10 are implemented in hardware, the program may be realized by various circuits, such as ASICs, FPGAs or DRPs. In the present embodiment, various information or concepts including various information are described, but they are represented by a set of binary bits consisting of 0 or 1, which are represented by a high or low signal value, and communication or computation may be performed by any of the above software or hardware modalities.
Although the exemplary embodiment of the present disclosure has been described for illustrative purposes, those skilled in the art will appreciate that various modifications, additions and substitutions are possible, without departing from the essential characteristic of the invention. Therefore, the exemplary embodiments disclosed in the present disclosure and the accompanying drawings are not intended to limit the technical spirit of the present disclosure, but are intended to illustrate the scope of the technical idea of the present disclosure, and the scope of the present disclosure is not limited by the exemplary embodiment and the accompanying drawings. The scope of the present disclosure shall be construed on the basis of the accompanying claims in such a manner that all of the technical ideas included within the scope equivalent to the claims belong to the scope of the present disclosure.
As described above, the exemplary embodiments have been described and illustrated in the drawings and the specification. The exemplary embodiments were chosen and described in order to explain certain principles of the invention and their practical application, to thereby enable others skilled in the art to make and utilize various exemplary embodiments of the present invention, as well as various alternatives and modifications thereof. As is evident from the foregoing description, certain aspects of the present invention are not limited by the particular details of the examples illustrated herein, and it is therefore contemplated that other modifications and applications, or equivalents thereof, will occur to those skilled in the art. Many changes, modifications, variations and other uses and applications of the present construction will, however, become apparent to those skilled in the art after considering the specification and the accompanying drawings. All such changes, modifications, variations and other uses and applications which do not depart from the spirit and scope of the invention are deemed to be covered by the invention which is limited only by the claims which follow.
| Number | Date | Country | Kind |
|---|---|---|---|
| 10-2023-0102871 | Aug 2023 | KR | national |
