Interactive games with prediction method

Abstract

A method for engaging a player or a pair of players in a motion related game including the steps of attaching plural colored elements onto selected portions of the player(s) garments and processing a video stream of each of the players to separately identify the positions, velocities an accelerations of the several colored elements. The method further comprises generation of a combatant competitor image and moving the image in a manor to overcome the player. In a further approach, two players are recorded and their video images are presented one screens frontal to the other of the players. The same colored elements are used to enable computer calculations of fighting proficiency of the players.

Description

BACKGROUND OF THE INVENTION

1. Field of the Subject Matter

This invention relates generally to games of interactive play between two or more entities including individuals and computer simulated opponents, i.e., the invention may be used by two individuals, an individual and a simulation, and even between two simulations, as for demonstration purposes, and more particularly to a computer controlled interactive movement and contact simulation game in which a player mutually interacts with a computer generated image that responds to the player's movement in real-time.

2. Description of Related Art

The following art defines the present state of this field:

Invention and use of computer generated, interactive apparatus are known to the public, in that such apparatus are currently employed for a wide variety of uses, including interactive games, exercise equipment, and astronaut training.

Ahdoot, U.S. Pat. No. 5,913,727 discloses an interactive contact and simulation game apparatus in which a player and a three dimensional computer generated image interact in simulated physical contact. Alternately two players may interact through the apparatus of the invention. The game apparatus includes a computerized control means generating a simulated image or images of the players, and displaying the images on a large display. A plurality of position sensing and impact generating means are secured to various locations on each of the player's bodies. The position sensing means relay information to the control means indicating the exact position of the player. This is accomplished by the display means generating a moving light signal, invisible to the player, but detected by the position sensing means and relayed to the control means. The control means then responds in real time to the player's position and movements by moving the image in a combat strategy. When simulated contact between the image and the player is determined by the control means, the impact generating means positioned at the point of contact is activated to apply pressure to the player, thus simulating contact. With two players, each players sees his opponent as a simulated image on his display device.

Lewis et al. U.S. Pat. No. 5,177,872 discloses a novel device for determining the position of a person or object. The device is responsive to head or hand movements in order to move a dampened substance contained within a confined tube past one or more sensors. Light passing through the tube is interrupted by the movement of the dampened substance. The intended use of the device, as disclosed, is changing the perspective shown on a video display.

Goo U.S. Pat. No. 4,817,950 teaches a video game controller for surfboarding simulation, and of particular interest is the use of a unique attitude sensing device to determine the exact position of the surfboard. The attitude sensing device employs a plurality of switch closures to determine the tilt angle of the platform and open and close a plurality of electrical contacts enabling a signal input to a computer control unit.

Good et al. U.S. Pat. No. 5,185,561 teaches the principals of tactile feedback through the use of a torque motor. As disclosed, the device consists of a hand held, one dimensional torque feedback device used to manipulate computer generated visual information and associated torque forces.

Kosugi et al. U.S. Pat. No. 5,229,756 disclose a combination of components forming an interactive image control apparatus. The main components of the device are a movement detector for detecting movement, a judging device for determining the state of the operator on the basis of the movement signal provided by the movement detector, and a controller that controls the image in accordance with the movement signal and the judgment of the judgment device. The movement detector, judging device and the controller cooperate so as to control the image in accordance with the movement of the operator. Kosugi requires that a detection means be attached adjacent to the operator's elbow and knee joints so as to measure the bending angle of the extremity and thus more accurately respond to the operator's movements.

The present invention employs a system in which the position of the player is continually monitored. Between the simple types of games of combat as typically found in game arcades, wherein the player's is via a simple control joystick and punch-buttons, and the very sophisticated and complex artificial reality types of game wherein the headgear provides a full sensory input structure, and a highly instrumented and wired glove allows manual contact on a limited basis with the simulation, there is a need for a fully interactive game. The present invention takes the approach to simulate a combat adversary image, while allowing the player to exercise every part of his body in combat with the image. This is the final and most important objective. The present invention fulfills these needs and provides further related advantages as described in the following summary.

Our prior art search with abstracts described above teaches interactive game technology, technique and know-how. However, the prior art fails to teach the instant technique featuring simulated “stand-up” combat between two individuals or between an individual and a computer simulation. The present invention fulfills these needs and provides further related advantages as described in the following summary.

SUMMARY

The present invention teaches certain benefits in construction and use which give rise to the objectives described below.

A best mode embodiment of the present invention provides a method for engaging a player or a pair of players in a motion related game including the steps of attaching plural colored elements onto selected portions of the player(s); processing a video stream from a digital camera to separately identify the positions, velocities an accelerations of the several colored elements in time; providing a data stream of the video recorder to a data processor; calculating the distance between the player and the camera as a function of time; predicting the motions of the players and providing anticipatory motions of a virtual image in compensation thereof.

A primary objective of the present invention is to provide an apparatus and method of use of such apparatus that yields advantages not taught by the prior art.

Another objective of the invention is to provide a game for simulated combat between two individuals.

A further objective of the invention is to provide a game for simulated combat between an individual and a simulated second player of the game.

A further objective of the invention is to provide a game for simulated combat between an individual carrying a sport instrument in hand and a simulated offense and defense players of the game.

A still further objective of the invention is to provide the virtual image to anticipate and predict the movement of the real player and to change the virtual image accordingly.

Other features and advantages of the embodiments of the present invention will become apparent from the following more detailed description, taken in conjunction with the accompanying drawings, which illustrate, by way of example, the principles of at least one of the possible embodiments of the invention.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings illustrate at least one of the best mode embodiments of the present invention. In such drawings:

FIG. 1 is a perspective view showing a method of the instant innovation providing video capture of the motions of a player and of projection of a competitor's image onto a screen;

FIG. 2 is a perspective view thereof showing one embodiment of the invention with a player at left and a simulated player's image at right;

FIG. 3 is a perspective view thereof showing a first and a second players in separate locations with video images of each projected onto a screen at the other player's location; and

FIGS. 4-5 are a logic diagram of the method of the invention.

DETAILED DESCRIPTION

The above described drawing figures illustrate the present invention in at least one of its preferred, best mode embodiments, which is further defined in detail in the following description. Those having ordinary skill in the art may be able to make alterations and modifications in the present invention without departing from its spirit and scope. Therefore, it must be understood that the illustrated embodiments have been set forth only for the purposes of example and that they should not be taken as limiting the invention as defined in the appended claims.

In the present apparatus and method, one or two players take part in a game involving physical movements. Such games may comprise simulated combat, games of chance, competition, cooperative engagement, and similar subjects. However, the present invention is ideal for use in games of hand-to-hand combat such as karate, aikido, kick-boxing and American style boxing where the players have contact but are not physically intertwined as they are in wrestling, Judo and similar sports. In this disclosure a combat game is described, but such is not meant to limit the range of possible uses of the present invention. In one embodiment of the instant combat game, a player 5 engages in simulated combat with an image 5′ projected onto a screen 10 placed in front of the player 5. In this embodiment, the image 5′ is computer generated using the same technology as found in game arcades. In an alternate embodiment, two players 5 stand in front of two separate screens 10 and engage in mutual simulated combat against recorded and projected images 5′ of each other. This avoids physical face-to-face combat where one of the players might receive injury. In this second approach, the images projected onto the screens 10 are not computer generated.

In the first approach, a player 5 is positioned in front of a rear projection screen 10. One or more video cameras 20, referred to here as a camera 20, is positioned behind the screen 10. The camera 20 is able to view the player 5 through the screen 10 and record the player's movements dynamically. If the screen 10 is not transparent enough for this to be done, the camera 20 is mounted on the front of the screen 10, or is mounted on or at the rear of the screen 10 viewing the player 5 through a small hole in the screen 10. The screen 10 may be supported by a screen stand (not shown) or it may be mounted on a wall 25 as shown. The screen 10 may also be mounted in the wall 25 with video equipment located on the side of the wall opposite the player 5 as shown in FIG. 1.

A video projector 30 projects a simulated image 5′ of a competitor combatant from the rear onto the screen 10 and this image 5′ is visible to the player 5 as shown in FIG. 2. In the approach where the camera 20 is located behind the screen 10, in order for the camera 20 to not record the projected image 5′, both the camera 20 and the projector 30 operate at identical rates (frames per second) but are set for recording and projecting respectively for only one-half of each frame, and are interlaced so that recording occurs only when the projector 30 is in an off state, and projecting occurs only when the camera 20 is in an off state. The net result is that the player 5, positioned at the front of the screen 10, sees the projected image while the camera 20 sees the player 5 and not the projected image.

The screen 10 may be a two-way mirror with visibility of objects in front of the screen 10 very clear from the rear of the screen 20, and with visibility through the screen 10 from the front not possible, yet visibility of images projected onto the back of the screen 10 highly visible from in front.

In both of the above described approaches, the player 5 wears colored bands as best seen in FIG. 2. Preferably, the player 5 has a band 51 secured at his forehead, above each elbow 52, on each wrist 53, around the waist 54, above each knee 55 and on each ankle 56. Each of these 10 bands is a different color. Further bands may be placed in additional locations on the player, but the 10 bands shown in FIG. 2 as described, are able to achieve the objectives of the instant innovation as will be shown. In the instant method, the image 5′ of the player 5, as recorded by camera 20 is converted into a digital electronic signal. This signal is split into 10 identical signals and each of these 10 signals is filtered for only the color component related to one of the 10 bands 51-56. Each of the filtered signals contains two pieces of information: the location on the plane of the recording device of its related colored band as determined by which pixels are disposed to the band, and the distance from the recording device to the band as determined by the total number of pixels disposed to the band. This information, from all ten bands is processed by a computer 60 to form a composite image 5′ of the player 5.

EXAMPLE 1

The player 5 stands facing the screen 10 with feet a comfortable distance apart, legs straight, and arms hanging at the player's sides. Each of the ten colored bands 51-56 are visible to the camera 20 and with a simple set of anatomical rules, the computer 60 is able to compose a mathematical model of the player's form that accurately represents the player's physical position and anatomical orientation at that moment. When a band moves, its image on the recording plane moves accordingly so that the computer 60 is able to calculate the motion trajectory of the band. When the number of pixels related to a particular band diminishes or grows, the computer 60 is able to calculate the band's trajectory in 3-space. When a band disappears, the computer 60 calculation takes into account the corresponding portion of the human anatomy, has moved so as to be hidden behind another portion of the anatomy of the player 5. This example is represented in FIG. 2.

The computer 60 produces a digital image 5′ representing a competitor combatant and projects this image 5′ onto the screen 10 initially in a starting position with body erect, feet spread apart and arms at sides. As the player 5 moves to attack the competitor image 5′, the computer 60 calculates the trajectory of motion of the attacking element, i.e., hand, arm, leg, etc., of the player 5 and moves the image 5′ to defensive postures or to counter attack. The computer 60 is able to calculate if the player 5 has moved successfully to overcome defensive postures or counter attacks of the image 5′ so as to award points to the player 5′.

EXAMPLE 2

Two players 5 stand facing their respective screens 10, each with feet a comfortable distance apart, legs straight, and arms hanging at their sides. Each of the ten colored bands 51-56 on each of the players 5 are visible to their respective cameras 20 so that the computer 60 is able to compose mathematical models of each of the players 5 in a mathematical 3-space that accurately represents each of the player's physical position and anatomical orientation at that moment relative to the other of the player 5. The vertical plane represented by the screen 10 of one player 5 represents a vertical bisector of the other player 5. Therefore, when one player 5 moves a fist, elbow, knee or foot toward his screen 10, the computer 60 calculates that motion as projecting outwardly toward the other player 5 from the other player's screen 10. In this manner the computer 60 calculates contacts between players 5 in offensive and defensive moves. As in real face-to-face combat, the players 5 initially and nominally stand slightly more than an arm's length away from their screen, i.e., mathematically from their opponent. Points are awarded to each of the players for successful offensive and defensive moves. The images are preferably projected with three-dimensional realism by use of the well known horizontal and vertical polarization of dual simultaneous projections with slight image separation as is well known, and with the players 5 wearing horizontally and vertically polarized lenses so as to see a combined image providing the illusion of depth. In this manner, each of the players 5 sees the illusion of the opponent players image projecting toward him from the screen 10. This example is represented in FIG. 3.

The present disclosure teaches an improved video frame processing method that enables the combative motions between two distant players 5 to be calculated and compared with respect to each other. This method is described as follows and is as shown in FIGS. 4-6. Once the game is initiated, a stream of frames from the video recorder 30 is processed. When motion is determined by a change in the position of any of the color elements 51-56 being recorded, position, velocity, as the differential of the position, and acceleration, as the second differential of the position of each of the ten color elements of the player 5, as discriminated by the signal filtering process described above, are calculated. Enablement of prediction is determined by evaluating the number of frames comprising a particular motion with a minimum number of frames set point. The calculations continue until the number of frames is at least equal to the set point. Depending on whether the motion is defensive, i.e., lagging the opponents movement, or offensive, i.e., independent of the opponent's movement, in any of the colored elements, the image is modified so as to defend against an offensive move by the player 5 or to initiate a new offensive move from an inventory of such moves. The final logical loops of this program are shown in FIGS. 5 and 6 and comprise the determination of incoming offense commands, calculation of the player's new coordinates, determination if the defense or offence is complete, and calculating the player's offensive positions as compared to the image defense moves and vice-versa, and determining a score for the player 5 in accordance with a stored table of score related motion and counter motion comparisons. For each of the motion and counter motion determinations for both offensive and defensive motions of players, a score is created and projected onto the screen.

The enablements described in detail above are considered novel over the prior art of record and are considered critical to the operation of at least one aspect of one best mode embodiment of the instant invention and to the achievement of the above described objectives. The words used in this specification to describe the instant embodiments are to be understood not only in the sense of their commonly defined meanings, but to include by special definition in this specification: structure, material or acts beyond the scope of the commonly defined meanings. Thus if an element can be understood in the context of this specification as including more than one meaning, then its use must be understood as being generic to all possible meanings supported by the specification and by the word or words describing the element.

The definitions of the words or elements of the embodiments of the herein described invention and its related embodiments not described are, therefore, defined in this specification to include not only the combination of elements which are literally set forth, but all equivalent structure, material or acts for performing substantially the same function in substantially the same way to obtain substantially the same result. In this sense it is therefore contemplated that an equivalent substitution of two or more elements may be made for any one of the elements in the invention and its various embodiments or that a single element may be substituted for two or more elements in a claim.

Changes from the claimed subject matter as viewed by a person with ordinary skill in the art, now known or later devised, are expressly contemplated as being equivalents within the scope of the invention and its various embodiments. Therefore, obvious substitutions now or later known to one with ordinary skill in the art are defined to be within the scope of the defined elements. The invention and its various embodiments are thus to be understood to include what is specifically illustrated and described above, what is conceptually equivalent, what can be obviously substituted, and also what essentially incorporates the essential idea of the invention.

While the invention has been described with reference to at least one preferred embodiment, it is to be clearly understood by those skilled in the art that the invention is not limited thereto. Rather, the scope of the invention is to be interpreted only in conjunction with the appended claims and it is made clear, here, that the inventor(s) believe that the claimed subject matter is the invention.

Claims

1. A method of playing a motion related hand-to-hand combat type game, between a player and an image of a competitor player; the method comprising the steps of: a) identifying portions of the players with individual colored elements; b) recording the players as video images and filtering the images into separate signals according to the colored elements for both of the players; c) determining positions in 3-space of the portions of the players on each video frame of each of the recordings of the players, and calculating changes in position between each of the frames, and further calculating velocity, acceleration and trajectory line of each of the portions; d) determining when one of the portions of one of the players intersects the impact plane; e) determining if a portion of the other of the players intersects the impact plane and also intersects the one of the portions of the one of the players; f) awarding a point to the one of the players; and g) repeating steps (c) through (f) until one of the players has achieved a set number of points.

2. The method of claim 1 wherein, utilizing a digital video camera interfaced to a distributed processor to capture real time 3-d motions of a player comprising the further steps of: a) calibrating the system by initially placing the player(s) at a fixed distance from the camera and having the colored elements, and bodily signatures of the player to be calibrated for real time 3-d motion detection; b) continually receiving the camera's real time electro-optical, auto focus, and zooming control information along with video camera signals measuring the 3 dimensional positions of the player(s) at motions; c) while in motion, the depth (z) is calculated by the ratio of the of the total pixel count of the colored elements worn by the player(s) to the total video pixels of the colored elements measured during initial calibration; d) utilizing a camera that could be commanded to perform auto focus or computer controlled focus; e) adjusting the pixel count information of the colored elements, and player(s) bodily signature based upon the received camera's auto-focus or computer controlled focus; f) trajectory of motion, speed, and acceleration of the players body parts is measured upon the differential changes of recent frame to the previous frame provide filtering of images to provide a sharp image and eliminate background noises; g) differential changes are measured from frame to frame by following the periphery of each colored element and measuring pixel changes; h) utilize a computer controlled camera that is commanded to focus and stay focus on a specific moving colored element; i) utilize a computer controlled camera that its zooming is computer controlled; j) placing the digital camera on a computer controlled gimbal to follow the player's motions the pixel count derived from step c will be further adjusted based upon the 2-d gimbal motions; k) utilizing the digital camera with inferred sensors to monitor the bodily temperature of the player.

3. The method of claim 1 wherein the computer's further actions are synchronized to the start of a player's motions, or verbal commands on a frame by frame basis, further comprising the steps of: a) each incoming frame is compared to the previous frame to detect the magnitude of change compared to the previous frame changes in the incoming frames surpassing a threshold are lead to further processing changes in the incoming frames not surpassing the threshold are counted, discarded, and led to further processing; b) continuous incoming frames not surpassing a threshold for a certain period of time (“c” number of frames) are counted, discarded, and led to an offense motion by the computer generated image; c) the voice activated command or other commands are analyzed and led to different processing stages, depending upon the nature of the commands;

4. The method of claims 1 wherein an event detection and prediction distributed digital image processor, continually monitors the movement of the player to detect motions that are consistent within certain time period (“b” number of frames) an event is defined as offense or a defense motion by the player; and wherein the event detector's algorithm is comprising of the steps: a) consecutive frames that have passed the threshold, each are compared to the previous frame to detect the magnitude of change changes are added to the previous trajectory of the player's motions; b) if received frame number is less than b number of frames, repeat previous step, otherwise go to the next step. c) at the end of “b” number of frames, does player's motions indicate an offense aimed at the image's sensitive parts? if yes, go to players offensive play (step f), otherwise continue; d) at the end of “b” number of frames does player's motions indicate a defense, protecting and dodging the image's offensive moves? if yes, go to step player's defense (step g), otherwise continue; e) at the end of “b” number of frames does player's motions indicate a combination of offense and defense against image's body parts? if yes, calculate the likelihood of hit success comparing that of player's and images' motions if player's offense is stronger, go to step f, if weaker, go to step g; f) predict a player's offense course of motion that is the continuation of motion detections in “b” plan a defensive image's motions in conjunction with the player's prediction calculate player and image's final trajectory and coordinates at the end of the predicted or planned period, and send it to event follower processor step h, then go to step a. g) predict the players defense course of motion that is the continuation of motion detections in “b” plan an offense course of motion for the image's motions in conjunction with the player's prediction calculate player, and image's final trajectory, coordinates at the end of the planned period and send results to event follower processor, step l, then go to step a; h) new player's offense command received from the event detection prediction processor? if no, go to the next step, otherwise continue displaying planned image and repeat this step; i) continually display the planned defense or offense motions of the image get next frame, process frame, calculate players new coordinates, add to the previous coordinate; j) end of the players prediction period, or image's planned defense? if yes, go to next step, if no go to previous step; k) compare player's offense compared to image's defense calculate and show scores, go step h; l) new player's defense command received from the event detection processor? if no, go to the next step, otherwise continue displaying planned image and repeat this step; m) continually display the planned defense or offense motions of the image get next frame, process frame, calculate players new coordinates, add to the previous coordinate; n) end of player's prediction or image's planned offense period? if yes go to next step, otherwise go to previous step; o) calculate image's offense compared to player's defense calculate and show scores, go to step l.

5. The method of claim 1 wherein the degree for player's speed, is decided by adjustment of “b” number of frames during the initialization, and the degree of expertise is decided by classifying predictions and plans.;

6. The method of claim 1 further comprising the step of increasing the number of cameras and display monitors to assist the player's view of the image at different angles while turning and facing from one camera to another, wherein: a) the image processor to provide an image 3d field of play for the player to use as a visual guidelines for his/hers movements in a field of play, while the image is moved around from one side of the field of play to the other; b) the image processor to detect player positions from different cameras and decide witch camera provides the best detection angle and display the image in a relevant field of play to be viewed by the player;

7. The method of claim 1 wherein two local players using two sets of camera(s), two sets of displays, and an image processors examine individual video pictures from each players and display the video or planned image of the opponent player.

8. The method of claim 1 wherein two remote players using two sets of camera(s), two sets of displays, and two sets of image processor further comprising the steps of: p) each processor to examine the videos from each relevant local player. q) each processor to receive the opponent's image motion information, (or actual opponent's, video and other relevant information) via remote transmission facilities on frame by frame basis; r) each processor to display an image of the opponent and control the image based upon the information received from the opponents motions; s) each processor to provide scores on each displays; t) one processor to determine the winner score, to be displayed on both monitors;

9. A method of playing baseball, tennis, golf, or other related games between a player having a sport instruments in hand and the images of offense and defense players wherein a ball, the images of a players, and an image of the field are generated by a processor; the method comprising the steps of: a. identify the play instrument and portions of the player body with individual colored elements; b. each computer generated thrown image ball, will be planned and played with a known trajectory, speed, acceleration, and a prediction, simulating a pro player; c. generate an image of the field of play in 3-d whereby offense and defense actions takes place, by image offense, and image defense players; d. processing of the incoming frames at the vicinity of the time of impact of the player's tennis with the image ball; e. recording the movements of the player and the instrument as a video image and electronically or optically filtering the image into separate signals according to the colored elements; f. determining positions in 3-space of the portions of the player and the instrument on each recording video frame; g. following the trajectory of the player's body parts and the sport instrument, utilizing the method of claim 4, to calculate changes in trajectory, velocity, and acceleration of the portions of the player's body and trajectory of the instrument; h. predicting the trajectory, velocity, and acceleration of the image ball being hit by the player's instrument; i. moving the image players, as the result of the predicted trajectory of the ball and the image's physical location in the field of play, at the moment of impact of instrument with the image ball; j. calculate the likelihood of success for the image ball to stay within the image 3-d field of play; k. displaying the predicted trajectory of the ball hit by the player instrument; l. display the images of players, playing offense, defense and reacting to the image ball, based upon the positions of the image player, and the prediction; m. compare the player's prediction and follow-on, to the initial planned trajectory (step b), and display scores.