Mobile holographic simulator of bowling pins and virtual objects

Abstract

A system with automatic positioning controls for the holographic display of three-dimensional objects interacting in real time with objects of the real world. The person observing the screen (18) has the sensation that the objects are real, which is achieved thanks to a real time perspective correction system (1). The system is used in particular to simulate bowling pins (9). It is positioned on the lane required by means of a dolly (6). The system determines a relation between the information received from the sensors: video cameras (1) (3), dimensions (26), weight (14) (27) and the mathematical simulation models. When the ball is bowled, its physical features are measured and when it hits the pin deck, some holographic pins are displayed on the screen (19). The screen holographically arranges the projected objects over what the bowler (18) sees behind the screen (19). The simulated movements of the pins and the physical interactions with the real objects are determined by exploiting the theory of mechanics in three-dimensional space.

Description

BACKGROUND ART

Definitions and Basic Concepts

Tenpin bowling is played on a wooden lane or one of synthetic material, according to the specifications of the American Bowling Club ABC/WIBC. 10 pins are positioned at one end of the lane while the player throws a ball from the other end in the attempt to knockdown as many pins as possible. Please refer to the ABC/WIBC specifications for the complete set of game rules, as this is merely a generic description. There are many types of games, which differ in the number of pins, shape, positions, score rules, shape and dimensions of the lanes, balls and pins; here are the most popular: 10 pin, 9 pin, candle pin, 5 pin, figure, duck pin, red pin. The wooden lane is rectangular, 19 meters long and 3 meters wide and the pins are positioned on top of it. The game is played on the lane, which consists in throwing bowling balls towards the pins in the attempt to knockdown as many pins possible. The pin deck is the minimum square area on which all the pins are positioned (9). In the case of tenpin bowling, the pins are positioned in the shape of a triangle. Pin 1 is the pin at the head of the triangle. The player or bowler is the person who throws the ball in the attempt to knock down the pins. The game of bowling is played between one or more bowlers and has the purpose of scoring the highest number of pins knocked down in a certain number of throws. For example, in the case of tenpin bowling, each game is split-up into 10 frames. One frame is a combination of two throws. The bowler throws the ball from the bowling deck: from the opposite end to where the pins are positioned. The bowler must not step over the foul line, which is 18 meters away from the pins. A bowler is said to have committed a foul if he steps over the foul line when throwing the ball. Hdcp is a bonus starting score assigned to a less expert player in order to make the game fair. The ball return (17) (23) is a mechanical device that returns the ball back to the same bowler so that he can make his next throw. The platform where the ball falls (21) is the lower part of the lane; its purpose is that of blocking and holding the ball thrown; it is set at a slant so that the ball, through the mere force of gravity, rolls towards the inlet of the ball return pit, through which the ball is returned to the beginning of the lane. It is situated at the opposite end of the lane compared to the deck where the ball is thrown. The mechanical pinsetter is an automatic system that puts the real pins knocked down back in place after each throw. Mechanical pinsetters channel, move, turn and therefore position the real pins in established positions exploiting mechanical systems. The lane area in which the pins are positioned by the mechanical pinsetters is marked specifically to point out the exact position of the pins. An animation is a sequence of static pictures that give the human eye the impression of moving objects. These pictures are sequentially projected against a screen at a speed of more than 25 pictures per second and differ from one another by slight movements in the objects contained within the pictures. The perspective view is the point in space from which we get the impression of depth of objects contained within a picture, when looking at a picture produced with the laws of perspective. The physical simulator is a computer that exploits the axioms, theorems and formulae of mechanics. It displays a perspective animation in real time on a screen. This animation has the fundamental feature of reproducing the behavior of objects in the real world according to the laws of nature. This simulator receives information from the real world through sensors that measure the speed, position and mass of real objects. With this information it then simulates the behavior of objects that in reality do not exist but that interact with the real objects. The combination of the real objects and the unreal objects is displayed on the screen with the dynamic behaviour and sounds that they would have if they were all real. The holographic pinsetter is a physical simulator that gives human beings the illusion of the presence of pins. It simulates the behaviour and the interaction of the virtual objects with the real objects, such as the ball and the lane. This means that the holographic pinsetter does not need all the mechanical parts, which make mechanical pinsetters the systems most susceptible to wear, slower, expensive, unstable, cumbersome, heavy and generally less efficient. The holographic screen is made of transparent material, which when hit by the light of a projector, diffuses the light semi-spherically, thus covering the light that crosses it. The effect perceived by the onlooker is that the pictures projected are seen on top of the real pictures produced by the light reflected by the bodies of the real objects. A reflecting screen reflects the light sent from a luminous source, such as a projector, but does not let the light through. The arrows (69) are drawn in the centre of the lane at a few meters from the foul line in certain positions and act as reference marks for the bowler. There are some black spots (47) on the lane that point out the exact position in which the pins are to be positioned. The visual angles of a video camera are the angles where all the objects within the two half-lines of the angle can be seen in the picture generated by the camera.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1: Side-on view of a lane with mobile holographic pinsetter, where: 1. Video camera, 2. Speaker, 3. Video camera, 4. RF reader, 5. Projector, 6. Dolly, 7. Track attachment brackets, 8. Pinsetter control unit, 9. Pin deck, 10. Ball before the screen, 11. Ball after the screen, 12. Ball stopping cushion, 13. Ball outlet hole, 14. Weight sensor, 15. Track control and data acquisition unit, 16. Console, 17. ball return pit, 18. Bowler or player, 19. Screen, 20. Foul line, 21. Slanted deck where the ball falls, 22. False ceiling that conceals the holographic pinsetter, 23. Outlet of the ball return, 24. Connection wire between unit 8 and projector 5, 25. Support for screen, cameras, projector, RF-reader, speaker and control unit, 26. Ball circumference measuring unit, 27. Rotary movement measuring sensor.

FIG. 2: Overhead view of a pair of lanes with mobile holographic pinsetter, where: 31. Photocell of the left-hand lane, 32. Photocell of the right-hand lane, 33. Track control and data acquisition unit, 34. Dolly driving motor, 35. Cable, 37. Ball stopping cushion, 38. Connection cable, 39. Weight sensors, 40. Mobile dolly, 41. Reflex reflector for the right-hand lane photocell, 42. Reflex reflector for the left-hand lane photocell, 43. Holographic pinsetter control unit, 44. Dolly transport tracks, 45. Cable return route, 46. 4 coloured spots, 47. Pin position spots, 48. 4 coloured spots, 49. Edging between gutter and lane, 50. Gutters, 69. Arrows, 70. Foul line

FIG. 3: Cross section view of a ball and detail of the dolly of the transport system, where: 51. Re-chargeable battery, 52. Control unit, 53. Gyroscopic sensor, 54. Connection cables, 55. Lights, 56. Lights, 57. Radio antenna, 58. Radio antenna, 59. Battery charger attachment at the bottom of the finger holes. 60. Photocell, 61. Reflex reflector, 62. Bracket used to secure the tracks to the ceiling, 63. Transport tracks, 64. Motor driven cable, 65. Rope attachment to dolly 16, 66. Dolly, 67. Reflex reflector, 68. Support bracket for the holographic pinsetter

FIG. 4: Example animation: 71. The ball rolls under the screen 77, 72. The ball strikes the virtual pins, 73. The ball strikes the virtual pins, 74. The ball strikes the virtual pins, 75. The ball strikes the virtual pins, 76. The ball falls to the slanted deck, 77. Screen

FIG. 5: Optical reaction of a holographic screen and a screen of normal translucent material, where: 81. Light source, 82. Holographic screen, 83. Projector, 84. Light beam, 85. Light beam after it has passed through the transparent screen, 86. Light emitted by the projector, 87. Light emitted by the projector after it has passed through the screen, 88. Light emitted by the projector, 89. External light, 90. Light emitted by the projector after it has been refracted by a screen of normal translucent material, 91. Light of the external source after it has been refracted by a screen of normal translucent material.

FIG. 6: Screen with mobile bottom part: 101. Screen, 102. Projector, 103. Ball, 104. Screen on the lane, 105. End of lane, 106. View point of bowler, 107. Outlet hole, 108. Screen, 111. Flexible flap, 112. Rigid flap. 113. Lifting piston, 114. Fixed screen, 115. Flaps in stand-by, 116. Ball, 117. Lane, 118. Screen, 119. Mobile part of the screen, 120. Lane, 121. Curved lane

DISCLOSURE OF INVENTION

When a mechanical pinsetter is faulty, the operator commands the control unit (15) (33) using a keyboard and instructs it to change the faulty lane. The control unit (15) (33) starts the motor (34) that moves the holographic pinsetter, commencing by moving the dolly (40) (6) (66) to which elements (1) (2) (3) (4) (5) (7) (8) (19) (24) (25) of the holographic pinsetter are connected. The control unit (15) (33) of the dolly starts to verify the position of the dolly (40) (6) (66) by reading the information sent to it by the position sensor (60), fitted on the tracks (63) (44) (77). The control unit (15) (33) counts the inputs sent from the sensor (60) and, considering that each lane has one, it is capable of deducing on which lane it is situated. Once it detects that the dolly has reached the position required, it stops sending the input that operates the motor (34) of the dolly (66) (40) (6).

When reading the first and the last lane, the sensor receives a double input, considering that the reflex reflector (61) consists of two reflex reflectors separated by a small non-reflecting gap. In this way, the control unit (15) (33) realises whether it is at the end or the beginning of the tracks. At this stage, the control unit (15)(33) sends an electrical signal to the control unit of the holographic pinsetter (8)(43) to instruct it to start the simulation. The above-mentioned transport system may not be installed and, if this is the case, each lane has a holographic pinsetter and the functions performed by the control unit (15) (33) and described in this report, are carried out by control units (8) and (43). This solution offers a few advantages: it is quicker, since there is no need to transport it to the lane required, in which case the costs are naturally higher.

The projector (5) starts to display the animated sequence of the pins and the bowler (18) gets the impression that the pins are actually on the pin deck (9).

The control unit (8) (43) continuously receives the pictures sent from the camera (1) and exploits them to change the perspective view of the animated sequence sent to the projector (5). In this way the bowler (18) standing on the lane, always has a perfect view of the pins, considering that it is that nearest to reality: this is achieved thanks to the fact that the perspective view of the projected picture is modified based on the position of the bowler's eyes. The control unit (8) (43) has a three-dimensional mathematical model of the position and direction of the lane, of all the sensors, of the screen and of all the objects involved in the simulation. It stores the dimensions and positions relative to all the parts making up a lane and their physical properties according to the ABC/WIBC specifications. The known properties of the balls for example, are the weight, the moment of inertia, the dimensions, the mass, the elasticity, the friction, the three-dimensional model, the positions and the directions in space. The pictures received from the video cameras (1) (3) are related mathematically to projections on a flat square surface, which corresponds to the three-dimensional model of the video cameras; through the identification in the projected pictures of at least 4 coplanar points known in the three-dimensional model, we can determine an unmistakeable association between the pictures and the three-dimensional model used in the simulation and therefore we can deduce the position, the direction and the inclination of the video cameras, of which we also know the optical characteristics, such as their visual angles. We can therefore determine an area of the picture, which corresponds to the deck where the ball is bowled, of which the significant chromatic variations are verified in order to identify the body of the bowler and his position in space. This is done by associating the chromatic variations with mathematical methods of probabilistic identification of the silhouette of the bowler. Simplifying principles are also assumed and, considering that we know the direction of the camera, we can deduce that the chromatic variations in the top part of the pictures relate to the upper body of the bowler and vice versa for the bottom part of the chromatic variations. The 4 points are determined by finding the foul line (70), the arrows on the lane (69) and the edgings (49) of the lane or by arranging coloured spots (46) (48) on the edges of the lane at fixed and known distances/positions. The control unit (8)(43) also controls the portion of picture corresponding to the adjacent area around the foul line and can establish if the bowler steps over the foul line (70) while throwing the ball: in other words, if any significant chromatic variations are detected, the event is saved and taken into consideration when calculating the score displayed on the screen (19) and saved in the unit (15) and (8). The chromatic variations must have a sufficiently long duration and be measured experimentally to be associated with the bowler's foot, otherwise they are associated with the ball passing the line as it is thrown.

The bowler starts to throw the ball towards the pins, which are projected on the holographic screen (19). Remember that the bowler only has the impression of seeing real pins on the pin deck (9).

The animated sequence produced takes into consideration some objects, such as the ball (11), which are set over the pins; where the animation takes place, space is left for the real objects, by not projecting the virtual objects. When the ball passes under the screen (19), the RF reader (4) receives information sent to it in radio frequency by the ball (11) and this information is then sent from the RF reader (4) to the control unit of the pinsetter (8)(43).

From here on the ball enters the visual range of the camera (3) and the control unit (8) exploits the pictures sent from the camera (3) to determine when the ball has rolled passed, calculating its speed, its trajectory, its dimensions and its angular speed. These measurements can only be achieved on a picture if there is a three-dimensional model and its relation with the pictures of the camera.

To obtain this relation, at least 4 known and coplanar points in the three-dimensional model must be identified on the two-dimensional pictures. To do this, the control unit (8) analyses the pictures and determines the gutters (50) (49) and the position of the 10 spots (47) that point out the position of the pins on the pin deck or 4 additional coloured spots (48) arranged on the edges of the lane at fixed and known distances. If the control unit (8) fails to identify the 4 optional coloured spots (48), it determines the 4 coplanar points with the 10 spots (47) and the edges of the lane (49) present on each lane. The angular speed for balls without gyroscopic sensors is determined by analysing the superficial movements of the ball, which can be enhanced for plain coloured balls by using at least 6 spots of different colours arranged on the ball surface and positioned so that at least one is always visible. The Spots are stuck to the surface of the ball and are crossed by 3 non-coinciding straight lines, arranged perpendicular to each other. The 3 straight lines meet in the geometrical centre of the ball. These movements are then reproduced in the three-dimensional model as projections, on the optical surface of the camera, of spots on the spherical surface of the ball positioned on the lane deck. In this way we can determine the angular speed of the ball with mathematical precision, using formulae of geometry, perspective and of classical physics. The control unit (8) receives all this information and processes it with the mathematical models of mechanics. It starts sending the signal that will show the simulation of the ball striking the pins to the projector (5). As the ball reaches the pin deck (9), the control unit (8) displays the pins colliding against the ball (72) (73) (74) (75) and starts to move them as if they were moving in reality.

The union in the animated sequence of the real ball and the virtual pins, and all this in real time, produces a visual effect that gives the bowler the impression of playing with physically real pins, as seen in FIG. 4. The bowler will have the sensation of playing with real pins, especially thanks to the use of holographic screens.

In FIG. 4 we have illustrated a sequence of pictures that represent an example of what the bowler sees after throwing a ball. In FIG. 4, each pictures has a frame with black border (77) that represents the transparent holographic screen of the holographic pinsetter. Everything that you can see in each of the pictures of FIG. 4 are objects that actually exist, with the exception of the pins, which as you can see, are always within the black frame, in other words within the transparent screen of the holographic pinsetter. The pins are in actual fact projected by the projector on the screen that sets them on top of the objects behind the transparent screen. The real game is simulated by synchronising the real objects, which move behind the screen (the ball), with the objects projected on the screen by the projector (the pins). In the sequence of pictures from (71) to (76) you can see that the real ball strikes and moves the virtual pins just as if it would do if the pins were actually on the lane.

You can also use non-transparent reflecting screens if you want to hide everything situated on the other side; this means that the animated sequence will also show objects covered by the screen, such as the ball and lane. In this case, all the colours of the ball and of the lane are captured by the pictures taken by the video camera (3) and are reproduced in the virtual sequence on the screen (19).

The real ball will then hit the end of the lane (12) (37) and, thanks to the slanted platform at the end (21), it will fall through the outlet hole (13). Its diameter will then be measured by the dimensional measuring unit (26) (39) and finally it will be weighed again by the weight sensor (14), of which there is one for the pair of lanes. This sensor will send the information to the control unit (15) (33) of the tracks (44) (63), which will then send this information to the control unit (8) (43) of the holographic pinsetter (FIG. 1) These sensors (14) (26) weigh and measure the dimensions of bowling balls that do not have a radio frequency transmission system of the weight/dimensions (52) (58) (57), so that anybody having a standard ball can still play using the holographic pinsetter (FIG. 1) without loosing any realistic effects.

To be pointed out is the fact that the ball stopping cushion (12) is equipped with a movement sensor (27) that measures the quantity of motion absorbed each time it is hit by the ball. It then sends this information to the control unit (8), which is aware of the known, determined and constant physical parameters of the cushion (12) and is capable of calculating the weight of the ball that hit it. Considering however that the weight and dimensions of the ball are measured after the ball reaches the end of the lane (12) this information is registered by the control unit (8) and utilised the next time the bowler throws his ball again. We therefore assume that all bowlers tend to use balls of the same weight and dimensions. This means that the system must have the list of bowlers and keep track of their score to be able to foresee whose turn it is to play and to be able to save the dynamic information of the balls used by each bowler; this information is entered either on the console (16) or the keyboard of the control unit (15) at the beginning of each game. The dimensions of the ball are also determined through a perspective calculation, using the pictures of the camera (3) and the three-dimensional model described previously.

Later, the ball will be returned to the bowler by the ball return (17) and the control unit (8) will show the pins still standing. The bowler can make his next throw, which will cause the repetition of the events in the sequence just described.

Considering that the main feature of the holographic pinsetter (FIG. 1) is that of displaying non-existent objects and making them seem real and interacting with real objects, many special effects can be created, such as that of showing a trail of fire along the trajectory of the ball just thrown, objects that score extra points if hit and so on.

The part of holographic pinsetter made up of elements (8) (6) (7) (23) (1) (2) (3) (4) (5) is covered and concealed behind the false ceiling (22), which has a triangular shape and hides the equipment away from the bowler's sight (video cameras (1) (3), control unit (8) and transport tracks (7) etc. Considering that the screen (19) and the supports that connect it to the dolly (6) are transparent, the bowler doesn't really see them.

The holographic screen (108) must remain a few meters away from the end of the lane and must remain suspended from the ground to leave enough room so that no interference is created with the ball (103) as it rolls along the lane. This distance from the pin set-up causes a perspective error that we can correct using the camera (1) that observes the bowler. To avoid this correction we can use a screen, as seen in (118) (119), where the picture is projected on the screen that touches the lane (120). The bottom part of this screen has some mobile elements (119) (115) so that when the ball (116) touches them they raise (111) (112) to let the ball through and return to their original position (115) as soon as the ball has reached the other side. In this way we can project the pins in their original position, thus minimising the perspective error.

This mobile part consists of reflecting flaps (119) that are; secured to the rigid screen (114). Their special feature is that they are mobile (112) or flexible (115) and have the same properties as a screen. To prevent the flaps from being hit by the ball and becoming dirty, we can lift them before the ball actually hits them (112) by installing a piston (113) that raises the flap. The piston is controlled by the control unit (8), which determines via camera (3), where the ball will hit the flaps (119); each flap has a piston (113) that is independently controlled by the control unit (8). Another variant is that of projecting also the picture on the lane using an additional screen (104). In this way the perspective error is reduced and the flaps eliminated (119). A curved screen (121) is added at the end of the lane to improve the perspective view. The transparent screen (101) is placed above the pin deck instead of a few meters away from it, as in position (108).

In this case there are two reflecting screens, one at the end of the lane (104), on the pin deck, and one in position (101). The picture projected by the projector (102) must be such to give the observer (106) the impression of seeing real pins on the pin deck. The two pictures projected on the screens (104) (101) are composed by the bowler's eyes (f. 7.6) because he has the impression that the pins are real.

Functional Description of the Project Elements

• • Screen (19): The function of the screen is that of displaying the animated sequence received from the control unit and of showing it to the bowler (18), giving the latter the impression that the pins (9) are actually on the pin deck. This effect is obtained by displaying a three-dimensional perspective animated sequence that has the bowler's eye as the perspective point. The screen is raised off the lane to let the ball past and is set at a minimum distance of approximately 4 meters away from the first pin so that the ball is hidden from the screen, thus enabling the screen to set the light emitted by the projector over that reflected from the ball and from the lane. This feature is extremely important to give the simulated game sequence the best possible realistic effect. FIG. 5 illustrates a holographic screen on the left with the characteristics requested and a normal transparent screen on the right. These screens are made of polymers, the composition of which is known to producers such as Hitachi or Holopro for example.

The light (84) emitted by an external source (81) reaches the screen and passes through it without being reflected (85); in this way the screen does not reflect the light of objects that are not involved in the simulation. The light (86) sent from the projector (83) is diffused towards the observer, as can be seen in (87), so that the system can display objects that the observer perceives to be behind the screen. This realistic sensation is improved through a perspective correction of the objects projected, which is done in real time, considering that we identify the variations in position of the perspective point, which in our case is the bowler's eye.

FIG. 5 shows that the light (88) (89) sent from the projector and the external sources, is refracted in all directions (90) (91) creating the undesired effects of normal transparent material. To be pointed out is that holographic screens prove useful if you wish to install the holographic pinsetter on a standard bowling lane, leaving the end part of the lane visible, while it proves useful to use opaque ones if you wish to hide everything beyond the screen.

• • Control unit (8)(43): This unit receives the pictures from the video cameras (1)(3) and determines the position in space of the video camera. It determines when the ball rolls past, detecting its speed, trajectory, dimensions and angular speed. It receives information from the RF-Tag reader (4) to read the dimensions and weight of the ball. It receives the picture from the camera 1(1) and determines the position of the bowler's head and checks if the bowler steps over the foul line (20). It saves the measurements sent to it by the weight and dimension sensors (14). It processes the information and produces a physical simulation of the game dynamics. It sends a signal to the projector (5) that displays the simulation in real time as the ball rolls past. It also sends some signals to the speakers (2), which the latter convert into sounds feasibly produced by the pins hitting each other or the ball or the lane. It also controls the actions performed on the console (16) and displays the requests on the screen (19), thus interacting with the user.
  • Projector (5): It receives the signal (24) from the control unit and projects it on the screen (19) giving the visual sensation of the pins and the ball.
  • Camera (1): It aims at the bowling deck and sends the pictures to the control unit, which determines the three-dimensional position of the eyes of the bowler. This enables an adjustment of the perspective view of the animated sequence so that the bowler always has a real view of the pin deck (9). This camera determines whether the player steps over the foul line (20). It does so by controlling significant changes in the chromatic range in the zone around the foul line.
  • Camera (3): It aims at the pin deck and the pictures sent are used by the control unit (8) to measure when the ball rolls past, its trajectory, dimensions, colour and speed of the ball (11) thrown by the bowler. This information is transmitted to the control unit in the form of pictures, which the latter then processes and converts into a format that can be used within the processes of the processing unit. The result obtained is a signal sent to the projector and utilised by the projector to display the simulated game sequence on the screen. To do this, the control unit knows the position in the three dimensions of the camera (3). In the picture sent to it by the camera (1), the control unit can determine the height, position and angle of the video camera (3) in space. This information is required to calculate the trajectory and speed of the ball throughout the game.
  • Ball stopping cushion (12): This is required to stop the ball and let it drop into the slanted surface (21), where it falls through the hole (13) and is channelled into the tracks of the ball return pit (17) to return it to the beginning of the lane (23). This cushion can also be equipped with a movement sensor (27) (46) that measures the motion absorbed each time it is hit by the ball; the information is sent to the control unit (15)(33) via a cable. This information is required to deduce the mass of the ball from its speed, from its dimensions, from the mass of the cushion and its movement. The dimensions of the ball and the speed are measured by the pictures of the Camera (3) while the movement sensor (27) (46) is fitted on one end of the cushion and connected to the control unit (33) (15). This sensor is an alternative solution to the weight sensor (14) (39) and is used to integrate or replace the RF-reader (4). Its main purpose is that of measuring balls that are not equipped with RF-transmitter and gyroscopic sensor.
  • Weight sensor (14)(39): It is positioned under the slanted surface (21) or near the inlet hole (13) of the ball return; the sensor is used to weigh the ball. This sensor represents an alternative to the RF sensor (58), since it makes the same measurement.
  • Ball dimension measuring unit (26): The sensor is fitted at the beginning of the ball return pit and is used to measure the diameter of the ball. This sensor consists of an elastic cylinder, which when crossed by the ball, stretches a position sensor fitted on the outer surface of the elastic cylinder. This sensor sends an electrical input in proportion with the amount it stretches and that will correspond to the circumference of the ball. To be pointed out is that the dimensions of the diameter of the ball are also measured by the control unit of the holographic pinsetter via the picture of camera 1.
  • RFreader (4): This device receives the information transmitted by the ball (11) in radio frequency and sends it to the control unit (8). This is repeated each time the ball passes under the screen (19).
  • Speakers (2): These convert the electric signals created by the control unit into sounds that simulate the sounds created by the pins when they hit each other and when they collide with the ball and the lane. This is synchronised with the physical simulation of the game and, in other words, with what is seen on the screen.
  • Console(16) : This is a keyboard that is used to send the codes of the keys pressed by the user to the control unit (8), which, based on the keys pressed, enables the user to enter, modify and cancel information. Some examples: You can enter names of the bowlers, you can modify the bowler's score or you can modify the pins knocked down during the game, you can choose the type of game and the type of pins, the level of difficulty and so forth.
  • Ball (f.3) (10) (11): The ball contains a control unit (52), which transmits an unmistakable code in radio frequency (57) (58) that depends on the weight and dimensions of the ball. The control unit (52) sends a code to the rf-reader (4) as the ball (11) passes under the screen (19), then the control unit (8) converts the code into the real weight and dimensions of the ball. In this way, each time the ball rolls past, the control unit is capable of knowing with which weight and diameter the physical simulation of the game is to be constructed. The ball contains a gyroscopic sensor (53), which transmits the angular speed of the ball to the control unit (52) each time the ball passes under the screen; the control unit (52) then transmits this speed to the control unit (8) via a radio frequency transmitter (57) (58). The control unit also triggers a lamp that is built-in the ball (55) (56) only when the ball is moving or better still only when the gyroscopic sensor (53) transmits a movement signal to the control unit (52). The lamp is battery powered (51), which is again fitted inside the ball. This lamp is positioned so that it illuminates the outer surface of the ball from the inside, thus illuminating the actual ball as it rolls along the lane. The battery is re-charged using two contacts (59) arranged in the bottom of the finger holes, which all bowling balls have.
  • Movement tracks (63) (44): This structure is used to position the holographic pinsetter on the lane required. A motor (34), which is controlled by the control unit (33)(15), which in turn is controlled by the user, transports the system onto the lane chosen. The tracks are secured to the ceiling by rigid supports (62), making the structure static. The screen (19), the control unit (8), the camera (1), the camera (3), the RF reader (4) and the speakers (2) are fitted to the structure (68) of a dolly (66)(40) that runs along these tracks. The dolly (66)(40) is driven in both directions by a motor (34) by means of the cable (64) arranged at the end of the track that is operated by a control unit (33), which 30 performs the commands given by the user. A position sensor (60) provides information on where the dolly is actually situated using some reflex reflectors (61) (67) positioned on the support brackets (62) of the tracks. This is needed to be able to stop the servo-motor (34) of the dolly when it has reached the position required.
  • Foul Photocells: If the bowler steps over the foul line (20), this is detected by the unit (8) that analyses the picture sent by camera (1). Another method of detecting the foul is carried out by more expensive photocells. The control unit (33)(15) receives information from an infrared photocell (31) (32). This photocell, by means of a reflex reflector (41) (42), sends electrical inputs to the unit (33) (15), which sends the information to the unit (8) (43). If the duration of this electrical input is long enough, it means that the bowler has stepped over the foul line, otherwise it means that it is just the ball that has crossed the photocell.
  • Track control unit (33) (15): This unit receives commands from the user via the keyboard. The user commands the control unit (33) of the tracks (44), informing them on which lane the mobile holographic pinsetter is to be positioned. This unit starts the motor (34) that moves the dolly (40) of the mobile unit, in the direction required. This command ends when the unit (33) receives information that the dolly (40) has reached the position required. Information on the position of the dolly is sent to the unit by the position sensor (60) installed on the dolly (66) (40). This sensor (60) is an infrared photocell which, as the reflex reflectors (61) (67), positioned on the supporting brackets (62) of the tracks (63) are crossed, sends a signal to the unit (33). An electrical input is thus sent from the dolly to the unit, which the latter interprets as information on the position. This unit (33) receives information from the weight sensor (14), from the foul sensors (31)(32) and from the dimensional sensors (26)(39) of all the lanes and saves all the information and transmits it to the control unit of the holographic pinsetter (43)(8). This unit (33) therefore has the task of receiving the data from the fixed sensors of which each lane is equipped with one for each type and that of sending the data to the mobile unit (43)(8).To be pointed out is that if we have a solution without tracks and therefore a permanent replacement of the mechanical pinsetter, the functions performed by this unit (33)(15) are carried out by unit (8)(43). The connections to the weight sensors (14), to the foul sensors (31)(32) and to the dimensional sensors (26) are accomplished through unit (8)(43). The track system having the task of moving the holographic pinsetter onto the lane required is no longer of any use, considering that each lane has a dedicated and fixed holographic pinsetter.

BEST MODE FOR CARRYING OUT THE INVENTION

As described below.

Claims

1-43. (canceled)

44. System that offers the visual and sonorous perception of virtual objects interacting dynamically and in real time with real objects having known physical dimensions, characterised by a physically and geometrically known surface on which the real objects may move, featuring a transparent holographic or almost transparent screen set between the real objects and the bowler with the function of showing the bowler virtual perspective objects interacting in real time with real objects and simultaneously allowing the bowler to see the real objects situated on the part of surface that he sees through the transparent holographic screen so that he sees the virtual objects as real, characterised by a projection system that projects a sequence of images showing virtual objects on the holographic screen dynamically interacting with real objects, characterised by a system that detects the position of the real objects, characterised by a system that re-creates sound effects that simulate the sounds produced when the virtual objects collide with the real objects and when the virtual objects collide with other virtual objects in real time, characterised by a control unit that processes the signals of the sensors working in real time and using a physical mathematical model that reproduces the real situation with the aim to generate and send the sequence of images from the projector to the screen and to generate and send the signals to the system to create the sound effects.

45. System that offers the visual and sonorous perception of virtual objects interacting dynamically and in real time with real objects having known physical dimensions, characterised by a physically and geometrically known surface on which the real objects may move, featuring a transparent holographic or almost transparent screen set between the real objects and the bowler with the function of showing the bowler virtual perspective objects interacting in real time with real objects and simultaneously allowing the bowler to see the real objects situated on the part of surface that he sees through the transparent holographic screen so that he sees the virtual objects as real, characterised by a projection system that projects a sequence of images showing virtual objects on the holographic screen dynamically interacting with real objects, characterised by a system that detects the position of the real objects, characterised by a system that re-creates sound effects that simulate the sounds produced when the virtual objects collide with the real objects and when the virtual objects collide with other virtual objects in real time, characterised by a control unit that processes the signals of the sensors working in real time and using a physical mathematical model that reproduces the real situation with the aim to generate and send the sequence of images from the projector to the screen and to generate and send the signals to the system to create the sound effects, characterised by a system that detects the position in the space of the bowler with the aim to use this point as the perspective point in generating the images to be projected on the holographic screen to provide a realistic perspective view on the screen.

46. System that offers the visual and sonorous perception of virtual objects interacting dynamically and in real time with real objects having known physical dimensions, characterised by a physically and geometrically known surface on which the real objects may move, featuring a transparent holographic or almost transparent screen set between the real objects and the bowler with the function of showing the bowler virtual perspective objects interacting in real time with real objects and simultaneously allowing the bowler to see the real objects situated on the part of surface that he sees through the transparent holographic screen so that he sees the virtual objects as real, characterised by a projection system that projects a sequence of images showing virtual objects on the holographic screen dynamically interacting with real objects, characterised by a system that detects the position of the real objects, characterised by a system that re-creates sound effects that simulate the sounds produced when the virtual objects collide with the real objects and when the virtual objects collide with other virtual objects in real time, characterised by a control unit that processes the signals of the sensors working in real time and using a physical mathematical model that reproduces the real situation with the aim to generate and send the sequence of images from the projector to the screen and to generate and send the signals to the system to create the sound effects, characterised by a system that determines the type of object among all those of known physical dimensions, characterised by a system that detects the angular speeds and that transmits them to the control unit that uses them to increase the realism of the dynamic simulation of the virtual objects.

47. System that offers the visual and sonorous perception of virtual objects interacting dynamically and in real time with real objects having known physical dimensions, characterised by a physically and geometrically known surface on which the real objects may move, featuring a transparent holographic or almost transparent screen set between the real objects and the bowler with the function of showing the bowler virtual perspective objects interacting in real time with real objects and simultaneously allowing the bowler to see the real objects situated on the part of surface that he sees through the transparent holographic screen so that he sees the virtual objects as real, characterised by a projection system that projects a sequence of images showing virtual objects on the holographic screen dynamically interacting with real objects, characterised by a system that detects the position of the real objects, characterised by a system that re-creates sound effects that simulate the sounds produced when the virtual objects collide with the real objects and when the virtual objects collide with other virtual objects in real time, characterised by a control unit that processes the signals of the sensors working in real time and using a physical mathematical model that reproduces the real situation with the aim to generate and send the sequence of images from the projector to the screen and to generate and send the signals to the system to create the sound effects, characterised by a system that detects the position in the space of the bowler with the aim to use this point as the perspective point in generating the images to be projected on the holographic screen to provide a realistic perspective view on the screen, characterised by a system that determines the type of object among all those of known physical dimensions, characterised by a system that detects the angular speeds and that transmits them to the control unit that uses them to increase the realism of the dynamic simulation of the virtual objects.

48. Arrangement in accordance with patent claim 44, 45, 46, 47 characterized by a holographic screen that is positioned between the bowler (18) and the position where the virtual objects should be if they were real (9); the screen is able to refract the high intensity rays emitted by the projection system to obtain visible virtual objects and characterized by the fact that it lets through any light of minor intensity without alteration or almost without alteration, being the light typically reflected by the real objects, showing a transparent view of the real objects just as they are.

49. The system of claim 48 includes air screens, fog screens with transparent liquids or gas, holographic crystals screens, made of almost transparent material, screens made of almost transparent fabric or of flexible transparent rubber.

50. In the system of claim 48 the screen is positioned in the point with the smallest perspective error from the point of view of the bowler and in that nearest the position in the space of the first virtual object (9). The screen must not stop the movement of the real objects on the surface where they move.

51. The system of claim 48 includes all types of solid and rigid screens that cannot be crossed by real objects, so the screen is raised off the known movement surface by at least the distance necessary to let the real objects through.

52. The system of claim 48 includes all types of non-solid or non-rigid screens that can be crossed by real objects, which are positioned on the known movement surface in the real point associated unmistakeably with the point of the mathematical model in which the virtual object nearest the bowler is positioned.

53. Arrangement in accordance with patent claim 44, 45, 46, 47 with the function of determining the position of the real objects; the system consists of a camera or an optic system.

54. Arrangement in accordance with patent claim 53 made up of a camera of which the images are processed to analyse the chromatic variations to determine the position of the objects on the known surface. This is determined using the mathematical model of the known surface, the model of the real objects and the physical/geometric/optical model of the camera that sets the images in unmistakeable relation with the real system in order to deduce the speed and the trajectory of the real objects in real time.

55. Arrangement in accordance with patent claim 53 made up of a system of photocells that detect the trajectory and the speed of the real object at a known time in a known point, made up of photocells crossed with known position and direction with the aim to detect the crossing times and to calculate the direction and the linear speed of the objects that cross it based on the times.

56. Arrangement in accordance with patent claim 46,47 with the function of determining the type of real object in order to extract and use the physical data that are saved and that are unmistakeable for each real object of the known set of objects. The system includes a camera or a radio identification system.

57. Arrangement in accordance with patent claim 56 made up of a camera of which the images are analysed in the chromatic variations and based on the comparison of probabilistic methods and chromatic filters and using the mathematical model of the system and of the camera to define the shape, size and superficial chromatic distribution of the object.

58. The information stated in claim 57 is used as an unmistakeable access key to extract the physical dimensions of the object from a limited archive.

59. Arrangement in patent claim 57 with the aim to determine the angular speed of the real objects by analysing the variations in the superficial chromatic distribution of the real object, since their geometric shape and position in the space in relation to the camera are known.

60. Arrangement in accordance with patent claim 56 with the function of determining the type of real object with the aim to extract and use the physical information, known unmistakeably, for each real object and being part of a known set of objects. The system consists of a receiver in radio frequency and of an rfid tag with unmistakeable code applied to the real objects. Each rfid is an unmistakeable access key to an archive of the physical/geometric dimensions of the object to be used to generate the simulation.

61. Arrangement in accordance with patent claim 60 characterized in that there is a coding system that enables each real object to send a code in radio frequency that represents its information in terms of mass, dimensions, type of material, balances, moment of inertia, colour, friction and number of holes.

62. Arrangement in accordance with patent claim 56 made up of a gyroscopic sensor situated in the real object to determine the angular speed and to send this information in radio frequency to a receiver that sends it to the control unit.

63. Arrangement in accordance with patent claim 45, 47 with the function of detecting the position of the bowler in the space with the aim to use this point as a prospective point in generating the sequence of images projected on the screen and to thus obtain a projection that the bowler perceives based on his position in the space. A camera is used to obtain an unmistakeable relationship between the chromatic variations of the camera and the position in the space of the eyes of the bowler. This relationship is obtained via probabilistic comparison methods of the chromatic variations with the typical chromatic variations of human shapes, the knowledge of the surface on which the bowler is standing and using the mathematical model of the camera of which the position and direction in the space are known.

64. Arrangement in accordance with patent claim 44, 45, 46, 47 including a projector used to project the images on the holographic screen.

65. Arrangement in accordance with patent claim 44, 45, 46, 47 including two projectors used to project the images on the holographic screens that can be crossed by the real objects and with the aim to reduce the shadows generated by the real objects as they move.

66. The projectors in patent claim 65 display the sequences of images turned, reflected and distorted to be adapted to where the projectors are positioned so that all the projectors display the same image at the same time on the screen to obtain one single projection on the screen; therefore where the light of one projector is obscured by an object, the other projector completes the image in the shadowed zone.

67. The projectors in patent claim 65 are positioned so that if an object obscures the projector light and creates a shadow on the screen, then the second projector reaches the zone of screen in the shadow with its rays.

68. In accordance with patent claim 67 the projectors are positioned behind the screen, one in the right side and one in the left side and they project on the screen in the bowler's direction.

69. In accordance with patent claim 67 the projectors are positioned, one in front of the screen and one behind it, so that one projects towards the bowler and one projects in the opposite direction.

70. Arrangement in accordance with patent claim 44, 45, 46, 47 characterized by the fact that the control unit (8) receives the data sent from the sensors and produces a physical simulation of the virtual objects, as near as possible to reality, using a mechanical three-dimensional model in real time.

71. Arrangement in accordance with patent claims 44, 45, 46, 47 characterized in that with this screen (19) we can change the properties of the real objects, such as the colour, projecting a virtual object over the real object in real time.

72. Arrangement in accordance with patent claim 44, 45, 46, 47 characterized in that the system acquires (8) and saves the physical parameters and is used to collect useful statistics to improve the bowler's ability; these parameters are transmitted to the control unit.

73. Arrangement in accordance with patent claim 44, 45, 46, 47 characterized in that we can see a replay of the real and virtual objects.

74. Arrangement in accordance with patent claim 44, 45, 46, 47 characterized in that we can modify some of the parameters of the real objects to simplify the game at pleasure.

75. A method for creating an optical and sonorous illusion towards a bowler standing in a limited space and far enough away from the holographic screen to perceive the perspective simulation as credible, to see the virtual objects interacting with the real objects, tracing the movements of known real objects in real time on a known surface, setting a transparent screen between the bowler and the space where the objects move, on which the sequences of images are projected that show virtual objects that simulate the physical, dynamic, optical and spatial behaviour of real objects in real time while the real objects are seen through the transparent screen. The sequence of images generated are generated so that the bowler perceives the holographic objects almost as if there were actually real objects behind the screen interacting dynamically and in real time with the real objects.