The present invention is directed to an interactive theater system with real-time feedback and dynamic special effects. In particular, the present invention is directed to an interactive theater system in which each user is presented with personalized real-time feedback and control over the user's interaction with the displayed video image, such as a video game.
In recent years, video games have become exponentially more popular. Over two billion people currently play video games on phones, video game consoles, personal computers, tablets, etc. This number is expected to increase.
Electronic sports (eSports) and streaming platforms such as Twitch, Livestream, Mixer, etc., have also become increasingly popular. Video game users either compete against each other in person or stream their video game display over the internet to spectators. However, there is little to no interaction between the video game users and/or spectators other than messaging or audio chat. Spectators, in particular, are observers in a manner similar to attending traditional sporting events.
Therefore, there clearly exists a need for an interactive theater system which allows all users to be active participants which provides an engaging personalized experience while still retaining a feeling of group or team interaction.
The interactive theater system of the present invention is primarily executed on an interactive gaming system (hereinafter referred to as an “IG PC”) configured to execute an interactive multi-player video game. The IG PC outputs a video stream to a display system, such as a projector and screen, which allows the players to view the multi-player video game. Each player sits in a multi-sensory seat for providing multi-sensory feedback to each player during execution of the interactive multi-player video game. The player provides input using an input device having an infrared camera and motion sensors. The infrared camera is used to image an infrared light arrangement positioned above-the multi-sensory seats. Infrared camera data and motion sensor data is used by the IG PC to accurately determine the real time position of each input device.
IG PC 102 is responsible for receiving and processing all input from input devices 104, rendering video for projectors or displays 106, and outputting show control data to show controller 108. IG PC 102 may be one or more networked or connected computers or servers in accordance with the requirements and demands of the game currently being executed by interactive theater system 100.
Each projector 106 is preferably a pair of stereoscopic projectors which project two images onto one or more curved or flat projection screens 112. Each user is preferably provided with a pair of 3D glasses so that the display from the projectors can be viewed in 3D. However, it should be apparent to one of ordinary skill in the art that projectors 106 can also be video projectors similar to those utilized in movie theaters. In other embodiments, the video output from IG PC 102 can also be shown on one or more active display screens or devices, such as LCD or OLED displays, lenticular displays, light field displays and other autostereoscopic devices. In some embodiments, each user may be provided with an HMD which receives video data from IG PC 102.
Input is received from each user of the interactive theater system 100 via one or more input devices 104. Input device 104 preferably has a form factor allowing it to easily be held or manipulated by a user. Input device 104 can comprise any standard input controls known in the art such as buttons, triggers, directional pads, joysticks, gesture tracking systems, computer vision, tablets, audible inputs (e.g. microphones), etc.
Preferably, each input device 104 is coupled to IG PC 102 via a wired connection or a wireless connection. For example, as depicted in
In some embodiments, input device 104 may also receive effect data 114 from IG PC 102. For example, input device 104 may be provided with haptic feedback and/or a speaker to provide individualized effect data 114 to the user in response to their input on input device 104, actions displayed on screen 112, or input from other users in interactive theater system 100. Each screen 112 may be surrounded by decorations or modular panels which depict scenery related to the video game currently being played. For example, if the video game is a shooter set in outer space, the periphery of the screen may be decorated with modular panels having the look of a bridge. The outer periphery of the projector's displayed video could then be designed to blend seamlessly with the modular panels or other decorations. In particular, modular panels or removable decorations allow the theater 400 to easily be repurposed for other video games.
In other embodiments, a second screen may be utilized. The second screen may comprise additional panels, surfaces, flooring or ceiling as a display surface aside from screen 112. A static image, similar to the decoration shown on scenic elements, could then be projected onto the second screen and blended into the main game image shown on screen 112. This allows the viewing area to be increased without requiring constant rendering for the imagery shown on the second screen. In another embodiment, a simple animation (e.g., a looped video) or pre-rendered video could be displayed on the second screen. In another embodiment, the primary game content may be expanded to be displayed on the secondary screen.
IG PC 102 outputs effect data 114 to show controller 108 in synchronization with the video signal output to projectors 106. Effect data 114 may include, but is not limited to, environmental audio, individualized audio, sound effects, digital multiplex (DMX) data, seat control data, and user audio. The environmental audio is output from show controller 108 to an audio digital signal processor (DSP) 116. The audio DSP 116 processes the received environmental audio and outputs it to theater environmental audio system 118.
Theater environmental audio system 118 preferably comprises a plurality of speakers and subwoofers arranged to provide immersive audio and surround sound to the users of interactive theater system 100. The number of speakers and subwoofers and their layout in theater environmental audio system 118 is determined based on the theater configuration as is known in the art.
Show controller 108 outputs the DMX data to lighting system 120. Lighting system 120 may comprise any desired combination of known lighting types such as PAR lights, strip lights, strobe lights, spotlights, moving lights, etc. in accordance with the requirements of the game or program executed by interactive theater system 100 and the configuration of the theater.
Show controller 108 further outputs seat control data and user audio directly to each multi-sensory seat 122. As will be explained in more detail later, each multi-sensory seat 122 provides individualized sensory effects to each user based on their operation of their assigned input device 104, as well as the resultant output of the game to create an individualized and immersive experience for each user.
Each multi-sensory seat 122 is preferably coupled to the show controller 108 via seat switch 124 via an Ethernet connection (e.g., CAT 6). This allows a single cable to simultaneously carry the seat control data and the user audio.
Each multi-sensory seat 122 preferably comprises a reader 126 capable of recognizing a user. For example, the reader 126 may be a NFC reader or RFID reader capable of reading a tag carried by the user sitting in the multi-sensory seat 122 which can be used to identify the user.
Thus, interactive theater system 100 allows for a fully interactive real-time system that gives every user agency to direct the outcome of the action and even affect the narrative unfolding before them on a massive, cinema-style screen 112 with professional level audio and effects.
Preferably, each component of interactive theater system 100 is individually addressable so that control signals can be provided to each component individually. IG PC 102 is able to send separate control signals to projectors 106, input devices 104, theater environmental audio system 118, lighting system 120, and multi-sensory seats 122. The address for each component of interactive theater system 100 is maintained in a database by interactive theater system 100. This allows a programmer of the video game a high level of customization because individual control signals can be sent to each component. This also allows the various components of interactive theater system 100 to be grouped together and provided with grouped control signals in addition to individualized control signals. For example, a plurality of input devices 104 may be assigned a group such that the input devices receive the same haptic feedback in addition to individualized haptic feedback for each user.
This also allows interactive theater system 100 to group any components of interactive theater system 100 into groups of 1 (individual components) up until a group encompassing the every component (e.g., for the entire theater). The groupings can also be rearranged or reevaluated in real time. For example, if a particular user's character dies in a video game, another user in the theater (or another remote theater) may be added to the group. The developer of the video game can thus take advantage of the individual and group control of interactive theater system 100 to provide an experience not possible with other systems that do not have the ability to assign groupings to components.
Referring next to
Control panel 204 can also be utilized to override or independently control lighting system 120 or theater environmental audio system 118 via show controller 108. Control panel 204 can also be utilized to control show door access, show action effects, e-stop, or any other systems of interactive theater system 100.
In the depicted embodiment, the audio DSP 116 processes the received audio signal and outputs it to theater environmental audio system 118. The signal for each audio component is preferably passed through audio amplifier 208 before being output to loudspeakers 210 (1X), subwoofers 212 (2X), side surround speakers 214 (4X), or rear surround speakers 216 (4X). As already explained, each audio component of interactive theater system 100 is individually addressable and separate control signals can be sent by IG PC 102 to any component of environmental audio system 118 in any grouping.
Each multi-sensory seat 122 may be coupled to a compressed air system 218 for providing a forced air effect to each seat (neck, legs, arms, etc.). Each multi-sensory seat 122 is capable of controlling the received compressed air in accordance with the effect data received from show controller 108. The effect data may be for system wide effects, effects for particular groupings of components, or for individual components.
This figure illustrates example groupings that may be employed by interactive theater system 100. For example, the input devices 104 and multi-sensory seats 122 of bank 1 may be assigned a different group than that of bank 2. The users of bank 1 would receive effect data 114 that are for every seat and other event data 114 for each individual user of bank 1. The shared effect data 114 for bank 1 provides the users with a shared experience while still allowing for each input device 104 and multi-sensory seat 122 to be provided with additional customized feedback.
The screen 112 may be curved or flat. In some embodiments, a plurality of curved screens 112 can be provided to form a partial or full hemisphere or dome display in the theater. In such embodiments, the arrangement of multi-sensory seats 122 could be arranged in an arc or circular configuration to take advantage of the increased display area (e.g., each bank could be assigned to a particular viewing region). In another embodiment, screens and multi-sensory seats can be arranged in an opposing configuration to provide versus, co-op and other gameplay styles.
A plurality of multi-sensory seats 122 are arranged in banks A-F. Each multi-sensory seat 122 has one or more input device(s) 104. The rear set of banks A-C may be positioned higher than banks D-F so that each user has a clear and unobstructed view of screen 112 as depicted in
Directly above the banks A-F is an arrangement of infrared (IR) lights 402 suspended from or integrated in the ceiling of theater 400. In some embodiments, each input device 104 comprises one or more infrared cameras configured to image the infrared light arrangement 402. The infrared camera data can then be used by IG PC 102 to pinpoint the exact spatial location of each input device 104 in the theater in real-time. Further, the received infrared camera data can also be used to identify the orientation of each input device 104, with or without additional positional information received from input device 104.
Other methods of tracking input devices 104 may also be employed. For example, an array of cameras (infrared or visible light) may be arranged throughout the theater and each input device 104 may comprise one or more markers. By imaging the markers on the input device 104, the location of each input device 104 can be tracked in real-time. Each input device 104 may further comprise a plurality of sensors such as an accelerometer, gyroscopic sensor, etc. which can be used to estimate the location of input devices 104 if one or more of the markers is temporarily occluded.
Further, because the asymmetric pattern can easily be changed, each theater 400 can be assigned a unique asymmetric pattern if needed. This would aid in the identification of the theater 400 in environments where input devices 104 are wireless and potentially used in multiple theaters 400.
It should be obvious to one of ordinary skill in the art that any type of tracking system can be utilized in connection to interactive theater system 100 as long as it is able to provide the real-time location of the input devices 104 within the theater. This allows interactive theater system 100 to easily be upgraded as new tracking technologies are developed because interactive theater system 100 only requires the location of the input device 104 to function and to provide the appropriate feedback.
Each multi-sensory seat 122 preferably comprises an input device holder 802 for holding input device 104 when not in use. An example input device holder 802 is depicted in
In particular, IR light arrangement 402 allows interactive theater system 100 to provide a unique, advanced targeting system providing accurate and persistent positional tracking that not only detects where users aim but how they move their controller. Because the tracking is very accurate, the input device can be used as multiple different in-game instruments such as a hammer, wand, shield, sword, gun, etc., in accordance with the story or the user's positioning of the device.
As already described, IG PC 102 is constantly aware of the location and position of each input device 104 being used in theater 400 (i.e., tracking 6DOF data). This allows the IG PC 102 to render the output of each user as if it is coming from their seat location and being directed toward the screen 112. For example, if a user used input device 104 as a rocket launcher, audio could be provided to multi-sensory seat 122 so that it sounds like the rocket originates from the user's seat. Further, IG PC 102 can adjust the trajectory of the rocket as it enters the virtual space shown on the display such that it appears as if the rocket originated at the user's seat instead of the center of the screen as occurs in most traditional shooters. This adds to the sense of realism and immersion for the user.
Other advanced effects are also possible using input device 104 because of its integrated motion sensors. For example, in the same scenario described above, a user could alter the course of the rocket left/right by moving the input device left/right or curving its position in space to represent the desired trajectory. Using this method, the user could hit enemies behind pillars/posts.
Thus, interactive theater system 100 provides an interactive 3D system that only requires 3D glasses for immersion. This allows users to be more aware of the actions and expressions of other users, which increases the feeling of comradery between players while still providing a tailored experience for each user.
The multiple input devices 104 can also be utilized to perform group actions since IG PC 102 has access to the real time movement of each. For example, if a portion of a video game required that a spaceship turn to avoid a collision, a certain predetermined number of users may be required to perform a certain action (e.g., move the input device 104 or rotate it left) to avoid the collision. Thus, input devices 104 can also be used to determine the outcome of group actions in the video game. As already explained, this feature is possible because each input device 104 is separately addressable and can be assigned to groups by IG PC 102.
Interactive theater system 100 can also track how input devices 104 are used or what percentage of people performed group actions and in what manner. This can be used to adjust future sensitivity required. For example, users may be required to tilt the input device 104 more or less to perform certain actions. This can be used to adjust sensitivity for new players or increase it for advanced players. In a gaming session with many participants, this adjustability can be used as a handicap to level the playing field between players. Further, interactive theater system 100 may have machine learning or AI capabilities to dynamically adjust the sensitivity for each player on the fly according to their strengths and weaknesses.
In some embodiments, users may register with interactive theater system 100 in advance and maintain an active user profile. When a user arrives at interactive theater system 100, they enter their information at kiosk 1102 which loads their profile from user database 1104. This allows each user to configure details about their profile or character ahead of time. For example, a user may choose a certain weapon type in advance for use with input device 104. This would then be loaded when the user checks in using kiosk 1102.
The user can also use their user profile in conjunction with other external applications 1106. For example, the interactive theater system 100 may be associated with a particular smartphone app 1106 or PWA app 1108. The user could interact with the smartphone app as their character and acquire unique weapons or other perks which could then be used in the interactive theater system 100 for particular games. Further, the user profile could be used by the user to check their current rankings, status, etc. Exemplary user profile systems compatible with the present invention are disclosed in U.S. Pat. No. 10,857,467, issued Dec. 8, 2020 and U.S. patent application Ser. No. 16/897,181, filed Jun. 9, 2020, the entire contents of which are hereby incorporated by reference in their entirety.
When the user enters the theater, the reader 126 on multi-sensory seat is used to identify the user sitting in the seat and any information from their user profile is loaded. The user's own cell phone with an NFC chip may be used to identify the user or a tag may be provided (e.g., carried on the user's keychain). It should be obvious that any type of technology capable of reading/identifying a user may be employed by interactive theater system 100 (e.g., card reader, RFID reader, NFC reader, barcode scanner, etc.).
However, it should be apparent that interactive theater system 100 can be used by users without user profiles. Those users would instead use kiosk 1102 to register and then be assigned a multi-sensory seat 122 for a gaming session.
SITE 1 may further incorporate a leaderboard 1110 to keep track of user and team statistics daily, weekly, etc. Or, in a tournament mode, leaderboard 1110 can be utilized to track the progress of each team or individual through the tournament. The operator can even use the microphone to act as an announcer or real time commentary, similar to commentators in traditional sporting events.
This application claims priority to U.S. Provisional Application Ser. No. 63/150,414, filed Feb. 17, 2021, the entire contents of which are hereby incorporated by reference in their entirety.
Number | Date | Country | |
---|---|---|---|
63150414 | Feb 2021 | US |