Patent Application
20070160960
The present invention relates to a system and method for determining the actual coordinates of a projectile impact. Particularly, the invention is directed to firearms and weapons training systems.
Military personnel, police and other law enforcement officers, hunters, sportsmen and especially ordinary citizens need extensive training prior to handling weapons or firearms. When training military and law enforcement personnel, in particular, it is also important for the training systems to employ live weapons and for the immediate conditions to mimic or simulate real life conditions. In real-life situations, these personnel have very little reaction time to respond to multiple stimuli. A bullet or projectile that accurately hits its intended target may reduce, or even eliminate, collateral civilian and property losses. Interactive training systems, which aid in improving shot accuracy, have become very popular. To simulate realistic conditions any such training system must also provide multiple true-to-life scenarios without artificially enforced interruptions to identify the impact location.
Current training systems use a simulated weapon firing a simulated projectile at traditional or virtual targets. The targets are then imaged on a video projection screen. The location of a projectile impact is determined visually or is roughly estimated. These simulators use a beam of light to simulate the projectile and the path of the projectile. The light beam is a narrowly focused beam of visible light or near infrared light, such as those wavelengths produced by low energy laser diodes, which can then be imaged by conventional video cameras or imagers. Sometimes a filter is used to enhance the ability of these cameras to discern the normal reflected light and the light from the simulated projectile. These simulators do not allow for the use of live projectiles, such as bullets. Live projectiles can be used in shooting ranges with virtual targets projected on the backstop or targeting screen. The hit or impact locations can, be determined, however, the shooter has to constantly stop to gauge shot accuracy.
Targets are typically made of paper, plastic, cardboard, polystyrene, wood and other tangible materials. Softer materials, such as paper, allow for easy monitoring of impact location as shown by the hole created in the material, but the projectiles quickly destroy these materials. Metal targets are more durable, however, their intrinsic hardness creates difficulty in determining the actual impact location. Self-healing elastomeric materials, like rubber, fall somewhere in between—they are more durable than the softer materials, but determining the exact impact coordinates is not very easy. Training simulators were developed to simulate continuous action and overcome some of the disadvantages associated with shooting at traditional targets. However, these simulators require the use of simulated weapons. Simulated weapons do not accurately convey the feel and recoil action of firearms. Trainees, not used to extensive target practice with live firearms, may be disadvantaged when required to handle firearms in combat situations. Current training simulators use technology that limits realism and the ability for through performance measurement.
A variety of methods have been disclosed in the prior art to detect the impact location of live projectiles. Most of these methods require direct or visual inspection by the shooter or trainee. Prior art methods detect holes, cold spots, spots of light or supersonic waves. Other methods calculate trajectories or monitor changes in electrical properties at the impact zone in order to estimate the impact location. The impact location of a projectile can be determined directly by locating the point of impact or penetration visually on the target itself. For example, paper or cardboard targets would show a hole in the target corresponding to the location of penetration of the projectile. Metal targets may show a hole, indentation, or surface mark where the projectile impacted or penetrated. These methods have limitations. They may only be used a limited number of times before the target is destroyed. If they are impacted multiple times, it becomes difficult to determine which shots correspond to which hole. To observe the target holes from a distance, telescopic optical means must be employed by the user or a spotter to detect hit location. To directly observe the impact location, the target must be observed up close, by approaching the target, or by mechanically retrieving the target. This requires stopping the training and increases the safety risk of the trainee. Furthermore, all systems using a fixed target are limited in size and maneuverability either in side-to-side motion or in front to back motion. In order to get around these limitations, several alternative methods have been suggested in the prior art to detect impact location of a projectile on a target without having to observe the target at close range. These methods include employing a backlit screen which, when penetrated by a projectile, shows a bright spot from the backlight, using acoustic sensors which detect the shock wave from the passing projectile, or using thermal means of heating the target to a uniform temperature and then looking for cold holes left by the penetrating projectile.
However, these methods only estimate impact coordinates. And, the fixed targets used in these training methods possess limited maneuverability. Finally, the trainee does not get to realistically experience the possible after effects of a projectile impact.
This invention relates to a system and method for calculating the actual pixel coordinates of a projectile launched from a projectile launching device, such as a firearm. In one embodiment, a sensor is used to capture images of the energy changes, or spikes, across a planar surface. The planar surface comprises one or more screens capable of displaying one or more targets. In this embodiment, the screen comprises a self-healing, elastomeric material. The targets can comprise live video, computer graphics, digital animation, three-dimensional images, two-dimensional images, virtual targets and moving targets. When a projectile impacts or penetrates the one or more screens, the sensor registers the impact by virtue of a corresponding change in energy across the screen surface. In one embodiment, the sensor is a thermal camera.
The sensor is connected to a computer. The system is calibrated such that computer has enough information to translate coordinates from a three-dimensional plane defined by the target to logical virtual screen coordinates that can be used by the computer's operating system. The computer further comprises software to calculate the exact pixel coordinates of the projectile impact from the logical virtual screen coordinates. Once the pixel coordinates have been calculated, the computer relays this information to the trainee using feedback mechanisms comprising a projector, monitor or any other electronic device capable of receiving and visually or graphically displaying this information. The process of calculating the impact coordinates and relaying the information back to the trainee is limited only by the computer's processing speed, and the process is virtually instantaneous.
In another embodiment, the system comprises a device such as a video player capable of recording and playing back true-to-life simulated training scenarios. The computer transmits information about the impact coordinates to the video player. The video player selects a scenario that depicts the after-effects or outcome of a projectile accurately hitting, nearly hitting or missing the target. The scenarios can be projected on to a screen or displayed on a monitor or any other feedback device.
The invention does not involve detecting holes or damage to the target to determine impact location. Nor is the impact estimated from a determination of the projectile trajectory. Sensors comprising image sensors or thermal sensors are used to detect an impact based on changes in energy at the screen surface. In another embodiment, the sensor comprises software to isolate thermal images of a projectile impacting the screen surface from continually captured thermal images of the screen surface. The isolated thermal images are sent to a computer attached to the sensor. The computer receives these coordinates as mouse clicks. The computer can calculate actual projectile impact coordinates, relative to a projected target on the screen surface, from the impact images transmitted by the sensor.
The invention can also be adapted to assist users of other types of projectile launchers such as bows, crossbows, spears, darts, balls, rocket launchers or other projectile launching devices, by detecting the heat energy transferred to the target upon impact or penetration.
This combination of accurately measuring the impact coordinates and conveying potential outcomes using training scenarios, aids in creating a realistic training experience. The invention improves the effectiveness and realism for training the military, police officers, marksmen, sportsmen or other firearm users, in a simulated environment using real weapons with real ammunition, by detecting the heat transferred to the target upon impact or penetration of the target by the projectile. The invention is effective because the training does not need to be halted to determine the impact location. The realism is improved because the trainee does not have to use a simulated or demilitarized weapon in training. Since actual weapons and ammunitions can be adapted for use with the system, the trainee experiences the sounds, recoil and discharge associated with the trainee's own weapon. The trainee is thus better able to handle real-life situations. The invention allows the trainee to determine the impact location without approaching the target. This aids in safer training because the trainee is not required to be within the range of fire to view where the projectile impacted a target.
The invention comprises a training system and a method to detect actual coordinates of a projectile launched at one or more targets projected onto one or more screens. The one or more targets comprise virtual targets, live video, computer graphics, digital animation, three-dimensional images, two-dimensional images and moving targets for receiving the projectile impact.
Any projectile launching device 1 can be adapted for use with the invention. These devices include chemical or explosive powered devices such as firearms, pneumatic or compressed gas or spring-piston powered devices, elastic or spring tension powered devices, laser guns and bows, and any other device capable of launching projectiles.
Various types of projectiles 2 may be deployed with this invention. The type of projectile used depends on the training requirements. The projectiles comprise bullets, including lead bullets, copper jacketed bullets, steel jacketed bullets, tracer bullets, frangible bullets, plastic bullets, shotgun shot of various sizes and materials, and shotgun slugs. Softair pellets, metal or plastic pellets, metal or plastic BBs, frangible pellets, arrows, spears, darts, stones, balls and hockey pucks, lasers, rockets, missiles, grenades and other objects, now known or later developed, that can leave a heat signature upon impact may be used as projectiles.
The projectiles 2 are launched at one or more screens 3. The screens 3 can be constructed from any of several materials comprising paper, cloth, plastic, metal or rubber. In its preferred embodiment, the screen comprises an elastomeric material such as rubber, vinyl, silicone or plastic. The flexible nature of elastomeric materials allows for various projectile types to impact the material and either bounce off or penetrate the screen while doing minimal damage to the screen. Upon impact 10 or penetration by a projectile 2, certain types of elastomeric materials such as rubber will allow the projectile 2 to open a hole the size of the projectile 2, allow the projectile 2 to pass through the material, and then close back up due to the elastic nature of the material. While the hole is still present in the material, it still presents a relatively smooth surface on the front surface of the screen 3. The front surface 3a of the one or more screens 3 is coated with a white or light colored reflective coating to allow one or more targets to be projected upon it. The back surface of the screen is preferably set up against a bullet trap or ballistic material. The screens 3 are compact and they can be hung on the walls of a shooting range, or inside a containerized shooting range, for instance. The screens 3 may comprise spring roller pull-down models, electrically operated types or the portable models. The screens 3 may be operated with remote controls or may be manually controlled. The screen sizes depend upon the distance between the screen and the projector. In an alternative embodiment, any planar surface that can receive one or more projected images can act as a “screen.” Examples of such surfaces are rock walls, concrete walls, etc.
The projectiles 2 are launched at targets projected on to the screen surface 3a. These projected targets can comprise digital animation, live videos, computer graphics, three-dimensional images, two-dimensional images, moving targets and other pictorial representations. The projected targets further comprise one or more virtual targets for receiving the projectile impact.
As illustrated in
In another embodiment, the sensor 4 comprises a thermal camera. The thermal camera 4 comprises an infrared core that can detect heat across the energy spectrum, including the infrared region of the energy spectrum. In one embodiment the thermal camera 4 comprises a frame rate of at least 30 frames per second to capture images of the energy spike due to the projectile impact. In another embodiment, the thermal camera 4 further comprises a frame rate of at least 60 frames per second. There are several commercially available examples of thermal cameras 4 that can be used with the training system. One such commercial example is the M3000 Thermal Imaging Module manufactured by DRS Nytech Imaging Systems, Inc. The thermal camera 4 contains a software interface manufactured by Lumenera, Inc.
The system further comprises a computer 5 to interpret and analyze the thermal images detected by the sensor. Preferably, the computer comprises 512 MB DDR, 40 GB hard drive capacity and a processing speed of 3 GHz. The computer 5 is connected to the sensor 4 through an USB2 or comparable interface. The computer 5 comprises software to receive the images captured by the sensor 4 by clicking the mouse or as mouse clicks. The computer 4 further comprises distortion calculation software libraries to calculate the actual pixel coordinates 9 of a projectile impact 10. Once the computer calculates the actual pixel coordinates 9, its software programs can digitally illustrate the impact coordinates. These illustrations are digitally transmitted to one or more feedback devices comprising a projector, monitor, printer or any other device capable of receiving digital signals. The computer further comprises software programs that trigger virtual training scenarios 12.
The sensor 4 is calibrated so that the computer 5 connected to the sensor 4 uses only the images relayed by the sensor 4 to determine impact coordinates 9. Calibration also compensates for the distortions produced by the sensor 4 lens and extrinsic factors such as the placement of the sensor 4 relative to the screen 3. The computer 5 can relate the pixel coordinates from a projected target 9 to calibrated logical virtual screen coordinates that can then be used by the computer's 5 operating system to determine actual impact coordinates 9.
The sensor 4 may be placed at an angle to the screen 3, that is, in front of the screen 3 and to the left, directly in front of the screen 3, looking down at the screen 3, etc. The sensor 4 does not have to be able to see the entire projected target. The computer 5 can actually define its own viewable area within the area defined by the screen 3. If the entire projected target is not viewable, then only the viewable areas of the screen 3 are calibrated. But, for instance, if the projected target is on a screen 3 that has borders containing materials that do not reflect light well, a projectile impact 10 in that border space may nevertheless be detected by the sensor 4.
The calculation software can also calculate and compensate for the radial and tangential distortions caused by the sensor lens. To find the coordinates to be used in the distortion calculation software library, the system projects onto the screen 3 an arbitrary number of evenly spaced vertical lines and horizontal lines, one at a time. The system attempts to create these lines so that they encompass the entire projected area. This ensures accuracy in calculating the impact coordinates. If the coordinates cannot be found, then the system adjusts the size, position, and pixel width of the lines until an arbitrary accuracy error percentage threshold is reached.
The system next projects a “black” image onto the screen. The pixel values from the black projected image are subtracted from the pixel values of the vertical projected image and the horizontal projected image. If both images produced by the subtraction contain pixels at the same place and their values are greater than an experimental threshold, their intersection defines one pixel coordinate. After all coordinates have been calculated in this manner, they are stored and processed in the one or more distortion calculation software libraries. The system also captures and stores thermal images comprising information on the baseline temperatures of each logical screen coordinate. When a projectile impacts the screen, energy is transferred to the screen. Thermal images of the screen are continually captured by the sensor and processed against the stored baseline screen images. If the current thermal images show a deviation from the captured thermal images, a projectile impact is registered.
Once the intrinsic parameters of the sensor are known, the extrinsic parameters of the system can be determined. Two vertical lines and two horizontal lines are projected onto the one or more screens, with each line in each set of lines being as far apart as possible. The same process described above is used to determine the intersection between the set of lines. These coordinates are then undistorted using the distortion calculation software library with the parameters found above. This process results in the determination of four undistorted corner coordinates of the projected image.
The corner coordinates and the coordinates contained in the quadrilateral defined by the four corners must also be related to coordinates within the surface area of the screen. A matrix capable of translating each coordinate to satisfy the above condition is created. The matrix is created as follows: The variables required consist of the captured corner coordinates determined above and the “ideal” coordinates defined by the surface area of the screen. Starting with the ideal coordinates, the two-dimensional perspective matrix defined by those coordinates is calculated. The matrix is used to transform the captured coordinates. Next, the deviation between each transformed captured coordinate and the relative ideal coordinate is calculated. This deviation is the absolute value of the difference between each relative X and Y coordinate. Each deviation is added to the appropriate component of the last set of coordinates used to find the perspective matrix. Those coordinates are then used in the next calculation of the perspective matrix, and this process is carried out until an arbitrary combined deviation is reached or a maximum number of iterations have been run.
The logical screen position for each coordinate from a captured image may be determined by “undistorting” it using the distortion calculation software library, and then transforming the undistorted coordinate by the matrix found above. The undistorted and transformed coordinate may be out of bounds of the virtual screen space.
The system further comprises an image-generating device comprising a liquid crystal display (LCD) projector, a digital projector, a digital light processing projector, a rear projection device, or a front projection device. In one embodiment, the system comprises a LCD projector 6. An image is formed on the liquid crystal panel of the LCD projector from a digital signal from the computer 5, for instance. This formed image is then displayed onto the screen 3.
The system further comprises a plurality of training scenarios 12 that aid in skills training. These training scenarios 12 comprise video scenarios, digital animation, two- and three-dimensional pictures and other electronic representations that may be projected onto the one or more screens 3. Depending on the projectile impact coordinates 9, the training scenarios 12 can lead or branch into several possible outcomes beginning from one initial scene. The trainees may pause or replay the completed scene to show the precise impact time and projectile impact coordinates 9 and thereby allow for detailed discussion of the trainee's actions. The training scenarios comprise anticipated real-life situations comprising arrests by law enforcement personnel, investigative scenarios, courthouse scenarios, hostage scenarios and traffic stops. The training scenarios also aid in judging when the use of force may be justified and/or necessary by showing the expected outcomes from a projectile impact 10.
In one embodiment, one or more targets are projected onto the one or more screens 3 or display surfaces using a projection device such as a projector 6 or any another graphics generating device that can project a target or scenario. The targets can comprise virtual targets. A projectile 2 launched from a projectile launching device 1 penetrates or impacts 10 the targets. A calibrated sensor 4 is directed at the one or more screens 3. When a projectile 2 impacts 10 the front surface of the screen 3, an energy spike or change in temperature is detected at the screen surface 3a. The sensor 4 continually captures thermal images of the one or more screens 3. The sensor 4 processes these thermal images against baseline thermal images of the screen surface. The sensor registers an impact when a deviation from the baseline is observed. The sensor 4 then isolates the impact images from the other captured screen images. The isolated impact images are transmitted to the computer 5 connected to the sensor 4. Since the computer 5 only receives images of the actual impact 10, it does not have to process superfluous thermal images of the screen surface in order to detect an impact 10. This greatly improves processing speed. The sensor 4 is calibrated so that the computer 5 is able to detect actual pixel coordinates 9 of the projectile impact 10 relative to the projected target. The computer 5 further comprises software to digitally illustrate the impact coordinates 9. Feedback devices comprising monitors 7, printers 8 or other electronic devices capable of receiving a digital signal from the computer may be used to visually or graphically depict the impact coordinates 9. The impact coordinates 9 may also be projected, using the projector 6 onto the one or more screens 3.
The system further comprises simulated training scenarios 12 that are triggered by the computer 5 upon the calculation of the actual projectile impact coordinates 9. These training scenarios 12 comprise video, digital animation or other virtual compilations of one or more situations that simulate real-life conditions. These situations comprise hostage scenarios, courthouse encounters, traffic stops and terrorist attacks. Each scenario comprises a compilation of one or more scenes. The scenes are compiled in such a manner that any given scene may further branch into one or more scenes based on input from the computer regarding the calculated impact coordinates. The branching simulates expected outcomes in similar real life situations. The impact coordinates 9 may further be superimposed against, say, a graphic of a target's body, and the coordinates “frozen” for the trainee to visually inspect the extent of any deviation from the expected shot location. The training scenarios 12 may also be used to display collateral damage that may be expected in real life situations.
The system further comprises a one or more projectile launching devices comprising laser-triggering devices. These laser-triggering devices may be used to fire one or more projectiles comprising lasers at the screens 3. The system further comprises software to detect the location of the laser device that launched a particular laser at the screens 3.
In yet another embodiment, the system comprises a thermal sensor 4 comprising a thermal camera directed at the one or more screens 3. The thermal camera 4 comprises software to detect and isolate thermal images of the one or more projectile impacting 10 the one or more screens 3. The thermal camera 4 transmits the impact images to a connected computer 5. The computer 5 is connected to the thermal camera 4 through an USB2 or comparable interface. The thermal camera 4 is calibrated so that the attached computer 5 can compute impact coordinates 9 relative to predetermined logical screen coordinates. The impact coordinates 9 are sent to feedback devices comprising projectors 6, printers 8, monitors 7 or other electronic devices capable of receiving a digital signal from the computer 5. The feedback devices can visually or graphically illustrate the impact coordinates. The system further comprises training scenarios 12 that comprise a compilation of imagery comprising video and animation figures. The scenes are compiled to simulate real-life incidents, such as hostage situations and traffic stops, which are encountered by the law enforcement and military personnel. The system comprises software that upon notification of the impact coordinates further branches into one or more possible outcome based scenarios. These outcome-based scenarios simulate real life responses. The system further comprises a video editor. The trainee can film their own video clips and import them into the editor. The imported video is converted into MPEG-4 or comparable format. The trainee can then create scenarios comprising branching points as desired. Branching conditions that are correlated to the coordinates of the projectile impact may also be defined. The trainee may ultimately group multiple scenarios together to present diverse training situations in a single training session.
In another embodiment, the thermal camera 4 continually captures current thermal images of the screen surface 3. The computer 5 connected to the thermal camera 4 receives these thermal images as mouse clicks. The computer 5 processes these images against baseline thermal images of the screen surface. If the computer 5 detects a deviation from the baseline, an impact is registered. The computer 5 further comprises software to calculate the projectile impact coordinates 9 from the impact images. Once the coordinates have been calculated, they are sent to feedback devices connected to the computer 5.
During the method for calculating the actual projectile impact coordinates 9, one or more projectiles 2 are launched at one or more projected targets. A thermal camera 4 is directed at one or more screens 3 comprising the projected targets. The thermal camera 4 continually detects and captures thermal images of the screen surface. The thermal camera 4 registers a projectile impact 10, by comparing current thermal images of the screen surface with previously captured baseline thermal images of the screen. Any deviation from the baseline is attributable to the energy change caused by the projectile impact. The thermal camera 4 isolates the impact images and transmits them to a computer 5. The computer 5 is connected to the thermal camera 4 through a USB2 or comparable interface. The thermal camera 4 is calibrated so that the computer 5 can calculate the actual impact coordinates 9 relative to the projected target. The computer 5 further comprises software to convert the impact coordinates 9 into digital signals. Feedback devices comprising a monitor 7, printer 8 or any other electronic device that can receive a digital signal from the computer 5 can be used to visually or graphically depict the impact coordinates. The impact coordinates can be displayed along a virtual X-axis 10 and a Y-axis 11 projected on the screen surface. A projector 6 may be used to project the impact coordinates images onto the screens 3 for immediate visual feedback to the trainee. Upon notification of the calculated projectile impact coordinates 9 by the computer 5, the software comprising outcome based training scenarios is triggered. These scenarios comprise a compilation of scenes that simulate real life responses or outcomes to a projectile impact. A projector 6 or monitor may further be used to project these scenarios onto the screen 3.
The foregoing description is illustrative and explanatory of several embodiments of the invention, it will by understood by those skilled in the art that various changes and modifications in form, materials and detail may be made therein without departing from the spirit and scope of the invention.
This application claims the benefit of priority to U.S. provisional patent application No. 60/776,002 filed Oct. 21, 2005.
| Number | Date | Country | |
|---|---|---|---|
| 60776002 | Oct 2005 | US |
