Patent Application
20200141698
The present disclosure relates to the data association function in complex multiple target detection and tracking applications and more particularly to processing multiple measurements from an EO/IR camera for multiple targets tracking (MTT) and maintaining their respective track files in a cohesive manner.
The multiple targets tracking (MTT) problem and its data association (DA) algorithms (i.e., nearest neighbor (NN), multiple hypotheses tracking (MHT), or joint probability data association (JPDA)) are well known in the sensors fusion and MTT community. However, MTT solutions suitable for an onboard weapon's guidance and control interactions, especially for closed-loop tracking accuracy needed in order to guide multiple weapons to intercept multiple desired targets in a low cost and computational less intensive manner are rarely available.
Wherefore it is an object of the present disclosure to achieve a closed-loop MTT and track file management system using a practical and simplified implementation approach allowing an on-board Guidance and Control (G&C) subsystem to timely close the respective loops (i.e., weapon/target pairing rate) for successful engagements of multiple targets to overcome the above-mentioned shortcomings and drawbacks associated with the conventional multi-object guidance systems.
It has been recognized that multiple target tracking (MTT) using either a single sensor or multiple sensors to detect and track multiple targets deployed in various domains (i.e., space, air, ground, or maritime) is effective for an Intelligence, Surveillance, and Reconnaissance (ISR) mission. However, for an on-board weapon system, an MTT used as part of the integrated Guidance, Navigation, and Control (GNC) scheme must provide the onboard weapon with the ability to pinpoint what the weapon can see in the context of multiple track files, and determine how the data can be used to assist the weapon to accomplish its end goal which is to acutely pair the tracking such that a projectile is guided to a collision course with an appropriate target via an active target track.
One aspect of the present disclosure is a simple MTT solution suitable as an on-board MTT system capable of producing the MTT functionalities (i.e. measurements to track association and track file management) while offering effective MTT performance without incurring MTT design complexity.
One embodiment of the system provides a self-contained MTT solution per sensor which is directly ready for interfacing and interacting with the on-board GN&C subsystems to achieve a closed loop engagement mission.
One aspect of the present disclosure is an improved tracking, targeting, and guidance system, comprising: at least one sensor installed on a vehicle, wherein the sensor is part of a fire control subsystem (FCS); the sensor having an on-board multiple target detection and tracking (MTT), data association (DA), and track file management (TFM) system installed thereon, wherein the multiple target detection and tracking (MTT), data association (DA), and track file management (TFM) system are configured to direct the following operations to deliver one or more correct track files to a guidance system to complete an engagement: process images from the at least one sensor in real-time; detect one or more target location measurements for one or more targets using the images from the at least one sensor to produce potential target tracks; process the one or more target location measurements to determine if the one or more target location measurements from the at least one sensor are correlated with predicted target tracks via the fire control system, if not, then uncorrelated target tracks are placed in a separate file for possible new target track initiation/creation; associate the potential target tracks via a gating system, wherein potential target tracks falling within a gating threshold are chosen as active target tracks; update and maintain active target tracks as part of a track file management (TFM) system, as target state estimates for individual targets; feed output from the track file management (TFM) system to a weapon target assignment system to calculate a correct acceleration profile to guide a projectile onto a collision course with the one or more targets by pairing the correct active target track with one or more targets.
One embodiment of the improved tracking, targeting, and guidance system is wherein the sensor is an EO/IR camera. The vehicle can be a land based vehicle such as a tank, armored carrier, and the like. In a further example the vehicle is sea based such as a frigate, destroyer, cruiser amphibious vehicle, submarine and the like. The vehicle in a further example is air based such as an unmanned aerial vehicle (UAV), drone, helicopter, fighter aircraft and the like.
Another embodiment of the improved tracking, targeting, and guidance system is wherein the target is on the ground. In some cases, the target is in the air.
Yet another embodiment of the improved tracking, targeting, and guidance system is wherein the uncorrelated target tracks are declared as clutters and no new track is initiated or created if the new measurements do not persist across continuous samples.
Another embodiment of the improved tracking, targeting, and guidance system is wherein if the potential target tracks do not satisfy the gating criterion they are rejected in real time as part of a clutter rejection mechanism. In some cases, if target tracks created in the track file management (TFM) system have not received continuous measurement updates for more than three consecutive samples, the tracks are deleted.
Still yet another embodiment of the improved tracking, targeting, and guidance system is wherein an active track file management (TFM) system contains all active tracks.
Another aspect to the present disclosure is a method of data association in a multi-projectile/multi target system, comprising: processing images from at least one sensor mounted on a vehicle in real-time, wherein the sensor is part of a fire control subsystem (FCS), the at least one sensor having an on-board multiple target detection and tracking (MTT), data association (DA), and track file management (TFM) system installed thereon, wherein the multiple target detection and tracking (MTT), data association (DA), and track file management (TFM) system are configured to direct the following operations to deliver one or more correct track files to a guidance system to complete an engagement; detecting one or more target location measurements for one or more targets using the images from the at least one sensor to produce potential target tracks; processing the one or more target location measurements to determine if the one or more target location measurements from the at least one sensor are correlated with predicted target tracks via the fire control system, if not, then uncorrelated target tracks are placed in a separate file for possible new target track initiation/creation; associating the potential target tracks via a gating system, wherein the potential target tracks that fall within a gating threshold are chosen as active target tracks; updating and maintaining the active target tracks as part of a track file management (TFM) system, as target state estimates for individual targets; feeding output from the track file management (TFM) system to a weapon target assignment system; pairing an active target track with a correct target; and calculating via a guidance system a correct acceleration profile to guide a projectile onto a collision course with the correct target.
One embodiment of the method of data association in a multi-projectile/multi target system is wherein the at least one sensor is an EO/IR camera. In some cases, the vehicle is a tank.
Another embodiment of the method of data association in a multi-projectile/multi target system is wherein the target is on the ground. In some cases, the fire control system is configured to perform surface-to-surface and/or surface-to-air missions.
Yet another embodiment of the method of data association in a multi-projectile/multi target system is wherein the uncorrelated target tracks are declared as clutters and no new track is initiated or created if the new measurements do not persist across continuous samples.
Still yet another embodiment of the method of data association in a multi-projectile/multi target system is wherein if the potential target tracks do not satisfy the gating threshold they are rejected in real-time as part of a clutter rejection mechanism.
In certain embodiments, if active target tracks created in the track file management (TFM) system have not received continuous measurement updates for more than three consecutive samples, the tracks are deleted.
These aspects of the disclosure are not meant to be exclusive and other features, aspects, and advantages of the present disclosure will be readily apparent to those of ordinary skill in the art when read in conjunction with the following description, appended claims, and accompanying drawings.
The foregoing and other objects, features, and advantages of the disclosure will be apparent from the following description of particular embodiments of the disclosure, as illustrated in the accompanying drawings in which like reference characters refer to the same parts throughout the different views. The drawings are not necessarily to scale, emphasis instead being placed upon illustrating the principles of the disclosure.
In one embodiment of the present disclosure, a system is provided to handle the data association problem caused by multiple measurements collected by multiple sensors (e.g., EO/IR or RF seeker) in a multiple targets/multiple projectiles operating situation. In some cases, multiple guided projectiles are used to address various target types. As used herein, bullets, rockets, missiles and other precision guided munitions are generally considered projectiles. In these applications, it is imperative that the precision guided munitions are associated with the correct objects being tracked. Otherwise, the state estimate of these multiple objects will not be consistently maintained to accurately support a Fire Control System (i.e., supplying the correct state vector to the guidance systems of each projectile in real-time for the effective ordering of engagement with the correct targets).
One aspect of this disclosure is a new data association function that matches the correct measurements to the correct tracks for real-time track update, maintenance, and management, thus solving the data association problem with a high probability of correct matching of measurements to tracks in a closely-spaced, multi-target situation. While many technical publications present the multi-targets/multi-sensors tracking problem; practical designs to accomplish high performance in the context of measurements to track association in the presence of object maneuvering uncertainties and closely spaced objects has not been seen.
The ability to maintain multiple track files in a cohesive manner while supplying their state vectors at a highly reliable and accurate level are important for the situational awareness, threat assessment, and on-board or off-board command guidance subsystems. This allows the projectiles to engage with the correct type of targets/threats based on either in-flight target update or onboard decision-making processing.
The present disclosure offers dynamic track files of respective multiple targets signatures by processing multiple target measurements collected by an EO/IR camera (either mounted on a vehicle or on-board a weapon platform) and sending those track files to projectiles' guidance subsystems to assign targets and complete the engagement process.
It is understood that Precision Guided Munitions (PGMs) need to have a practical track file management system to maintain precise object (i.e., multiple projectile and multiple target) state estimate vectors so that they can effectively and timely support the order of engagement when interfacing with G&C subsystems. Some mainstream multiple target tracking (MTT) algorithms (i.e., Global Nearest Neighbor Filter (GNNF) and its variants, Multiple Hypothesis Tracker (MHT) based design and its variants, Joint Probability Hypothesis Data Association (JPDA) filter and its variants) are deemed to be suitable for an ISR mission but not to be used as an onboard MTT feeding the G&C subsystems the correct track file in a cohesive manner. The table below illustrates the self-contained data association and track file management features of one embodiment of the present disclosure developed for a single EO/IR sensor mounted on a tank detecting and tracking multiple targets (either ground-based or air-based). There, multiple target track files (target state estimates) are provided to the on-board guidance and control subsystems for successful engagements of respective targets assigned to the weapon via the weapon to target assignment (WTA).
The following table represents one embodiment of a top level summary of the main multiple target tracking (MTT) and track file management (TFM) function (MTT_DA_TFM.m).
In certain embodiments of the present disclosure, an effective method of keeping the state estimate vectors of respective multiple targets driving the guidance subsystem for proper order of engagement execution has been demonstrated. In some embodiments, measurements to track association and track file management infrastructure defined in the above table have been effectively carried out for this ground based mission's fire control sensor definition.
Specific applications of this present disclosure are threefold: (1) the MTT system is self-contained for installation on a ground-based vehicle's (e.g., a tank) fire control system (FCS) to operate independently/individually or to operate as a team; (2) the MTT system provides the ground-based FCS precision MTT track files of multiple targets for the guidance subsystem to process to complete the order of engagement; and (3) the MTT system is robust enough to handle EO/IR angle only measurements.
Referring to
Referring to
Still referring to
Referring to
On the left side of
Once the predicted state measurement 44 is completed, it will be used to compare against the multiple targets measurements data stream 47 for respective multiple residuals calculations 46. These multiple residuals (i.e., the individual target state estimation errors, predicted vs current track estimate) from all measurements against all existing tracks will be compared against a selected gating threshold 49 to complete the data association and measurement to track association functions. The residual distance squared d2 48 is used in gating 50, where the gating threshold 49 uses distance squared 48 against each EKF as individual target matching for data association. In one embodiment, the gate threshold Gth 49 there is one satisfied residual (within threshold) used for the state update. The EKF update stage 52 uses a satisfied residual for state update 54. That Updated state is then utilized for the initial state 38 for a subsequent iteration 56.
Still referring to
Referring to
Still referring
Referring to
Referring to
Referring to
The proposed MTT, DA, and TFM software functionalities are integrated into an EO/IR sensor (and applicable to any other active sensors) turning these sensors into plug-and-play units serving multiple missions. Multiple mission include ground-based, sea-based, air-based, or even space-based. These have been practically evaluated using a high fidelity engagement simulation.
Referring to
The computer readable medium as described herein can be a data storage device, or unit such as a magnetic disk, magneto-optical disk, an optical disk, or a flash drive. Further, it will be appreciated that the term “memory” herein is intended to include various types of suitable data storage media, whether permanent or temporary, such as transitory electronic memories, non-transitory computer-readable medium and/or computer-writable medium.
It will be appreciated from the above that the invention may be implemented as computer software, which may be supplied on a storage medium or via a transmission medium such as a local-area network or a wide-area network, such as the Internet. It is to be further understood that, because some of the constituent system components and method steps depicted in the accompanying Figures can be implemented in software, the actual connections between the systems components (or the process steps) may differ depending upon the manner in which the present invention is programmed. Given the teachings of the present invention provided herein, one of ordinary skill in the related art will be able to contemplate these and similar implementations or configurations of the present invention.
It is to be understood that the present invention can be implemented in various forms of hardware, software, firmware, special purpose processes, or a combination thereof. In one embodiment, the present invention can be auto-coded and embedded into an advanced chip based processor or FPGA to address a large target population for a single integrated picture compilation serving situational awareness purpose for traffic monitoring and alert actions implemented in software as an application program tangible embodied on a computer readable program storage device. The software can be used to serve ground, air, or space traffic monitoring and management system with appropriate modifications. The application program can be uploaded to, and executed by, a machine comprising any suitable architecture.
While various embodiments of the present invention have been described in detail, it is apparent that various modifications and alterations of those embodiments will occur to and be readily apparent to those skilled in the art. However, it is to be expressly understood that such modifications and alterations are within the scope and spirit of the present invention, as set forth in the appended claims. Further, the invention(s) described herein is capable of other embodiments and of being practiced or of being carried out in various other related ways. In addition, it is to be understood that the phraseology and terminology used herein is for the purpose of description and should not be regarded as limiting. The use of “including,” “comprising,” or “having,” and variations thereof herein, is meant to encompass the items listed thereafter and equivalents thereof as well as additional items while only the terms “consisting of” and “consisting only of” are to be construed in a limitative sense.
The foregoing description of the embodiments of the present disclosure has been presented for the purposes of illustration and description. It is not intended to be exhaustive or to limit the present disclosure to the precise form disclosed. Many modifications and variations are possible in light of this disclosure. It is intended that the scope of the present disclosure be limited not by this detailed description, but rather by the claims appended hereto.
A number of implementations have been described. Nevertheless, it will be understood that various modifications may be made without departing from the scope of the disclosure. Although operations are depicted in the drawings in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed, to achieve desirable results.
While the principles of the disclosure have been described herein, it is to be understood by those skilled in the art that this description is made only by way of example and not as a limitation as to the scope of the disclosure. Other embodiments are contemplated within the scope of the present disclosure in addition to the exemplary embodiments shown and described herein. Modifications and substitutions by one of ordinary skill in the art are considered to be within the scope of the present disclosure.
